循环Pandas目录

Question

我在一个目录中有许多csv文件，每个目录有两列

miRNA  read_counts  
miR1      10
miR1      5
miR2      2
miR2      3
miR3     100

如果miRNA id是相同的，我想计算read\u计数

结果：

miRNA  read_counts  
miR1      15
miR2      5
miR3     100

为此我写了一个小剧本。然而，我不知道如何循环它通过我所有的csv文件，所以我不必复制粘贴文件名和输出每次。任何帮助都将不胜感激。谢谢你的帮助

import pandas as pd

df = pd.read_csv("modified_LC1a_miRNA_expressed.csv")
df_new = df.groupby('miRNA')['read_count'].sum()
print(df_new)
df_new.to_csv('sum_LC1a_miRNA_expressed.csv')

Answer 1

不是想偷答案。如果我有足够的代表，我会把这个放在@Asif Ali的回答下面的评论里

假设所有输入.csv文件都遵循以下格式： “修改了\u文件名}.csv的\u{rest\u”

您希望输出为： “sum{same\u rest\u of the \u file\u name}.csv”

import os
import glob

path = "./your/path"
files = glob.glob(os.path.join(path, "*.csv"))

for file in files:
    df = pd.read_csv(file)
    df_new = df.groupby('miRNA')['read_count'].sum()
    print(df_new)
    df_new.to_csv(file.split('modified')[:-1] + \
                  'sum' + \
                  '_'.join(file.split('modified')[-1:]))

Answer 2

尝试查看glob模块

from glob import glob
import os

path = "./your/path"
files = glob(os.path.join(path, "*.csv"))

dataframes = []
for file in files:
    df = pd.read_csv(file)
    # rest you would want to append these to dataframes
    dataframes.append(df)

然后，使用pd.concat连接数据帧并执行groupby操作

编辑1: 根据评论中提到的要求：

results = {}
for file in files:
    df = pd.read_csv(file)
    # perform operation
    df_new = df.groupby('miRNA')['read_count'].sum()
    results[file] = df_new