使用文件名从循环中删除多个数据帧

Question

我试图从目录中的原始.csv文件创建多个好/坏文件

我对Python相当陌生，但是已经拼凑了下面的内容，但是它没有保存多个文件，只保存了x1“good”和x1“bad”文件。在目录中，我有testfile1和testfile2。输出应该是testfile1good testfile1bad testfile2good testfile2bad

任何帮助都将不胜感激

谢谢

import pandas as pd
from string import ascii_letters
import glob
from pathlib import Path


files = glob.glob('C:\\Users\\nickn\\OneDrive\\Documents\\Well\\*.csv')


for f in files:
    filename = []
    filename = Path(f)

#Can not be null fields    
df = pd.read_csv(f)
emptyvals = []
emptyvals = df['First Name'].isnull() | df['Last Name'].isnull()

#Bank Account Number is not 8 digits long
accountnolen = []
ac = []
accountnolen = df['AccNumLen'] = df['Bank Account Number'].astype(str).map(len)
ac =  df[(df['AccNumLen'] != 8)]
acd= ac.drop(['AccNumLen'],axis=1)

#Create Exclusions
allexclusions = []
allexclusions = df[emptyvals].append(acd)
allexclusions.to_csv(filename.stem+"bad.csv",header =True,index=False)

#GoodList
#for f in files:
#    filename = []
#    filename = Path(f)
origlist = df
df = pd.merge(origlist, allexclusions, how='outer', indicator=True)
cl =  df[(df['_merge'] == 'left_only')]
cld = cl.drop(['_merge','AccNumLen'],axis=1)
cld['Well ID'] = cld['Well ID'].str.rstrip(ascii_letters)

cld.to_csv(filename.stem+'good.csv',header =True,index=False)

Answer 1

我认为您可以循环，但将其保留，然后在第14行执行其余的操作-在那里您设置了文件名，并保存了一次数据

您需要做的是循环，其余的应该在每次迭代中进行，因此代码应该如下所示：

import pandas as pd
from string import ascii_letters
import glob
from pathlib import Path


files = glob.glob('C:\\Users\\nickn\\OneDrive\\Documents\\Well\\*.csv')


for f in files:
    filename = []
    filename = Path(f)

    #EDIT: we stay in loop and process each file one by one with following lines:

    #Can not be null fields    
    df = pd.read_csv(f)
    emptyvals = []
    emptyvals = df['First Name'].isnull() | df['Last Name'].isnull()
    
    #Bank Account Number is not 8 digits long
    accountnolen = []
    ac = []
    accountnolen = df['AccNumLen'] = df['Bank Account Number'].astype(str).map(len)
    ac =  df[(df['AccNumLen'] != 8)]
    acd= ac.drop(['AccNumLen'],axis=1)
    
    #Create Exclusions
    allexclusions = []
    allexclusions = df[emptyvals].append(acd)
    allexclusions.to_csv(filename.stem+"bad.csv",header =True,index=False)
    
    #GoodList
    #for f in files:
    #    filename = []
    #    filename = Path(f)
    origlist = df
    df = pd.merge(origlist, allexclusions, how='outer', indicator=True)
    cl =  df[(df['_merge'] == 'left_only')]
    cld = cl.drop(['_merge','AccNumLen'],axis=1)
    cld['Well ID'] = cld['Well ID'].str.rstrip(ascii_letters)
    
    cld.to_csv(filename.stem+'good.csv',header =True,index=False)

换句话说，您迭代在目录中找到的文件名，然后获取最后一个“文件名”，并一次性处理它。通过向代码的其余部分添加4个空格，我们向python解释器表明，这部分代码是循环的一部分，应该为每个文件执行。希望它有意义