If语句条件满足但未执行（Python）

Question

您好，我有一个windows路径对象列表，我正在这些对象上运行if语句。背景：我有几个csv文件。我的代码检查这些csv文件。如果csv文件良好，脚本会将文件移动到名为“存档”的目录中。如果有错误，则将其移到“错误”，如果为空，则将移到“空”

所以我有一个文件被转移到了档案室。我将此文件复制回base dir，以便脚本处理它但是，应该捕获此副本的if语句不会执行，而是脚本尝试将文件移动到存档目录。发生这种情况时，由于我使用Path.rename（）方法移动文件，因此会出现以下错误： FileExistError:[WinError 183]无法在文件已存在时创建该文件：“C:\Users\sys\u nsgprobeingestio\Documents\dozie\odfs\odfshistory\06_17_2020_FMGN520.csv”->；”C:\Users\sys\u nsgprobeangestio\Documents\dozie\odfs\odfshistory\archive\06\u 17\u 2020\u FMGN520.csv'

这些就是所涉及的功能。有人知道为什么会这样吗

def make_dict_of_csvprocessing_dirs():
    dir_dict = process_dirconfig_file(dirconfig_file)
    # print(dir_dict)
    dictofpdir_flist = {} #dictionary of lists of files in different processing dirs
    csvbase_file_dir = dir_dict["base_dir"]
    csvhistory_Phandler = Path(csvbase_file_dir)
    csvbase_path_list = [file for file in csvhistory_Phandler.glob("*.*")]
    dictofpdir_flist["csvbase_path_list"] = csvbase_path_list

    archive_dir = dir_dict["archive_dir"]
    archive_Phandler = Path(archive_dir)
    archivefiles_path_set = {file for file in archive_Phandler.rglob("*.*")}
    dictofpdir_flist["archivefiles_path_set"] = archivefiles_path_set

发生错误的函数：

def odf_history_from_csv_to_dbtable(db_instance):
    odfsdict = db_instance['odfs_tester_history']
    #table_row = {}
    totalresult_list = []

    dir_dict, dictofpdir_flist = make_dict_of_csvprocessing_dirs()
    print(dir_dict)
    csvbase_path_list = dictofpdir_flist["csvbase_path_list"]
    archivefiles_path_set = dictofpdir_flist["archivefiles_path_set"]

    for csv in csvbase_path_list:  # is there a faster way to compare the list of files in archive and history?
        if csv in archivefiles_path_set:
            print(csv.name + " is in archive folder already")
        else:
            csvhistoryfilelist_to_dbtable(csv, db_instance)
            df_tuple = process_csv_formatting(csv)
            df_cnum, odfscsv_df = df_tuple
            if df_cnum == 1:
                trg_path = Path(dir_dict['empty_dir'])
                csv.rename(trg_path.joinpath(csv.name))

    return totalresult_list

当我调试Pycharm时，它会给我以下值：注意目录列表的刻度是如何颠倒的。我想知道这是否是问题所在？：

archivefiles_path_set={WindowsPath('C:/Users/sys_nsgprobeingestio/Documents/dozie/odfs/odfshistory/archive/06_17_2020_FMGN520.csv')}

csv = {WindowsPath}C:\Users\sys_nsgprobeingestio\Documents\dozie\odfs\odfshistory\06_17_2020_FMGN520.csv

csvbase_path_list = 
[WindowsPath('C:/Users/sys_nsgprobeingestio/Documents/dozie/odfs/odfshistory/06_17_2020_FMGN520.csv')]

Answer 1

获取要复制的文件的最快方法（如果您是访问两个目录的唯一进程）：

from os import listdir 

basedir = r"c:/temp/csvs"
archdir = os.path.join(basedir,"temp")

def what_to_copy(frm_dir, to_dir):
    return set(os.listdir(frm_dir)).difference(os.listdir(to_dir))

copy_names = what_to_copy(basedir, archdir)
print(copy_names) # you need to prepend the dirs when copying, use os.path.join

看起来你的代码相当复杂（许多存储在dict中的东西需要传输才能再次取出），只需要一点点任务。这就是它的工作原理：

import os

# boiler plate code to create files and make some of them already "archived"
names = [ f"file_{i}.csv" for i in range(10,60)]
basedir = r"c:/temp/csvs"
archdir = os.path.join(basedir,"temp")

os.makedirs(basedir, exist_ok = True)
os.makedirs(archdir, exist_ok = True)

def create_files():
    for idx, fn in enumerate(names):
        # create all files in basedir
        with open(os.path.join(basedir,fn),"w") as f:
            f.write(" ")
        # every 3rd file goes into archdir as well
        if idx%3 == 0:
            with open(os.path.join(archdir,fn),"w") as f:
                f.write(" ")


create_files()

函数“复制”尚未存在的文件：

def copy_from_to_if_not_exists(frm,to):
    """'frm' full path to file, 'to' directory to copy to"""
    # norm paths so they compare equally regardless of C:/temp or C:\\temp
    frm = os.path.normpath(frm)
    to =  os.path.normpath(to)

    fn  = os.path.basename(frm)
    dir = os.path.dirname(frm)

    if dir != to:
        if fn in os.listdir(to):
            print(fn, " -> already exists!")
        else:
            # you would copy the file instead ...
            print(fn, " -> could be copied")

# print whats in the basedir as well as the archivedir (os.walk descends subdirs)
for root,dirs,files in os.walk(basedir):
    print(root + ":", files, sep="\n")

for file in os.listdir(basedir):
    copy_from_to_if_not_exists(os.path.join(basedir,file),archdir)

如果硬盘驱动器的读缓存优化对您来说不够好，您可以缓存os.listdir(to)的结果，但它可能很好

输出：

c:/temp/csvs:
['file_10.csv','file_11.csv','file_12.csv','file_13.csv','file_14.csv','file_15.csv',
 'file_16.csv','file_17.csv','file_18.csv','file_19.csv','file_20.csv','file_21.csv',
 'file_22.csv','file_23.csv','file_24.csv','file_25.csv','file_26.csv','file_27.csv',
 'file_28.csv','file_29.csv','file_30.csv','file_31.csv','file_32.csv','file_33.csv',
 'file_34.csv','file_35.csv','file_36.csv','file_37.csv','file_38.csv','file_39.csv', 
 'file_40.csv','file_41.csv','file_42.csv','file_43.csv','file_44.csv','file_45.csv',
 'file_46.csv','file_47.csv','file_48.csv','file_49.csv','file_50.csv','file_51.csv', 
 'file_52.csv','file_53.csv','file_54.csv','file_55.csv','file_56.csv','file_57.csv',
 'file_58.csv','file_59.csv']

c:/temp/csvs\temp:
['file_10.csv','file_13.csv','file_16.csv','file_19.csv','file_22.csv','file_25.csv', 
 'file_28.csv','file_31.csv','file_34.csv','file_37.csv','file_40.csv','file_43.csv',
 'file_46.csv','file_49.csv','file_52.csv','file_55.csv','file_58.csv']

file_10.csv  -> already exists!
file_11.csv  -> could be copied
file_12.csv  -> could be copied
file_13.csv  -> already exists!
file_14.csv  -> could be copied
file_15.csv  -> could be copied
file_16.csv  -> already exists!
file_17.csv  -> could be copied
file_18.csv  -> could be copied
[...snipp...]
file_55.csv  -> already exists!
file_56.csv  -> could be copied
file_57.csv  -> could be copied
file_58.csv  -> already exists!
file_59.csv  -> could be copied 

< ^ > {{a1}）用于缓存函数结果的方法，并考虑将^ {< CD2>}放入一个函数中，如果IO读数成为瓶颈（先测量，然后优化），则将结果缓存在