如何分割csv文件和条件？

Question

我有这个csv文件：

89,Network activity,ip-dst,80.179.42.44,,1,20160929
89,Payload delivery,md5,4ad2924ced722ab65ff978f83a40448e,,1,20160929
89,Network activity,domain,alkamaihd.net,,1,20160929
90,Payload delivery,md5,197c018922237828683783654d3c632a,,1,20160929
90,Network activity,domain,dnsrecordsolver.tk,,1,20160929
90,Network activity,ip-dst,178.33.94.47,,1,20160929
90,Payload delivery,filename,Airline.xls,,1,20160929
91,Payload delivery,md5,23a9bbf8d64ae893db17777bedccdc05,,1,20160929
91,Payload delivery,md5,07e47f06c5ed05a062e674f8d11b01d8,,1,20160929
91,Payload delivery,md5,bd75af219f417413a4e0fae8cd89febd,,1,20160929
91,Payload delivery,md5,9f4023f2aefc8c4c261bfdd4bd911952,,1,20160929
91,Network activity,domain,mailsinfo.net,,1,20160929
91,Payload delivery,md5,1e4653631feebf507faeb9406664792f,,1,20160929
92,Payload delivery,md5,6fa869f17b703a1282b8f386d0d87bd4,,1,20160929
92,Payload delivery,md5,24befa319fd96dea587f82eb945f5d2a,,1,20160929

我需要将这个csv文件分成4个csv文件，其中条件是每行开头的事件号。到目前为止，我创建了一个包含所有事件号{89,90,91,92}的集合，我知道我需要在一个循环中创建循环，并将每一行复制到其专用的csv文件中。在

Answer 1

data = {
        '89': [],
        '90': [],
        '91': [],
        '92': []
    }

with open('yourfile.csv') as infile:
    for line in infile:
        prefix = line[:2]
        data[prefix].append(line)

for prefix in data.keys():
    with open('csv' + prefix + '.csv', 'w') as csv:
        csv.writelines(''.join(data[prefix]))

但是，如果您对python以外的解决方案持开放态度，那么可以通过运行四个命令轻松实现这一点

^{pr2}$

其他值也一样。在

Answer 2

最好不要在代码中硬编码事件号，这样它就不依赖于数据的值。我还喜欢使用csv模块，该模块已经过优化，可以读写.csv文件。在

有一种方法可以做到：

import csv

prefix = 'events'  # of output csv file names
data = {}

with open('conditions.csv', 'rb') as conditions:
    reader = csv.reader(conditions)
    for row in reader:
        data.setdefault(row[0], []).append(row)

for event in sorted(data):
    csv_filename = '{}_{}.csv'.format(prefix, event)
    print(csv_filename)
    with open(csv_filename, 'wb') as csvfile:
        writer = csv.writer(csvfile)
        writer.writerows(data[event])

更新

上面实现的方法首先将整个csv文件读入内存，然后将与每个事件值关联的所有行写入一个单独的输出文件，一次一个。在

一种更节省内存的方法是同时打开多个输出文件，并在将每一行读到正确的目标文件后立即将其写入。这样做需要跟踪哪些文件已经打开。文件管理代码需要做的其他事情是确保在处理完成时关闭所有文件。在

在下面的代码中，所有这些都是通过定义并使用PythonContext Manager类型来集中处理可能生成的所有csv输出文件，这些输出文件取决于输入文件中有多少不同的事件值。在

^{pr2}$

下面是如何使用它：

prefix = 'events'  # to name of each csv output file

with open('conditions.csv', 'rb') as conditions:
    reader = csv.reader(conditions)
    with MultiCSVOutputFileManager() as file_manager:
        for row in reader:
            csv_filename = '{}_{}.csv'.format(prefix, row[0])  # row[0] is event
            writer = file_manager.get_csv_writer(csv_filename)
            writer.writerow(row)

Answer 3

如果没有遇到第一个字段，您甚至可以通过保持该id和关联文件的映射来动态创建结果文件：

files = {}
with open('file.csv') as fd:
    for line in fd:
        if 0 == len(line.strip()): continue  # skip empty lines
        try:
            id_field = line.split(',', 1)[0] # extract first field
            if not id in files.keys():       # if not encountered open a new result file
                files[id] = open(id + '.csv')
            files[id].write(line)            # write the line in proper file
        except Exception as e:
            print('ERR', line, e)            # catchall in case of problems...