如何从inFile读取头文件并将头文件写入outFile？（Python3）

Question

我在文件read中使用next是为了解析数据而不是头行，因此read完全避免了第一行。如何在读操作（而不是解析标题行）中提取标题，然后在写操作中写入标题？你知道吗

我要对的实际数据集是30col和80k行，所以我尝试在一次读取操作中完成这个操作。你知道吗

试验数据：

date, animal, color
3/14/2015, cat, blue
3/24/2015, dog, green

代码：

from dateutil.parser import *
import csv

with open('testin.csv', 'r', encoding='utf-8') as inFile, open('testout.csv', 'w', encoding='utf-8') as outFile:
    exampleReader = csv.reader(inFile)
    next(exampleReader, 1)
    exampleData = list(exampleReader)
    exampleWriter = csv.writer(outFile)
    # print a few to see what it's doing
    print('the list', exampleData)
    for item in exampleData:
        item[0] = str(parse(item[0])) # converting date format for sqlite
        del item[2] # dropping column that is not needed
        print('date corrected', item) 
        exampleWriter.writerow(item)

Answer 1

我会使用pandas来处理这样大量的数据：

import io
import pandas as pd

data = """\
date, animal, color, junk
3/14/2015, cat, blue, aaa
3/24/2015, dog, green, bbb
"""
num_cols = 4
all_cols = set(range(num_cols))
skip_cols = set([2,3])

# replace `io.StringIO(data)` with the CSV filename    
df = pd.read_csv(io.StringIO(data),
                 sep=',',
                 skipinitialspace=True,
                 parse_dates=[0],
                 usecols=(all_cols - skip_cols))
print(df)

# save DF as CSV file
df.to_csv('/path/to/new.csv', index=False)

# save DF to SQLite DB
import sqlalchemy
engine = sqlalchemy.create_engine('sqlite:///my_db.sqlite')
df.to_sql('my_table', engine, if_exists='replace')

示例：

In [150]: data = """\
   .....: date, animal, color, junk
   .....: 3/14/2015, cat, blue, aaa
   .....: 3/24/2015, dog, green, bbb
   .....: """

In [151]: num_cols = 4

In [152]: all_cols = set(range(num_cols))

In [153]: skip_cols = set([2,3])

In [154]: df = pd.read_csv(io.StringIO(data),
   .....:                  sep=',',
   .....:                  skipinitialspace=True,
   .....:                  parse_dates=['date'],
   .....:                  usecols=(all_cols - skip_cols))

In [155]: print(df)
        date animal
0 2015-03-14    cat
1 2015-03-24    dog