如何在不使用Python中任何内置库的情况下处理其中一列中存在的列分隔符？

Question

我想在Python3中读取CSV文件，但由于某些限制，无法使用任何库。几乎在每一行中，一个或多个列都包含逗号（“，”），使用row.split(',')会导致列数增加而出现问题

我的代码是：

import csv

file_name = "train_1.csv"
columns = [
    "PassengerId",
    "Survived",
    "Pclass",
    "Name",
    "Sex",
    "Age",
    "SibSp",
    "Parch",
    "Ticket",
    "Fare",
    "Cabin",
    "Embarked"
]
print("Total columns should be: {}".format(len(columns)))
with open(file_name, 'r') as reader:
    for line in reader.readlines():
        row_data = line.split(',')
        if len(row_data) != len(columns):
            print('This row does not have the required # of columns: {}'.format(
                len(row_data)))
            print(row_data)

我的输出（错误）是：

['1', '0', '3', '"Braund', ' Mr. Owen Harris"', 'male', '22', '1', '0', 'A/5 21171', '7.25', '', 'S\n']

相反，它应该是：

['1', '0', '3', '"Braund, Mr. Owen Harris"', 'male', '22', '1', '0', 'A/5 21171', '7.25', '', 'S']

额外的列是由于名称被拆分为两个而不是一个，以及最后一列中的\n

然而，我主要关心的是额外的列被拆分。注意：这个问题由CSV阅读器解决，但由于库的限制，我不能真正使用任何库

部分输入为：

PassengerId,Survived,Pclass,Name,Sex,Age,SibSp,Parch,Ticket,Fare,Cabin,Embarked
1,0,3,"Braund, Mr. Owen Harris",male,22,1,0,A/5 21171,7.25,,S
2,1,1,"Cumings, Mrs. John Bradley (Florence Briggs Thayer)",female,38,1,0,PC 17599,71.2833,C85,C
3,1,3,"Heikkinen, Miss. Laina",female,26,0,0,STON/O2. 3101282,7.925,,S
4,1,1,"Futrelle, Mrs. Jacques Heath (Lily May Peel)",female,35,1,0,113803,53.1,C123,S

完整的数据可用here

Answer 1

名称列值中的逗号将名称分成两列。下面的解决方案解决了这个问题，并从列值中删除新行

print("Total columns should be: {}".format(len(columns)))
with open(file_name, 'r') as reader:
    for line in reader.readlines():
        row_data = line.replace('\n', '').split(',')
        if len(row_data) != len(columns):
            row_data[3] = (row_data[3]+ ',' + row_data[4])
            del row_data[4]
            print(row_data)
        else:
            print(row_data)

Answer 2

不能使用任何内置模块是一个奇怪的限制，但是创建自己的csv解析器非常简单

正如您所注意到的，您必须处理值包含逗号的情况，CSV通过引用整个字符串来处理逗号

在完整数据链接中，还有一行添加了另一个褶皱：

889,0,3,"Johnston, Miss. Catherine Helen ""Carrie""",female,,1,2,W./C. 6607,23.45,,S

这是一个带有嵌入逗号的值，因此它被引用。但是，它在中也有一个引号，因此CSV格式通过将引号加倍来“转义”这些引号。我假设您需要保留这些转义的引号

def csv_values(text_line, delim=','):
    row = []
    embedded = False
    parts = []

    for word in text_line.split(delim):
        # Set flag marking start of quoted value
        if word.startswith('"'):
            embedded = True

        if embedded:
            # If scanning a quoted value (with embedded commas),
            # add the current portion to the accumulator
            # word = word.replace('""', r'"')
            parts.append(word)
        else:
            # Otherwise, append the value to the collection
            row.append(word)

        # Unset flag, marking end of quoted value
        if word.endswith('"'):
            embedded = False
            # Add the accumulated value
            # row.append(','.join(parts)[1:-1])
            row.append(','.join(parts))
            # Reset the accumulator
            parts = []

    return row

这个实现是我的“原样”方法，这意味着我所做的唯一事情就是积累具有嵌入逗号的值。我使用第882-891行得到这个结果：

['882', '0', '3', '"Markun, Mr. Johann"', 'male', '33', '0', '0', '349257', '7.8958', '', 'S']
['883', '0', '3', '"Dahlberg, Miss. Gerda Ulrika"', 'female', '22', '0', '0', '7552', '10.5167', '', 'S']
['884', '0', '2', '"Banfield, Mr. Frederick James"', 'male', '28', '0', '0', 'C.A./SOTON 34068', '10.5', '', 'S']
['885', '0', '3', '"Sutehall, Mr. Henry Jr"', 'male', '25', '0', '0', 'SOTON/OQ 392076', '7.05', '', 'S']
['886', '0', '3', '"Rice, Mrs. William (Margaret Norton)"', 'female', '39', '0', '5', '382652', '29.125', '', 'Q']
['887', '0', '2', '"Montvila, Rev. Juozas"', 'male', '27', '0', '0', '211536', '13', '', 'S']
['888', '1', '1', '"Graham, Miss. Margaret Edith"', 'female', '19', '0', '0', '112053', '30', 'B42', 'S']
['889', '0', '3', '"Johnston, Miss. Catherine Helen ""Carrie"""', 'female', '', '1', '2', 'W./C. 6607', '23.45', '', 'S']
['890', '1', '1', '"Behr, Mr. Karl Howell"', 'male', '26', '0', '0', '111369', '30', 'C148', 'C']
['891', '0', '3', '"Dooley, Mr. Patrick"', 'male', '32', '0', '0', '370376', '7.75', '', 'Q']

如果您希望不使用封闭引号并取消对嵌入引号的转义，可以取消注释行14&；24，并注释掉第25行。然后，该方法将给出：

['882', '0', '3', 'Markun, Mr. Johann', 'male', '33', '0', '0', '349257', '7.8958', '', 'S']
['883', '0', '3', 'Dahlberg, Miss. Gerda Ulrika', 'female', '22', '0', '0', '7552', '10.5167', '', 'S']
['884', '0', '2', 'Banfield, Mr. Frederick James', 'male', '28', '0', '0', 'C.A./SOTON 34068', '10.5', '', 'S']
['885', '0', '3', 'Sutehall, Mr. Henry Jr', 'male', '25', '0', '0', 'SOTON/OQ 392076', '7.05', '', 'S']
['886', '0', '3', 'Rice, Mrs. William (Margaret Norton)', 'female', '39', '0', '5', '382652', '29.125', '', 'Q']
['887', '0', '2', 'Montvila, Rev. Juozas', 'male', '27', '0', '0', '211536', '13', '', 'S']
['888', '1', '1', 'Graham, Miss. Margaret Edith', 'female', '19', '0', '0', '112053', '30', 'B42', 'S']
['889', '0', '3', 'Johnston, Miss. Catherine Helen "Carrie"', 'female', '', '1', '2', 'W./C. 6607', '23.45', '', 'S']
['890', '1', '1', 'Behr, Mr. Karl Howell', 'male', '26', '0', '0', '111369', '30', 'C148', 'C']
['891', '0', '3', 'Dooley, Mr. Patrick', 'male', '32', '0', '0', '370376', '7.75', '', 'Q']

在任何情况下，您都可以使用如下功能：

with open(file_name, 'r') as in_file:
    csv_lines = in_file.splitlines()

# Separate header from rest
headers, lines = csv_lines[0], csv_lines[1:]

for line in lines:
    print(csv_values(line))

Answer 3

在观察了csv文件之后，我发现名称列很混乱，应该进行处理

file_name = "train_1.csv"
columns = [
    "PassengerId",
    "Survived",
    "Pclass",
    "Name",
    "Sex",
    "Age",
    "SibSp",
    "Parch",
    "Ticket",
    "Fare",
    "Cabin",
    "Embarked"
]
print("Total columns should be: {}".format(len(columns)))
header = False
with open(file_name, 'r') as reader:
    for line in reader.readlines():
        line = line[:-1]
        if not header:
            header = True
            continue
        line_pre_name = line.split('"', 1)[0].split(',')[:-1]
        name = [line.split('"', 2)[1]]
        line_post_name = line.split('"')[-1].split(',')[1:]
        row_data = line_pre_name + name + line_post_name
        if len(row_data) != len(columns):
            print('This row does not have the required # of columns: {}'.format(
                len(row_data)))
            print(row_data)