在Python中向csv文件写入超过65535行？

Question

我在Python脚本中有以下逻辑：

def importAndAnalyze(rowLimit = 3):
    layerProperties = iface.addVectorLayer("Downloads/parcels-shp/parcels.shp", "Parcels", "ogr")
    if not layerProperties:
        print("layerProperties failed to load!")
    layerEntrances = iface.addVectorLayer("Downloads/Metro_Stations-shp/Metro_Stations.shp", "Metro_Stations", "ogr")
    if not layerEntrances:
        print("layerEntrances failed to load!")
    features = layerProperties.getFeatures()
    counter = 0
    featuresSelected = []
    Path('Desktop/output.csv').touch()

    with open('Desktop/output.csv', 'w') as csvfile:
        csvWriter = csv.writer(csvfile, delimiter=',', quotechar='"', quoting=csv.QUOTE_NONNUMERIC)
        csvWriter.writerow(['Tooltip', 'Lat','Lon', 'Distance'])
        for feature in features:
            if counter < rowLimit:
                csvWriter.writerow([feature['TOOLTIP'], feature.geometry().centroid().asPoint().y(),feature.geometry().centroid().asPoint().x(), Ranker.calculateDistance(feature, layerEntrances)])
                if counter % 100 == 0:
                    csvfile.flush()
            else:
                break
            counter += 1

Ranker.importAndAnalyze(1000000)

当我向它提供800000行信息时，rowLimit变量设置为一百万，它当前将65535行保存到output.csv文件中，然后停止保存新行，即使脚本继续运行数小时

如何让脚本保存其他734465行

===

编辑：关于我的rowLimit变量是否实际设置为一百万，在评论/答案中有很多合理的推测。因此，我添加了更多的代码来显示如何运行的上下文

Answer 1

with open('Desktop/output.csv', 'w') as csvfile:
    csvWriter = csv.writer(csvfile, delimiter=',', quotechar='"', quoting=csv.QUOTE_NONNUMERIC)
    csvWriter.writerow(['Tooltip', 'Lat','Lon', 'Distance'])
    counter = 0
    for feature in features[:rowLimit]:
        csvWriter.writerow([feature['TOOLTIP'], feature.geometry().centroid().asPoint().y(),feature.geometry().centroid().asPoint().x(), Ranker.calculateDistance(feature, layerEntrances)])
        if counter % 100 == 0:
            csvfile.flush()
        counter += 1
    csvfile.flush()

Answer 2

csv模块本身没有施加这样的约束。当您的代码到达变量rowLimit（您的问题中没有定义）时，它将停止编写，因此显而易见的简单解决方案是去掉该变量

这里有一个重构的尝试，它还重命名了变量以符合Python约定

with open('Desktop/output.csv', 'w') as csvfile:
    csv_writer = csv.writer(csvfile, delimiter=',', quotechar='"', quoting=csv.QUOTE_NONNUMERIC)
    csv_writer.writerow(['Tooltip', 'Lat','Lon', 'Distance'])
    for counter, feature in enumerate(features):
        # if counter == row_limit:
        #     raise StopIteration('row_limit reached')
        csv_writer.writerow([feature['TOOLTIP'], feature.geometry().centroid().asPoint().y(),feature.geometry().centroid().asPoint().x(), Ranker.calculateDistance(feature, layerEntrances)])
        if counter % 100 == 0:
            csvfile.flush()

当然，您可能使用一个程序来读取生成的CSV文件，该文件无法处理任意大的文件；至少Excel似乎对大于65535行的工作表有问题（但朋友不允许朋友使用Excel）