将大型SQL文件转换为CSV文件

2024-09-17 02:09:24 发布

您现在位置:Python中文网/ 问答频道 /正文

我下载了一个大数据集(压缩文件17G(.sql文件)),我想将其转换为csv或导入jupyter笔记本

我尝试了以下代码:

import sqlite3
import pandas as pd
# Open the file
f = open('output.csv', 'w')
# Create a connection and get a cursor
connection = sqlite3.connect('steam.sql')
cursor = connection.cursor()
# Execute the query
cursor.execute('select * from steam')
# Get data in batches
while True:
    # Read the data
    df = pd.DataFrame(cursor.fetchmany(1000))
    # We are done if there are no data
    if len(df) == 0:
        break
    # Let's write to the file
    else:
        df.to_csv(f, header=False)

# Clean up
f.close()
cursor.close()
connection.close()

但我有一个错误:

DatabaseError
Traceback (most recent call last)
in
8 cursor = connection.cursor()
9 # Execute the query
10 cursor.execute('select * from steam')
11 # Get data in batches
12 while True:

DatabaseError: file is not a database

数据集来自https://steam.internet.byu.edu/#

求你了-我该怎么做

我需要我的项目中的一些数据


Tags: csvthe数据inimportdfclosesql