正在尝试用python从网站下载.csv

Question

我正在尝试从以下网站下载.csv:https://www.nasdaq.com/market-activity/stocks/screener?exchange=nyse&letter=0&render=download

到目前为止，我试图：

    import requests
    import shutil
    url = "https://www.nasdaq.com/market-activity/stocks/screener?exchange=nyse&letter=0&render=download"
    r = requests.get(url)
    if r.status_code != 200:
        print("Failure")
        exit()
    else:
        r.raw.decode_content = True
        with open("Stocks.csv", "w") as f:
            shutil.copyfileobj(r.raw, f)
        print("Success")

这是：

    import pandas as pd
    df = pd.read_csv(url)
    df.to_csv("Stocks.csv")

然而，由于某种原因，这只会导致什么也不返回，甚至不会返回错误。这就好像它被困在一个无限的循环中。任何帮助都将不胜感激

Answer 1

正如@JoeAkanesuvan所指出的，信息是通过JSON API获得的。这可以使用requests库访问。然后可以使用Python将其转换为CSV文件，如下所示：

import requests
import csv

headers = {
    "User-Agent" : "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:84.0) Gecko/20100101 Firefox/84.0",
}

url = "https://api.nasdaq.com/api/screener/stocks?tableonly=true&limit=3296&exchange=nyse"
r = requests.get(url, headers=headers)
j = r.json()

table = j['data']['table']
table_headers = table['headers']

with open('Stocks.csv', 'w', newline='') as f_output:
    csv_output = csv.DictWriter(f_output, fieldnames=table_headers.values(), extrasaction='ignore')
    csv_output.writeheader()

    for table_row in table['rows']:
        csv_row = {table_headers.get(key, None) : value for key, value in table_row.items()}
        csv_output.writerow(csv_row)

我建议您print(j)更好地理解返回数据的结构

这将使您的输出开始：

Symbol,Name,Last Sale,Net Change,% Change,Market Cap
BABA,Alibaba Group Holding Limited American Depositary Shares each representing eight Ordinary share,$260.25,-5.67,-2.132%,"704,141,925,150"
TSM,Taiwan Semiconductor Manufacturing Company Ltd.,$121.74,-4.91,-3.877%,"631,343,640,000"
JNJ,Johnson & Johnson Common Stock,$167.88,-2.60,-1.525%,"441,951,263,775"