如何使用python下载csv列表？

Question

我正在运行一个python程序，从canada.ca下载选定的CSV文件列表。我有所有我需要的网址，但我不知道如何下载到我的本地目录。我相信我必须使用请求，并在循环中写入文件。但是我有点不知道怎么做，提前谢谢

en_urls = []
for link in soup.find_all('a'):
    if 'EN.csv' in link.get('href', []):
        en_urls.append(link.get('href'))


   Output
['http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/ae-ei/Positive_Employers_EN.csv',
 'http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/ae-ei/2015_Positive_Employers_EN.csv',
 'http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/ae-ei/2016_Positive_Employer_EN.csv',
 'http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/ae-ei/2017Q1Q2_Positive_EN.csv',
 'http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/ae-ei/2017Q3_Positive_Employer_Stream_EN.csv',
 'http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/ae-ei/2018Q1_Positive_Employer_EN.csv',
 'http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/ae-ei/2018Q2_Positive_Employer_EN.csv',
 'http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/ae-ei/2017Q4_Positive_Employer_EN.csv',
 'http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/ae-ei/2018Q3_Positive_EN.csv',
 'http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/ae-ei/2018Q4_Positive_EN.csv',
 'http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/imt-lmi/TFWP_2019Q1_employer_positive_EN.csv',
 'http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/imt-lmi/TFWP_2019Q2_employer_positive_EN.csv',
 'http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/imt-lmi/TFWP_2019Q3_Positive_EN.csv',
 'http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/imt-lmi/TFWP_2019Q4_Positive_EN.csv',
 'http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/imt-lmi/TFWP_2020Q1_Positive_EN.csv']

Answer 1

可以在循环中使用^{}

例如：

import urllib.request

lst = ['http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/ae-ei/Positive_Employers_EN.csv',
 'http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/ae-ei/2015_Positive_Employers_EN.csv',
 'http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/ae-ei/2016_Positive_Employer_EN.csv',
 'http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/ae-ei/2017Q1Q2_Positive_EN.csv']

for i in lst:
    print('Downloading {}..'.format(i))
    local_filename, _ = urllib.request.urlretrieve(i, filename=i.split('/')[-1])
    print('File saved as {}'.format(local_filename))

印刷品：

Downloading http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/ae-ei/Positive_Employers_EN.csv..
File saved as Positive_Employers_EN.csv
Downloading http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/ae-ei/2015_Positive_Employers_EN.csv..
File saved as 2015_Positive_Employers_EN.csv
Downloading http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/ae-ei/2016_Positive_Employer_EN.csv..
File saved as 2016_Positive_Employer_EN.csv
Downloading http://www.edsc-esdc.gc.ca/ouvert-open/bca-seb/ae-ei/2017Q1Q2_Positive_EN.csv..
File saved as 2017Q1Q2_Positive_EN.csv

Answer 2

试试这个：

en_urls = []
for link in soup.find_all('a'):
    if 'EN.csv' in link.get('href', []):
        en_urls.append(link.get('href'))
for link in en_urls:
    with open(f'{link.split("/")[-1]}', 'wb') as file:
        r = requests.get(link, stream=True)
        if r.ok:
            for block in r.iter_content(2*1024**2):
                file.write(block)
        else:
            print(f'Download faild on {link} with {r}')