分析数据，Python

Question

我对Python很感兴趣。我需要解析来自Fred（https://fred.stlouisfed.org/series/A191RI1A225NBEA）的数据。所以，我需要让Python访问这个网站，然后单击“下载”按钮，然后单击“CSV（数据）”按钮，并用Python保存这些数据。实际上，a在执行此任务时遇到了一个很大的问题，我尝试使用selenium：

selenium.webdriver.common.keys import Keys
browser = webdriver.Chrome('/Users/davidabramyan/Downloads/chromedriver 3')
browser = browser.get('https://fred.stlouisfed.org/series/A191RI1A225NBEA')
elm = browser.find_element_by_link_text('Download')
browser.implicitly_wait(5)
elm.click()

使用BeautifulSoup：

response = urllib.request.urlopen(url)
return response.read()

def parse(html):
soup = BeautifulSoup(html, 'lxml')
table = soup.find('span', class_='pull-right col-xs-1')
rows = table.find('fg-download-menu')
print(table.prettify())

def main():
parse(get_html('https://fred.stlouisfed.org/series/A191RI1A225NBEA#0'))

if __name__ == '__main__':
main()

但它不起作用。我觉得美团更好，但我真的不知道怎么做。你能帮我吗？提前谢谢！你知道吗

Answer 1

您不能使用BeautifulSoup，因为“CSV（data）”的下载url是由ajax生成的

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait

browser = webdriver.Chrome('/Users/davidabramyan/Downloads/chromedriver 3')
# make sure above is chromedriver executable path not directory
wait = WebDriverWait(browser, 10)
browser.get('https://fred.stlouisfed.org/series/A191RI1A225NBEA')

dlButton = browser.find_element_by_id("download-button")
dlButton.click()
wait.until(lambda driver: browser.execute_script("return $('#download-data-csv').attr('href') != '#'"))
dlButton = browser.find_element_by_id("download-data-csv")
dlButton.click()

# get url to download with python
# csv_download_url = dlButton.get_attribute('href')
# csv_file = urllib2.urlopen(download)
# ....