刮取angellist启动数据

Question

我想从https://angel.co/companies?locations[]=1688-United+States中提取数据。谁能告诉我该怎么办

我知道我应该使用BeautifulSoup或Selenium，但最终我发现这个网页不是静态的，它会一次又一次地更改其数据，有人能指导我吗

我认为angellist API网页不再工作了

Answer 1

您需要等待几秒钟，直到生成第页上的表：

from selenium import webdriver
import os
import time

chrome_driver = os.path.abspath(os.path.dirname(__file__)) + '/chromedriver'
browser = webdriver.Chrome(chrome_driver)
browser.get("https://angel.co/companies?locations[]=1688-United+States")
time.sleep(3)

data_row = browser.find_elements_by_class_name('base.startup')

for item in data_row:
    print('-'*100)
    company = item.find_element_by_class_name('name').text
    location = item.find_element_by_class_name('column.location').text
    print(company)
    print(location)

输出：

                                                  
WP Engine
Austin
                                                  
Kissmetrics
San Francisco
                                                  
Bluesmart
San Francisco
                                                  
Star.me
Los Angeles
...
...