使用selenium在.click（）之后获取新的html

Question

我使用selenium来点击一个链接，但是我不能得到新的表。我用什么代码检索新页面？在

    df_list = []
    url = 'https://www.cartolafcbrasil.com.br/scouts/cartola-fc-2018/rodada-1' #+ str(i)
    page = requests.get(url)
    soup = BeautifulSoup(page.text, 'html.parser')
    table = soup.find_all('table')[0]
    df = pd.read_html(str(table), encoding="UTF-8")

    driver = webdriver.PhantomJS(executable_path = 'C:\\Python27\\phantomjs-2.1.1-windows\\bin\\phantomjs')
    driver.get('https://www.cartolafcbrasil.com.br/scouts/cartola-fc-2018/rodada-1') 
    driver.find_element_by_xpath("/html[1]/body[1]/form[1]/div[1]/div[2]/div[3]/div[1]/div[1]/div[2]/div[1]/div[1]/div[2]/div[1]/div[2]/div[1]/table[1]/tbody[1]/tr[52]/td[1]/table[1]/tbody[1]/tr[1]/td[2]/a[1]").click()



    ?????
    table = soup.find_all('table')[0]
    df = pd.read_html(str(table), encoding="UTF-8")

Answer 1

欢迎来到这里。这是另一种方法，脚本将遍历所有表（页）并获取数据。在

df_list = []
url = 'https://www.cartolafcbrasil.com.br/scouts/cartola-fc-2018/rodada-1' #+ str(i)
page = requests.get(url)
soup = BeautifulSoup(page.text, 'html.parser')
table = soup.find_all('table')[0]
df = pd.read_html(str(table), encoding="UTF-8")

driver = webdriver.PhantomJS(executable_path = 'C:\\Python27\\phantomjs-2.1.1-windows\\bin\\phantomjs')
driver.get('https://www.cartolafcbrasil.com.br/scouts/cartola-fc-2018/rodada-1')
# get the number of pages and iterate each of them
numberOfPage = driver.find_element_by_xpath("(//tr[@class='tbpaging']//a)[last()]").text
for i in range(2,int(numberOfPage)):
    # click on each page link and then get the details
    driver.find_element_by_xpath("(//tr[@class='tbpaging']//a)[" + i +"]").click()
    soup = BeautifulSoup(driver.page_source, 'html.parser')
    table = soup.find_all('table')[0]
    df = pd.read_html(str(table), encoding="UTF-8")

Answer 2

如果我理解你的问题，那就是“如何从我的driver对象中为我加载的新页面获取HMTL？”。答案是driver.page_source

driver.find_element_by_xpath("Some crazy shenanigans of an xpath").click()
html_from_page = driver.page_source
soup = bs4.BeautifulSoup(html_from_page, 'html.parser')
# more stuff