如何使用自定义参数运行site js函数？

Question

我需要从搜索输入刮谷歌的建议。现在我使用selenium+phantomjswebdriver。你知道吗

search_input = selenium.find_element_by_xpath(".//input[@id='lst-ib']")
search_input.send_keys('phantomjs har python')
time.sleep(1.5)
from lxml.html import fromstring
etree = fromstring(selenium.page_source)
output = []
for suggestion in etree.xpath(".//ul[@role='listbox']/li//div[@class='sbqs_c']"):
            output.append(" ".join([s.strip() for s in suggestion.xpath(".//text()") if s.strip()]))

但我在firebug XHR请求中看到类似this。和响应-简单的文本文件的数据，我需要。然后我看看日志：

selenium.get_log("har")

我看不到这个请求。我怎么能抓住它？我需要这个url作为模板使用的请求库使用它与其他搜索词。或者可以运行js，用其他参数（不是从输入字段）启动这个请求，是吗？你知道吗

Answer 1

你只能用Python+Selenium+PhantomJS来解决这个问题。你知道吗

以下是我为实现这一目标所做的工作清单：

• 通过所需功能
• 使用Explicit Waits
• 询问直接的https://www.google.com/?gws_rd=ssl#q=phantomjs+har+python网址

工作方案：

from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver


desired_capabilities = webdriver.DesiredCapabilities.PHANTOMJS
desired_capabilities["phantomjs.page.customHeaders.User-Agent"] = "Mozilla/5.0 (Linux; U; Android 2.3.3; en-us; LG-LU3000 Build/GRI40) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1"

driver = webdriver.PhantomJS(desired_capabilities=desired_capabilities)
driver.get("https://www.google.com/?gws_rd=ssl#q=phantomjs+har+python")

wait = WebDriverWait(driver, 10)

# focus the input and trigger the suggestion list to be shown
search_input = wait.until(EC.visibility_of_element_located((By.NAME, "q")))
search_input.send_keys(Keys.ARROW_DOWN)
search_input.click()

# wait for the suggestion box to appear
wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "ul[role=listbox]")))

# parse suggestions
print "List of suggestions: "
for suggestion in driver.find_elements_by_css_selector("ul[role=listbox] li[dir]"):
    print suggestion.text

印刷品：

List of suggestions: 
python phantomjs screenshot
python phantomjs ghostdriver
python phantomjs proxy
unable to start phantomjs with ghostdriver