我发送带有有效负载和搜索文本aaa的请求https://www.cnmv.es/portal/Consultas/BusquedaPorEntidad.aspx,但得到javascript响应。所以我需要渲染javascript,但我不想使用Selenium。我也不确定我的有效载荷是否良好
url = 'https://www.cnmv.es/portal/Consultas/BusquedaPorEntidad.aspx'
search_text = 'aaa'
r = requests.get('https://www.cnmv.es/portal/Consultas/BusquedaPorEntidad.aspx')
soup = BeautifulSoup(r.content, 'html.parser')
VIEWSTATE = soup.find(id="__VIEWSTATE")['value'] + '%3D&'
VIEWSTATEGENERATOR = '__VIEWSTATEGENERATOR=' + soup.find(id="__VIEWSTATEGENERATOR")['value']
EVENTVALIDATION = '&__EVENTVALIDATION' + soup.find(id="__EVENTVALIDATION")['value']
SEARCH = "&ctl00%24wBusqueda%24txtBusqueda=&ctl00%24ContentPrincipal%24txtBusqueda={0}&ctl00%24ContentPrincipal%24btnBuscar=Search".format(search_text)
payload = '__EVENTTARGET=&__EVENTARGUMENT=&__VIEWSTATE=' + VIEWSTATE + VIEWSTATEGENERATOR + EVENTVALIDATION + SEARCH
headers = {
'Connection': 'keep-alive',
'Cache-Control': 'max-age=0',
'Upgrade-Insecure-Requests': '1',
'Origin': 'https://www.cnmv.es',
'Content-Type': 'application/x-www-form-urlencoded',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36',
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
'Sec-Fetch-Site': 'same-origin',
'Sec-Fetch-Mode': 'navigate',
'Sec-Fetch-User': '?1',
'Sec-Fetch-Dest': 'document',
'Referer': 'https://www.cnmv.es/portal/Consultas/BusquedaPorEntidad.aspx',
'Accept-Language': 'en-US,en;q=0.9',
}
response = requests.request("POST", url, headers=headers, data = payload)
print(response.text.encode('utf8'))
我没有测试你的
payload
,但我不知道你为什么要将%3D
添加到__VIEWSTATE
我使用的字典}中使用{}而不是{},等等
requests
将自动转换为字符串,而不必手动添加&
。我不必在{顺便说一句:代码在没有标题的情况下对我有效,但我会对它们进行注释
结果:
相关问题 更多 >
编程相关推荐