<p>您缺少站点可能需要的某些标题</p>
<p>我建议使用<code>requests</code>包而不是<code>urllib</code>,因为它更灵活。请参见下面的工作示例:</p>
<pre><code>import requests
url = "https://www.idealista.com/areas/alquiler-viviendas/?shape=%28%28wt_%7BF%60m%7Be%40njvAqoaXjzjFhecJ%7BebIfi%7DL%29%29"
querystring = {"shape":"((wt_{F`m{e@njvAqoaXjzjFhecJ{ebIfi}L))"}
payload = ""
headers = {
'authority': "www.idealista.com",
'cache-control': "max-age=0",
'upgrade-insecure-requests': "1",
'user-agent': "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36",
'accept': "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
'sec-fetch-site': "none",
'sec-fetch-mode': "navigate",
'sec-fetch-user': "?1",
'sec-fetch-dest': "document",
'accept-language': "en-US,en;q=0.9"
}
response = requests.request("GET", url, data=payload, headers=headers, params=querystring)
print(response.text)
</code></pre>
<p>从那里,您可以使用bs4解析正文:</p>
<pre><code>pageSoup = soup(response.text, "html.parser")
</code></pre>
<p>但是,请注意,您试图抓取的站点可能会显示验证码,因此您可能需要旋转<code>user-agent</code>头和IP地址</p>