擅长:python、mysql、java
<p>如果使用Chrome <code>Network Tools</code>或<code>Firebug</code>打开该网站,您会发现它使用<code>cookies</code>来验证请求。在</p>
<p>您可以通过使用Python创建一个<code>dict</code>来模拟cookies,并将它们与您的请求一起发送。在</p>
<p>在我的示例中,我使用<a href="http://docs.python-requests.org/en/master/" rel="nofollow noreferrer">requests</a>。另外,你不应该把<code>.text</code>放在循环中,它会给你一个错误。在</p>
<pre><code>from bs4 import BeautifulSoup
import requests
cookies = {
'PHPSESSID':'1gd0sknluds2uvumsglth523g5',
'visid_incap_965359':'UGNtvJR1TAmP1y+/M85QuJ1s3lgAAAAAQUIPAAAAAAB5IOYuRCw/9mMOpTnRDCJ6',
'incap_ses_315_965359':'PRZ8WIgqnhyeicz5PxxfBLFs3lgAAAAAYWoblc6exwqhEeGRPqgA5Q=='
}
response = requests.get('http://www.airlinequality.com/airline-
reviews/vietjetair/?sortby=post_date%3ADesc&pagesize=100', cookies=cookies)
soup = BeautifulSoup(response.content, "html.parser")
for Reviewer in soup.findAll(attrs={"class": "text_sub_header userStatusWrapper"}):
print(Reviewer.get_text(strip=True))
</code></pre>
<p><a href="https://i.stack.imgur.com/jhsz5.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/jhsz5.png" alt="Cookies"/></a></p>