使用请求从Zomato餐厅提取评论

Question

我在Zomato网站上尝试网页抓取。 我只想对1家餐厅发表评论

import requests
from bs4 import BeautifulSoup
import re

headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36'}
r = requests.get('https://www.zomato.com/mumbai/joeys-pizza-malad-west/reviews',headers=headers)
soup = BeautifulSoup(r.text, 'html.parser')
#regex = re.compile('.*comment.*')
results = soup.find_all('p', {'class':'sc-1hez2tp-0 sc-eomEcv kOjze'})
reviews = [result.text for result in results]

但是这个代码花费了太多的时间。 我的实现是错误的？

我想从这个URLhttps://www.zomato.com/mumbai/joeys-pizza-malad-west/reviews中刮取所有用户的评论

Answer 1

将headers更改如下：

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36'}

此外，类名是动态加载的，并不断更改。相反，您可以做的是找到没有更改的类sc-1hez2tp-0 fKvqMN，然后使用^{}方法找到包含所需输出的前一个p

import requests
from bs4 import BeautifulSoup


headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36"
}
r = requests.get(
    "https://www.zomato.com/mumbai/joeys-pizza-malad-west/reviews", headers=headers
)
soup = BeautifulSoup(r.text, "html.parser")

for tag in soup.find_all("p", class_="sc-1hez2tp-0 fKvqMN"):
    print(tag.find_previous("p").text)

输出：

I had eaten veg chefs favourite ( mushroom ) a couple of years back , and I still remembered the delicious taste of mushrooms and that's why ordered it again today , but today all it had was boiled mushrooms , totally disappointed , ruined my taste buds , will never ever order again from here...I guess they did a shortcut due to high demand as delivery was also done very late after a reminder
JOEYS IS THE BEST PIZZA EVER. They give a lot of stuffings , the cheese is to die for . Joey’s Will always be my first choice . Paneer makhni and chicken makhani are the best ❤️
Their pizza may be good but they hve worst delivery patterns… my todays order i have been waiting for 1 1/2 hr for single potion pizza… worst and frustrating service… not for people who wants to enjoy pizza… now i feel its over rated… 

Famous Pizza corner Joey's pizza was so amazing and delicious also the toppings was so amazing . The thin crust cheese burst pizza was so yummy 😋🤤