我是Scrapy的新手,我试图刮取一个酒店评论,当我刮取链接时,我只得到前五个评论,我需要从tripadvisor上的以下窗口获得评论 我和BeautifulSoup在Jupyter笔记本上这样做 如果有人能帮助我,我将非常感激
代码如下
In[1] import requests
from bs4 import BeautifulSoup as soup
In[2] html = requests.get('https://www.tripadvisor.in/Hotel_Review-g150812-d156853-Reviews-Mahekal_Beach_Resort-Playa_del_Carmen_Yucatan_Peninsula.html')
html.status_code
Out[3] 200
In[4] bsobj = soup(html.content,'lxml')
In[5] links = []
for review in bsobj.findAll('a',{'class':'pageNum'}):
a = review['href']
a = 'https://www.tripadvisor.in'+ a
print(a)
print(a.find('Reviews'))
a = a[:(a.find('Reviews')+7)] + '-or{}' + a[(a.find('Reviews')+7):]
print(a)
links.append(a)
links
Out[] https://www.tripadvisor.in/Hotel_Review-g150812-d156853-Reviews-or5-
Mahekal_Beach_Resort-Playa_del_Carmen_Yucatan_Peninsula.html
56
https://www.tripadvisor.in/Hotel_Review-g150812-d156853-Reviews-or{}-or5-
Mahekal_Beach_Resort-Playa_del_Carmen_Yucatan_Peninsula.html
https://www.tripadvisor.in/Hotel_Review-g150812-d156853-Reviews-or10-
Mahekal_Beach_Resort-Playa_del_Carmen_Yucatan_Peninsula.html
56
https://www.tripadvisor.in/Hotel_Review-g150812-d156853-Reviews-or{}-or10-
Mahekal_Beach_Resort-Playa_del_Carmen_Yucatan_Peninsula.html
https://www.tripadvisor.in/Hotel_Review-g150812-d156853-Reviews-or15-
Mahekal_Beach_Resort-Playa_del_Carmen_Yucatan_Peninsula.html
56
https://www.tripadvisor.in/Hotel_Review-g150812-d156853-Reviews-or{}-or15-
Mahekal_Beach_Resort-Playa_del_Carmen_Yucatan_Peninsula.html
https://www.tripadvisor.in/Hotel_Review-g150812-d156853-Reviews-or20-
Mahekal_Beach_Resort-Playa_del_Carmen_Yucatan_Peninsula.html
56
https://www.tripadvisor.in/Hotel_Review-g150812-d156853-Reviews-or{}-or20-
Mahekal_Beach_Resort-Playa_del_Carmen_Yucatan_Peninsula.html
https://www.tripadvisor.in/Hotel_Review-g150812-d156853-Reviews-or25-
Mahekal_Beach_Resort-Playa_del_Carmen_Yucatan_Peninsula.html
56
https://www.tripadvisor.in/Hotel_Review-g150812-d156853-Reviews-or{}-or25-
Mahekal_Beach_Resort-Playa_del_Carmen_Yucatan_Peninsula.html
https://www.tripadvisor.in/Hotel_Review-g150812-d156853-Reviews-or2155-
Mahekal_Beach_Resort-Playa_del_Carmen_Yucatan_Peninsula.html
56
https://www.tripadvisor.in/Hotel_Review-g150812-d156853-Reviews-or{}-or2155-
Mahekal_Beach_Resort-Playa_del_Carmen_Yucatan_Peninsula.html
['https://www.tripadvisor.in/Hotel_Review-g150812-d156853-Reviews-or{}-or5-
Mahekal_Beach_Resort-Playa_del_Carmen_Yucatan_Peninsula.html',
'https://www.tripadvisor.in/Hotel_Review-g150812-d156853-Reviews-or{}-or10-
Mahekal_Beach_Resort-Playa_del_Carmen_Yucatan_Peninsula.html',
'https://www.tripadvisor.in/Hotel_Review-g150812-d156853-Reviews-or{}-or15-
Mahekal_Beach_Resort-Playa_del_Carmen_Yucatan_Peninsula.html',
'https://www.tripadvisor.in/Hotel_Review-g150812-d156853-Reviews-or{}-or20-
Mahekal_Beach_Resort-Playa_del_Carmen_Yucatan_Peninsula.html',
'https://www.tripadvisor.in/Hotel_Review-g150812-d156853-Reviews-or{}-or25-
Mahekal_Beach_Resort-Playa_del_Carmen_Yucatan_Peninsula.html',
'https://www.tripadvisor.in/Hotel_Review-g150812-d156853-Reviews-or{}-or2155-
Mahekal_Beach_Resort-Playa_del_Carmen_Yucatan_Peninsula.html']
in[6] from random import randint
from time import sleep
reviews = []
for link in links:
d = [5,10,15,20,25]
headers = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; WOW64)
AppleWebKit/537.36
(KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36'}
html2 = requests.get(link.format(i for i in d),headers=headers)
sleep(randint(1,5))
bsobj2 = soup(html2.content,'lxml')
for r in bsobj2.findAll('q'):
reviews.append(r.span.text.strip())
print(r.span.text.strip())
reviews
Out[] My husband and I stayed 5 nights here with some other couples. It was very
relaxing. The staff was incredible. They knew all of our names. The food was
great. We went to El Fuego and Las Olas several days and nights. We had a hut
on the beach with a plunge pool. It was fabulous. They cleaned it daily, even
chocolate on my pillow. There was 24/7 security everywhere on property. My
only complaint was there was a huge party on the beach one night that raged
until 6am, but that had nothing to do with the resort. We would return
anytime!
My wife and I had an great stay at beautiful Mahekal Beach Resort. We arrive
only hours after hurricane Zeta. They and a truck size generator powering the
place due to power outage and staff of the resort did an amazing job with
clean up. We we're lucky to have chosen a location prepared and ready to keep
things open. Within a day you could hardly tell a storm went by. The
restaurant had great food, but the attentive service was what made it
special. They all knew our names and room within a day. They go above and
beyond to make sure you are happy and enjoying every moment of your stay.
Thanks so much for a wonderful stay. Rogelio and all the staff made it more
than just a beautiful beach vacation. We'll remember the people as much as
the place.
Just returned from a one week stay and LOVE this resort. To compare - we have
been to Secrets Maroma/Playa Mujeres/Capri/ Grand Residences/ and Zoetry
Paraiso. I would never return to any of those resorts- and only Mahekal
moving forward. We stayed in the King Penthouse ocean front- and I believe
this is the best room at the resort because not only is the room very large-
with a huge outdoor space right on the beach- but it is also PRIVATE. The
ocean front rooms are basically on the beach- and offer little to no privacy
at all. We loved the location and the room was beautiful- bed was a little
hard but still comfortable. The resort location was awesome- we went out to
eat every night and walked to most locations- but did try some restaurants
where we took a short $8 cab ride.
My wife and I had another great time at Mahekal. We love this resort with its
beautiful tropical garden, very clean and beautiful palapas (rooms), a
magnificent beach and amazing staff ! We have been at Mahekal many times and
always come back. This time we enjoyed our meals mainly at Fuego Restaurant
and Cantina, since the other restaurants were closed due to Covid. Fuego is a
great place, close to the beach, we were always happy to go there and enjoy
the delicious food and the fantastic people that work there, Zuri, Vianney,
Enrique, Ismael, Angel, Rogelio, Ignacio, Marcos, Manuel, Rene, Rubén, Sergio
with the daily breakfast bread and homemade marmalade ! And Manolo, Alberto
and Israel at the beach and also Gilberto. We also enjoyed some life music
during dinner, nice songs
My wife and I had the most magical time and it would not be possible with out
the amazing people at Mahekal! Staff is amazing and will do their best to
make you feel right at home. We stayed in room #5. Amazing view front row on
the beach. Music plays at night but the sounds of the ocean take care of you.
Manolo was a person we wont forget. Super friendly and great at his job. He
was there for anything we needed while enjoying the beach view from our room.
Everyone at Fuego is incredible! Rogelio & Vianney took very good care of us.
You would think They knew Us. The restaurant is named Fuego for a Reason.
Everything
is Delicious. We will definitely be back and this time with our family ! I
cannot wait to see everyone again and enjoy the warm days at Mahekal. One
visit isn’t
目前没有回答
相关问题 更多 >
编程相关推荐