Python网页抓取无法找到网页中的所有标记

Question

我在试着刮一张特别的网页码。但是我找不到里面所有的段落标签。在

我已经把下面的问题做完了

Beautiful Soup findAll doen't find them all，但这似乎不能解决问题。在

这是一个不断刷新的动态网页，如果我点击页面底部的“加载更多评论”按钮，就会加载额外的内容。在

代码：

from bs4 import BeautifulSoup
import requests

r = requests.get("http://www.cricbuzz.com/live-cricket-scores/18127")
data = r.text

soup = BeautifulSoup(data)
p = soup.find_all('p')

len(p) 

10

^{pr2}$

Boult to Hardik Pandya, FOUR, that is probably the blunder which will cost KKR the match. It shouldn't have been any more than a single. A low full toss which Hardik can't find any elevation with. He smacks it down to long-on, where Surya attacks the ball nicely but he misfields and the ball sneaks through

不管怎样，我能从这个网页上获取所有的评论数据吗？在

Answer 1

你得到的是段落p[9]（p-tag）我想，你需要把打印语句放在一个循环中才能打印所有段落。像这样：

body = soup.body
for p in body.find_all('p')
    print(p.text)

Answer 2

要获取所有评论，可以使用siteapi:http://push.cricbuzz.com/match-api/18127/commentary-full.json。它以json格式返回所有数据，您可以轻松解析和提取所需的数据：

import requests

r = requests.get('http://push.cricbuzz.com/match-api/18127/commentary-full.json').json() 

all_comments = r['comm_lines']

# print first 10 comments
for comment in all_comments[:10]:
    if 'comm' in comment:
        print(comment['comm'])