使用BeautifulSoup收集数据

Question

我试图得到价格值，但我无法得到它，它总是返回一个“-”

我想让它退回1376000

HTML

---

<span class="price_big_right">
    <span id="pc-lowest-1" data-price="1,376,000">
        1,376,000 
        <img alt="c" class="coins_icon_l_bin" src="https://example.com/123"/>
    </span>
</span>

这就是我想要做的

import requests
from bs4 import BeautifulSoup
import smtplib

URL = 'https://www.futbin.com/22/player/426'
def test():

    page = requests.get(URL)

    soup = BeautifulSoup(page.content, 'html.parser')

    price = soup.find(id="pc-lowest-1").get_text()


# price
print(price)

test()

Answer 1

此页面中的数据正在由JavaScript加载。所以beautifulsoup不起作用。您必须使用selenium来获取数据

但是，如果您在Chrome DevTools中的“网络”选项卡下看到，页面将从下面的端点加载数据

https://www.futbin.com/22/playerPrices?player=20801&rids=50352449,67129665

您可以直接向该端点发出请求并获取数据

这是你可以做到的

import requests

URL = 'https://www.futbin.com/22/playerPrices?player=20801&rids=50352449,67129665'

page = requests.get(URL)
x = page.json()
data = x['20801']['prices']
print(data['pc']['LCPrice'])

1,349,000