如何从网站<script>中的列表中刮取数据？

Question

这是我第一次做网页抓取，我不知道如何从script标签内的字典列表中抓取数据。因为脚本标记没有类，所以我不知道如何从特定的脚本标记访问内容

到目前为止，守则是：

from bs4 import BeautifulSoup
import requests

source = requests.get('https://champion.gg/statistics/').text


soup = BeautifulSoup(source, 'lxml')

stats = soup.find('script')

这是我想从中获取的一小部分数据：

<script>
      matchupData.stats = [{"key":"Ezreal","role":"ADC","title":"Ezreal","general":{"winPercent":0.5046896283323604,"playPercent":0.17104628134933184,"banRate":0.03167301835610511,"experience":8.02309599159886,"kills":6.8780177725887,"deaths":5.2307193357981445,"assists":7.356567425569177,"totalDamageDealtToChampions":23163,"totalDamageTaken":18062,"totalHeal":2607,"largestKillingSpree":8,"minionsKilled":175.73481222027632,"neutralMinionsKilledTeamJungle":6.87918531491211,"neutralMinionsKilledEnemyJungle":1.9552831290134267,"goldEarned":11840,"overallPosition":1,"overallPositionChange":0}},{"key":"LeeSin","role":"Jungle","title":"Lee Sin","general":{"winPercent":0.47603732897085066,"playPercent":0.11936072603416044,"banRate":0.016735176155369534,"experience":11.93326860841424,"kills":6.229476502082094,"deaths":5.414578375966687,"assists":7.773758179654967,"totalDamageDealtToChampions":12340,"totalDamageTaken":26015,"totalHeal":7518,"largestKillingSpree":8,"minionsKilled":25.68465571088638,"neutralMinionsKilledTeamJungle":71.98670806067817,"neutralMinionsKilledEnemyJungle":8.807053093396787,"goldEarned":10255,"overallPosition":6,"overallPositionChange":0}},{"key":"Thresh","role":"Support","title":"Thresh","general":{"winPercent":0.4940108608284812,"playPercent":0.11318544159496746,"banRate":0.012075421458170381,"experience":10.618539868530172,"kills":1.975553333725421,"deaths":5.545197906251838,"assists":12.660628516536297,"totalDamageDealtToChampions":6957

Answer 1

数据是json格式的。你可以这样做：

import json

stats = soup.find_all('script')
for s in stats:
    if s.string and "matchupData.stats" in s.string:
        target = s.string.strip().split(" = ")[1][:-1]        
json.loads(target)

输出：

[{'key': 'Ezreal',
  'role': 'ADC',
  'title': 'Ezreal',
  'general': {'winPercent': 0.5046896283323604,
   'playPercent': 0.17104628134933184,
   'banRate': 0.03167301835610511,

等等，等等