使用HTTP Post和请求库在Cloudflare后面使用API

2024-09-29 17:20:31 发布

您现在位置:Python中文网/ 问答频道 /正文

我对Python有点陌生,但我是Mathematica(Wolfram)语言的专家,所以我不是一个太多的noob程序员。我做了很多网站刮,并已成功地在它。最近,我成功抓取的一个网站发生了变化,现在我无法使用他们的API

一般来说,在使用API时,我的方法是打开Chrome并使用inspector查找XHR调用。然后将该调用复制为cURL,然后使用此site将其转换为python请求调用。即使是在cloudflare背后的站点,这在过去也发挥了神奇的作用。对于cloudflare背后的站点,我将手动登录并将cookie复制到代码中。这一直有效,直到该网站更改了他们的代码

我完成了这个过程,下面是我的python请求代码。我正在努力清理https://sportsbet.io/sports网站。这个特别的电话是为了得到所有的篮球联赛

import requests

headers = {
    'authority': 'sportsbet.io',
    'accept': '*/*',
    'authorization': '',
    'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36',
    'content-type': 'application/json',
    'origin': 'https://sportsbet.io',
    'sec-fetch-site': 'same-origin',
    'sec-fetch-mode': 'cors',
    'sec-fetch-dest': 'empty',
    'referer': 'https://sportsbet.io/sports/basketball/inplay',
    'accept-language': 'en-US,en;q=0.9'
    }

data = '${"operationName":"SportEventListQuery","variables":{"language":"en","site":"sportsbet","slug":"basketball","timePeriod":"LIVE","leagueTournaments":"LIVE","featuredLeagueTournaments":"LIVE","tournamentEventCount":"LIVE"},"query":"query SportEventListQuery($language: String\\u0021, $slug: String\\u0021, $timePeriod: SportsbetNewGraphqlSportLeagues\\u0021, $leagueTournaments: SportsbetNewGraphqlLeagueTournaments\\u0021, $featuredLeagueTournaments: SportsbetNewGraphqlFeaturedLeagueTournaments\\u0021, $tournamentEventCount: SportsbetNewGraphqlTournamentEventCount\\u0021) {\\\\n sportsbetNewGraphql {\\\\n id\\\\n getSportBySlug(slug: $slug) {\\\\n id\\\\n featuredLeague {\\\\n id\\\\n name(language: $language)\\\\n tournaments(childType: $featuredLeagueTournaments) {\\\\n id\\\\n name(language: $language)\\\\n eventCount(childType: $tournamentEventCount)\\\\n league {\\\\n id\\\\n name(language: $language)\\\\n __typename\\\\n }\\\\n __typename\\\\n }\\\\n __typename\\\\n }\\\\n name(language: $language)\\\\n leagues(childType: $timePeriod) {\\\\n id\\\\n name(language: $language)\\\\n slug\\\\n tournaments(childType: $leagueTournaments) {\\\\n id\\\\n name(language: $language)\\\\n eventCount(childType: $tournamentEventCount)\\\\n __typename\\\\n }\\\\n __typename\\\\n }\\\\n __typename\\\\n }\\\\n __typename\\\\n }\\\\n}\\\\n"}'

response = requests.post('https://sportsbet.io/graphql', headers=headers, data=data)
print(response)

我得到403个错误。正在寻找有关如何使用(刮取)此API的一些指导


Tags: namehttpsioapiidlive网站site

热门问题