cloudscraper.exceptions.CloudflareChallengeError:检测到Cloudflare版本2挑战。将cloudscraper模块与python一起使用时出错

2024-10-02 20:37:02 发布

您现在位置：Python中文网/ 问答频道 /正文
8563 3

因此，我试图绕过网站的cloudflare保护，从中删除一些项目，但Cloudscraperpython模块不起作用

无论何时运行，我都会收到以下错误：

cloudscraper.exceptions.CloudflareChallengeError: Detected a Cloudflare version 2 challenge, This feature is not available in the opensource (free) version.

下面是我正在使用的简化代码：

import cloudscraper
from bs4 import BeautifulSoup as soup


url = "http://adventurequest.life/"
scraper = cloudscraper.create_scraper()
html = scraper.get(url).text
page_soup = soup(html, "html.parser")
print(page_soup)

你们知道怎么解决这个问题吗


Tags： 模块项目importurl网站versionhtml错误
1条回答
网友
1楼 · 发布于 2024-10-02 20:37:02

我在使用scrapy+cloudscraper时遇到了相同的错误，但随后我将cookie_enable=true设置得很好：

错误

Traceback (most recent call last):
cloudscraper.exceptions.CloudflareChallengeError: Detected a Cloudflare version 2 Captcha challenge, This feature is not available in the opensource (free) version.
2021-04-27 09:59:30 [scrapy.core.scraper] ERROR: Error downloading <GET https://www.forever21.com/us/shop/catalog/category/f21/lingerie>
Traceback (most recent call last):
StopIteration: <403 
https://www.forever21.com/us/shop/catalog/category/f21/lingerie>

之前：

import cloudscraper

browser = cloudscraper.create_scraper()

# in middleware
req = spider.browser.get(url,
                         proxies={'http': proxy,
                                  'https': https_proxy
                                  headers={'referer': url},
                         )

之后：

'COOKIES_ENABLED': True

但是在bs4中，Cookies是默认添加的，所以我尝试了你的代码，发现它是正常的

url = "http://adventurequest.life/"
scraper = cloudscraper.create_scraper()
html = scraper.get(url).text
page_soup = soup(html, "html.parser")
print(page_soup)

<!DOCTYPE doctype html>
<html lang="en" style="min-height: 100%;">
<head>
<!  Required meta tags  >
<meta charset="utf-8"/>
<meta content="width=device-width, initial-scale=1, shrink-to-fit=no" name="viewport"/>
<meta content="Auto Quest Worlds" name="twitter:title"/>
<meta content="aqw bots, adventure quest bots, aqw cheat, aqw hack, aqw exploits, grimoire download, adventure quest worlds bot, leveling bot aqw, botting mmorpg, aqw private server, aqworlds private server, aqw server, aqw ps, aqw private, skidson, aqw pirata, servidor de aqw, adventure quest worlds private, dragonfable private server, adventure quest private server, free to play mmorpg, free online games, browser games, jogos online, jogos criancas, jogos de navegador, best aqw private server, best online mmorpg, best browser mmorpg, habbo servidor privado, habbo retro, habbo private server, runescape private server, high rates aqw, aqw items, aqworlds wiki" name="keywords"/>
<meta content="https://adventurequest.life/" name="twitter:url"/>

也许您应该检查您的机器opennssl版本，然后更新或升级cloudscraper版本

我的cloudscraper版本是：cloudscraper====>；1.2.58

相关问题 更多 >

    编程相关推荐