我正在尝试为一个大学项目的鞋爬行一个网站。从下面的代码中,我得到了3-4只鞋,但是在这之后,对于剩余的鞋,我得到了响应“请启用cookies”,尽管我设置了Cookies_enabled=True
和Cookies_debug=True
。问题在哪里
import scrapy
from scrapy.http import Request
class FirespiderSpider(scrapy.Spider):
name = 'firespider'
page_number = 2
allowed_domains = ['kicksonfire.com']
start_urls = ['https://www.kicksonfire.com/app/upcoming?page=1']
def parse(self, response):
shoe_links = response.xpath ('//div/a/@href').extract()[1:None]
for link in shoe_links:
yield scrapy.Request(link, callback=self.parse_page, dont_filter=True)
def parse_page(self, response):
shoe_name = response.xpath('//div/h2/text()').extract()[0].strip()
print('Schuh:' + shoe_name)
First part of the logResponse "Please enable cookies" in the log
目前没有回答
相关问题 更多 >
编程相关推荐