当我运行下面的代码时,我得到以下错误:http://pastebin.com/AJqRxCpM 我的设置:http://pastebin.com/9nzaXLJs
这是我的代码:
import scrapy
import requests
from scrapy.http import Request
import scrapy
from FinalSpider.items import Page # Defined in items.py
URL = "http://url.com/PopUp.aspx?IDCoupon=%d"
starting_number = 60000
number_of_pages = 80
class FinalSpider(scrapy.Spider):
name = "FinalSpider"
allowed_domains = ['url.com']
start_urls = [URL % starting_number]
def __init__(self):
self.page_number = starting_number
def start_requests(self):
for i in range (self.page_number, number_of_pages, -1):
yield request(url = URL % i, callback=self.parse)
def parse(self, response):
filename = response.url.split("/")[-2] + '.html'
with open(filename, 'wb') as f:
f.write(response.body)
解决了。只需添加以下内容:
相关问题 更多 >
编程相关推荐