爬行蜘蛛不爬行所需的网页

Question

这是我试图爬网的网站链接。 http://search.epfoservices.in/est_search_display_result.php?pageNum_search=1&totalRows_search=72045&old_rg_id=AP&office_name=&pincode=&estb_code=&estb_name=&paging=paging 下面是我的刮痧，因为这是第一次尝试刮痧，所以请原谅愚蠢的错误。请看一看，并建议任何修改，这将使我的代码运行。你知道吗

项目.py

import scrapy


class EpfoCrawl2Item(scrapy.Item):
    # define the fields for your item here like:
    # name = scrapy.Field()
    from scrapy.item import Item, Field
    S_No = Field()
    Old_region_code = Field()
    Region_code = Field()
    Name = Field()
    Address = Field()
    Pin = Field()
    Epfo_office = Field()
    Under_Ro = Field()
    Under_Acc = Field()
    Payment = Field()
    pass

电子对焦绘图1_蜘蛛.py

import scrapy
from scrapy.selector import HtmlXPathSelector


class EpfoCrawlSpider(scrapy.Spider):
"""Spider for regularly updated search.epfoservices.in"""
name = "PfData"
allowed_domains = ["search.epfoservices.in"]
starturls = ["http://search.epfoservices.in/est_search_display_result.php?pageNum_search=1&totalRows_search=72045&old_rg_id=AP&office_name=&pincode=&estb_code=&estb_name=&paging=paging"]

def parse(self,response):
    hxs = HtmlXPathSelector(response)
    rows = hxs.select('//tr"]')
    items = []
    for val in rows:
        item = Val()
        item['S_no'] = val.select('/td[0]/text()').extract()
        item['Old_region_code'] = val.select('/td[1]/text').extract()
        item['Region_code'] = val.select('/td[2]/text()').extract()
        item['Name'] = val.select('/td[3]/text()').extract()
        item['Address'] = val.select('/td[4]/text()').extract()
        item['Pin'] = val.select('/td[5]/text()').extract()
        item['Epfo_office'] = val.select('/td[6]/text()').extract()
        item['Under_ro'] = val.select('/td[7]/text()').extract()
        item['Under_Acc'] = val.select('/td[8]/text()').extract()
        item['Payment'] = val.select('a/@href').extract()
        items.append(item)
        yield items

下面是运行“scrapy crawl PfData”后的日志

016-05-25 13:45:11+0530 [scrapy] INFO: Enabled item pipelines: 
2016-05-25 13:45:11+0530 [PfData] INFO: Spider opened
2016-05-25 13:45:11+0530 [PfData] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2016-05-25 13:45:11+0530 [scrapy] DEBUG: Telnet console listening on 127.0.0.1:6023
2016-05-25 13:45:11+0530 [scrapy] DEBUG: Web service listening on 127.0.0.1:6080
2016-05-25 13:45:11+0530 [PfData] INFO: Closing spider (finished)
2016-05-25 13:45:11+0530 [PfData] INFO: Dumping Scrapy stats:
    {'finish_reason': 'finished',
     'finish_time': datetime.datetime(2016, 5, 25, 8, 15, 11, 343313),
     'log_count/DEBUG': 2,
     'log_count/INFO': 7,
     'start_time': datetime.datetime(2016, 5, 25, 8, 15, 11, 341872)}
2016-05-25 13:45:11+0530 [PfData] INFO: Spider closed (finished)

请提供建议。你知道吗

Answer 1

开始URL列表必须是start_urls，而不是starturls