scrapy中xpath返回的空列表

Question

我正在研究scrapy，我试图从一个网站收集一些数据

蜘蛛码

class NaaptolSpider(BaseSpider):
    name = "naaptol"
    domain_name = "www.naaptol.com"
    start_urls = ["http://www.naaptol.com/buy/mobile_phones/mobile_handsets.html"]

    def parse(self, response):
        hxs = HtmlXPathSelector(response)
        cell_matter = hxs.select('//div[@class="gridInfo"]/div[@class="gridProduct gridProduct_special"]')
        items=[]
        for i in cell_matter:
               cell_names = i.select('//p[@class="proName"]/a/text()').extract()
               prices = i.select('//p[@class="values"]/strong/text()').extract()
               item = ExampleItem()
               item['cell_name'] = cell_names
               item['price'] = prices
               items.append(item) 
        return [FormRequest(url="http://www.naaptol.com/faces/jsp/search/searchResults.jsp",
            formdata={'type': 'cat_catlg',
            'catid': '27',
            'sb' : '9,8',
            'frm' : '1',
            'max' : '15',
            'req': 'ajax'
            },
            callback=self.parse_item
            )]

def parse_item(self, response):
     hxs = HtmlXPathSelector(response) 
     cell_matter = hxs.select('//div[@class="gridInfo"]/div[@class="gridProduct gridProduct_special"]')
     for i in cell_matter:
               cell_names = i.select('//p[@class="proName"]/a/text()').extract()
               prices = i.select('//p[@class="values"]/strong/text()').extract()
               print cell_names
               print prices 

结果：

^{pr2}$

实际上，我发布表单是为了实现javascript分页

这里我收到parse_item方法中parse方法的响应，但是当我使用与parse方法相同的xpath时，它会返回一个空列表，有人能告诉我为什么它返回一个空数组，我的代码有什么问题吗。在

提前谢谢

Answer 1

响应采用JSON格式：

{
  "prodList": [
    {
      "pid": "955492",
      "pnm": "Samsung Star 3 Duos",
      "mctid": "27",
      "pc": "5,650",
      "mrp": "6290",
      "pdc": "10",
      "pimg": "Samsung-Star-3-duos-1.jpg",
      "rt": "8",
      "prc": "1",
      "per": "Y",
      (...)
    },
    (...)
}

为了解析它，可以使用python的json模块。下面是您要实现的一个示例：Empty list for hrefs to achieve pagination through JavaScript onclick functions。在