<p>如果页面上没有找到某些xpath,我需要能够重试请求。所以我编写了这个中间件:</p>
<pre><code>class ManualRetryMiddleware(RetryMiddleware):
def process_response(self, request, response, spider):
if not spider.retry_if_not_found:
return response
if not hasattr(response, 'text') and response.status != 200:
return super(ManualRetryMiddleware, self).process_response(request, response, spider)
found = False
for xpath in spider.retry_if_not_found:
if response.xpath(xpath).extract():
found = True
break
if not found:
return self._retry(request, "Didn't find anything useful", spider)
return response
</code></pre>
<p>并在<code>settings.py</code>中注册:</p>
^{pr2}$
<p>当我跑蜘蛛的时候</p>
<pre><code>AttributeError: 'Response' object has no attribute 'xpath'
</code></pre>
<p>我试图手动创建选择器并在其上运行xpath。。。但是响应没有<code>text</code>属性,<code>response.body</code>是字节,而不是str。。。在</p>
<p>那么如何在中间件中检查页面内容呢?有可能有些页面不包含我需要的详细信息,所以我想稍后再试一次。在</p>