我正在复习那篇无聊的教程http://doc.scrapy.org/en/latest/intro/tutorial.html 我一直跟着它直到我执行这个命令
scrapy crawl dmoz
它给了我一个错误的输出
^{pr2}$我对python不是很熟悉,也不知道它在抱怨什么
这是我的domz_蜘蛛网.py文件
from scrapy.spider import BaseSpider
class DmozSpider(BaseSpider):
name = "dmoz"
allowed_domains = ["dmoz.org"]
start_urls = [
"http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
"http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"
]
def parse(self, response):
filename = response.url.split("/")[-2]
open(filename, 'wb').write(response.body)
And here is my items file
# Define here the models for your scraped items
#
# See documentation in:
# http://doc.scrapy.org/en/latest/topics/items.html
from scrapy.item import Item, Field
class DmozItem(Item):
title = Field()
link = Field()
desc = Field()
这是目录结构
scrapy.cfg
tutorial/
tutorial/items.py
tutorial/pipelines.py
tutorial/settings.py
tutorial/spiders/
tutorial/spiders/domz_spider.py
这是设置.py文件
BOT_NAME = 'tutorial'
SPIDER_MODULES = ['tutorial.spiders']
NEWSPIDER_MODULE = 'tutorial.spiders'
好吧,我发现这个解决了这个问题
sudo pip安装升级zope.接口在
我不知道一旦发出这个命令,发生了什么,但这解决了我的问题,现在我看到了这一点
相关问题 更多 >
编程相关推荐