来自斯皮迪的mongodb

2024-09-26 22:51:58 发布

您现在位置:Python中文网/ 问答频道 /正文

我写了一个简单的脚本来测试Mongo db

import scrapy
from mango.items import MangoItem
class Quote(scrapy.Spider):

    name = "Quote"

    def start_requests(self):
        urls= ['http://quotes.toscrape.com/']
        for url in urls:
            yield scrapy.Request(url=url,callback=self.parse)

    def parse(self,response):
        item=MangoItem()
        rows = response.xpath('//div[@class="quote"]')
        for row in rows:

            item['quote'] = row.xpath('span/text()').extract_first()
            item['author'] = row.xpath('span[2]/small/text()').extract_first()
            item['tags'] = row.xpath('div[@class="tags"]/meta/@content').extract_first()
            yield item

这是我的pipline.py

import pymongo

class MangoPipeline(object):


    def __init__(self):
        self.conn = pymongo.MongoClient(
            'localhost',27017
            )
        db=self.conn['myquotes'] #create db
        self.collection = db['Quotes']#create table or adds to it if exist


    def process_item(self, item, spider):
        self.collection.insert(item)
        return item

刮下的条目正确地显示在终端中,并且不会给出任何错误,但是mongoshell中没有创建任何数据库


Tags: importselfurldbdefextractitemurls

热门问题