从数据库获取起始URL

Question

我有个问题想问
如果我想从数据库中选择链接并让它成为start_urls
我能做什么？？在

assum我的数据库保存这些链接和标题

1: link : 'http://test.com/id=1'  ,  title : 'math'   
2: link : 'http://test.com/id=2'  ,  title : 'english'       
3: link : 'http://test.com/id=30' ,  title : 'sports'    

如果我能把标题作为蜘蛛的参数会更好。在

我的想法是：

^{pr2}$

Answer 1

您需要重写^{}方法和yield/return^{}实例：

This method must return an iterable with the first Requests to crawl for this spider.

This is the method called by Scrapy when the spider is opened for scraping when no particular URLs are specified.

class MySpider(Spider):  
    name = "linktest"

    def start_requests(self):
        for obj in Learning.objects.all():
            yield Request(obj.link)

---

要将obj.title传递给回调，请使用^{}属性：

^{pr2}$