只输出一个打开的b

Question

我正在尝试在数学/科学/经济学页面下刮取所有可汗学院页面的标题和URL。然而，目前它只输出一个开放的括号，在这之前，它只会刮开始网址。你知道吗

from openbar_index.items import OpenBarIndexItem
from scrapy.contrib.spiders import CrawlSpider, Rule
from scrapy.contrib.linkextractors.sgml import SgmlLinkExtractor


class OpenBarSpider(CrawlSpider):
    """
    scrapes website URLs from educational websites and commits urls/webpage names/text to a document
    """

    name = 'openbar'
    allowed_domains = 'khanacademy.org'
    start_urls = [

        "https://www.khanacademy.org"

    ]

     rules = [

            Rule(SgmlLinkExtractor(allow = ['/math/']), callback='parse_item', follow = True),
             Rule(SgmlLinkExtractor(allow = ['/science/']), callback='parse_item', follow=True),
             Rule(SgmlLinkExtractor(allow = ['/economics-finance-domain/']), callback='parse_item', follow=True)
    ]

    def parse_item(self, response):

         item = OpenBarIndexItem()
         url = response.url
         item['url'] = url
         item['title'] = response.xpath('/html/head/title/text()').extract()
         yield item

有人知道为什么会发生这种情况，或者知道如何解决吗？你知道吗

Answer 1

问题是对allowed_domains的赋值。根据documentation，这不能是string，而是list。由于没有有效的域，因此使用该字符串，可能的结果将被scrapy过滤为异地请求。你知道吗

所以在下一行加上方括号应该可以解决这个问题

    allowed_domains = ['khanacademy.org']