值错误：使用媒体管道时请求url:h中缺少方案

Question

我试图从一个网站下载pdf，我遵循了scrapy网站提供的说明，但我得到了这个错误：

File "/home/joseph/ENV/lib/python3.5/site-packages/scrapy/http/request/__init__.py", line 58, in _set_url
    raise ValueError('Missing scheme in request url: %s' % self._url)
ValueError: Missing scheme in request url: h
2017-09-12 17:47:40 [scrapy.core.scraper] ERROR: Error processing {'file_urls': 'https://www.sec.gov/divisions/corpfin/cf-noaction/2008/jpmorgan080409-405.pdf',
 'title': ('JPMorgan Chase & Co.',)}

设置.py

^{pr2}$

项目.py

import scrapy

class LetterItem(scrapy.Item):
    title = scrapy.Field()
    file_urls = scrapy.Field()
    files = scrapy.Field()

蜘蛛网.py

import scrapy
from sec_scrape.items import LetterItem

class QuotesSpider(scrapy.Spider):
    name = "corporate_finance"
    allowed_domains = ["sec.gov"]
    start_urls = ['https://www.sec.gov/divisions/corpfin/cf-noaction.shtml']

def parse(self, response):
    for letter in response.xpath('//table[2]/tr/td[3]/ul[74]/li/a'):
        item = LetterItem()
        item['title'] = letter.xpath('text()').extract_first(),
        item['file_urls'] = response.urljoin(letter.xpath('@href').extract_first())
        yield item

知道我为什么会犯这个错误吗？在

谢谢你

Answer 1

file_urlsitem属性必须是list，而将其设置为string（要下载的文件的URL）。换乘路线

item['file_urls'] = response.urljoin(letter.xpath('@href').extract_first())

到

^{pr2}$