ImportError:没有名为toripchanger的模块,如何使用正确的python版本启动scrapy

2024-10-05 14:28:09 发布

您现在位置:Python中文网/ 问答频道 /正文

python3不是我的默认版本。我想使用它,因为我要使用的包toripchanger只在Python3下可用。

所以我的pip3版本是:

C:\Users\Truc>pip3 -V
pip 18.1 from c:\python\python37\lib\site-packages\pip (python 3.7)

我用的是蜘蛛屑我曾经在Python2.7版本中运行,它运行得非常好,但是我知道我使用3.7来使用toripchanger我在文件的顶部对其进行了精确的描述:

^{pr2}$

上面的cd6{cd6>

middlewares.py中,我添加了一些行:

...
from scrapy.conf import settings
from toripchanger import TorIpChanger
...
ip_changer = TorIpChanger(reuse_threshold=3)

class ProxyMiddleware(object):
    _requests_count = 0

    def process_request(self, request, spider):
        self._requests_count += 1
        if self._requests_count > 10:
            self._requests_count = 0 
            ip_changer.get_new_ip()

        request.meta['proxy'] = settings.get('HTTP_PROXY')
        spider.log('Proxy : %s' % request.meta['proxy'])

当我运行spider scrapy crawl spider时,问题是我得到了以下错误:

File "C:\Users\Truc\...\middlewares.py", line 13, 
in <module> from toripchanger import TorIpChanger
ImportError: No module named toripchanger

您可能会在本主题的末尾看到整个错误行

对我来说,问题是我尝试了topic中提供的内容,所以我做了pip3 install setuptools甚至pip3 install --upgrade setuptools,我做了{},因为在我的python3.7版本中有必要安装scrapy

我的文件里有14行是错误的

[in] scrapy crawl spider
[out]

2019-02-18 14:27:11 [scrapy.utils.log] INFO: Scrapy 1.5.0 started (bot:I_AM_A_POLITE_ROBOT)

2019-02-18 14:27:11 [scrapy.utils.log] INFO: Versions: lxml 4.2.1.0, libxml2 2.9.5, 
cssselect 1.0.3, parsel 1.4.0, 
w3lib 1.19.0, Twisted 17.9.0, 
Python 2.7.9 (default, Dec 10 2014, 12:24:55) [MSC v.1500 32 bit (Intel)], 
pyOpenSSL 17.5.0 (OpenSSL 1.1.0g  2 Nov 2017), 
cryptography 2.1.4, Platform Windows-7-6.1.7601-SP1
2019-02-18 14:27:11 [scrapy.crawler] INFO: Overridden settings
{'NEWSPIDER_MODULE': 'folder.spiders', 
'ROBOTSTXT_OBEY': True, 'SPIDER_MODULES': ['folder.spiders'],
'RETRY_TIMES': 5, 'BOT_NAME': 'I_AM_A_POLITE_ROBOT',
'RETRY_HTTP_CODES': [401, 403, 404, 408, 500, 502, 503, 504],
'AUTOTHROTTLE_ENABLED': True,'DOWNLOAD_DELAY': 2}

2019-02-18 14:27:11 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.logstats.LogStats',
'scrapy.extensions.telnet.TelnetConsole',
'scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.throttle.AutoThrottle']

2019-02-18 14:27:11 [py.warnings] WARNING: 
c:\python27\lib\site-packages\scrapy\utils\deprecate.py:
156:ScrapyDeprecationWarning: 
`scrapy.contrib.downloadermiddleware.useragent.UserAgentMiddleware` 
class is deprecated, use
`scrapy.downloadermiddlewares.useragent.UserAgentMiddleware`
instead ScrapyDeprecationWarning)
Unhandled error in Deferred:

2019-02-18 14:27:11 [twisted] CRITICAL: Unhandled error in Deferred:

2019-02-18 14:27:11 [twisted] CRITICAL:
Traceback (most recent call last):
File "c:\python27\lib\site-packages\twisted\internet\defer.py", line 1386, in
_inlineCallbacks
result = g.send(result)
File "c:\python27\lib\site-packages\scrapy\crawler.py", line 98, in crawl
six.reraise(*exc_info)
File "c:\python27\lib\site-packages\scrapy\crawler.py", line 80, in crawl
self.engine = self._create_engine()
File "c:\python27\lib\site-packages\scrapy\crawler.py", line 105, in _create_engine
return ExecutionEngine(self, lambda _: self.stop())
File "c:\python27\lib\site-packages\scrapy\core\engine.py", line 69, in __init__self.downloader = downloader_cls(crawler)
File "c:\python27\lib\site-packages\scrapy\core\downloader\__init__.py", line88, in __init__self.middleware = DownloaderMiddlewareManager.from_crawler(crawler)
File "c:\python27\lib\site-packages\scrapy\middleware.py", line 58, in from_crawler
return cls.from_settings(crawler.settings, crawler)
File "c:\python27\lib\site-packages\scrapy\middleware.py", line 34, in from_settings
mwcls = load_object(clspath)
File "c:\python27\lib\site-packages\scrapy\utils\misc.py", line 44, in load_object
mod = import_module(module)
File "c:\python27\lib\importlib\__init__.py", line 37, in import_module__import__(name)
File "C:\Users\...\middlewares.py", line 13, in <module>
from toripchanger import TorIpChanger
ImportError: No module named toripchanger

Tags: infrompyimportselflibpackagesline
2条回答

好吧,如果您在3.7之后安装了Python2.7,那么Windows默认情况下会将最新安装的Python版本作为默认版本。在

Windows将始终使用默认的python。在

https://docs.python.org/3/faq/windows.html#how-do-i-make-python-scripts-executable

要在不更改Windows默认值的情况下使用其他python版本,必须执行以下操作(具体路径将因系统设置而异):

C:\Program Files\[your python path]\python.exe "myfile.py"

相关问题 更多 >