为什么requests.get（）不返回？requests.get（）使用的默认超时是什么？

Question

在我的脚本中，requests.get从不返回：

import requests

print ("requesting..")

# This call never returns!
r = requests.get(
    "http://www.justdial.com",
    proxies = {'http': '222.255.169.74:8080'},
)

print(r.ok)

可能的原因是什么？有什么补救措施吗？get使用的默认超时是什么？

Answer 1

来自requests documentation：

You can tell Requests to stop waiting for a response after a given number of seconds with the timeout parameter:

>>> requests.get('http://github.com', timeout=0.001)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
requests.exceptions.Timeout: HTTPConnectionPool(host='github.com', port=80): Request timed out. (timeout=0.001)

Note:

timeout is not a time limit on the entire response download; rather, an exception is raised if the server has not issued a response for timeout seconds (more precisely, if no bytes have been received on the underlying socket for timeout seconds).

在我看来，requests.get（）需要很长时间才能返回，即使timeout是1秒。有几种方法可以克服这个问题：

1。使用内部类TimeoutSauce

发件人：https://github.com/kennethreitz/requests/issues/1928#issuecomment-35811896

import requests from requests.adapters import TimeoutSauce

class MyTimeout(TimeoutSauce):
    def __init__(self, *args, **kwargs):
        if kwargs['connect'] is None:
            kwargs['connect'] = 5
        if kwargs['read'] is None:
            kwargs['read'] = 5
        super(MyTimeout, self).__init__(*args, **kwargs)

requests.adapters.TimeoutSauce = MyTimeout

This code should cause us to set the read timeout as equal to the connect timeout, which is the timeout value you pass on your Session.get() call. (Note that I haven't actually tested this code, so it may need some quick debugging, I just wrote it straight into the GitHub window.)

2。使用来自kevinburke的请求分支：https://github.com/kevinburke/requests/tree/connect-timeout

从它的文档中：https://github.com/kevinburke/requests/blob/connect-timeout/docs/user/advanced.rst

If you specify a single value for the timeout, like this:

r = requests.get('https://github.com', timeout=5)

The timeout value will be applied to both the connect and the read timeouts. Specify a tuple if you would like to set the values separately:

r = requests.get('https://github.com', timeout=(3.05, 27))

注：The change has since been merged to the main Requests project。

3。使用类似问题中已经提到的evenlet或signal：Timeout for python requests.get entire response

Answer 2

What is the default timeout that get uses?

默认超时为None，这意味着它将等待（挂起）直到连接关闭。

传入超时值时会发生什么情况？

r = requests.get(
    'http://www.justdial.com',
    proxies={'http': '222.255.169.74:8080'},
    timeout=5
)

Answer 3

回顾了所有的答案，得出了问题仍然存在的结论。在某些站点上，请求可能无限挂起，并且使用多处理似乎是过度的。以下是我的方法（Python3.5+）：

import asyncio

import aiohttp


async def get_http(url):
    async with aiohttp.ClientSession(conn_timeout=1, read_timeout=3) as client:
        try:
            async with client.get(url) as response:
                content = await response.text()
                return content, response.status
        except Exception:
            pass


loop = asyncio.get_event_loop()
task = loop.create_task(get_http('http://example.com'))
loop.run_until_complete(task)
result = task.result()
if result is not None:
    content, status = task.result()
    if status == 200:
        print(content)