在googleappengine中使用urllib2引发“等待HTTP响应时超过了截止时间”网址：。。。"

Question

我在python中为googleappengine（GAE）使用urllib2。 应用程序经常会因为以下错误而崩溃：

Deadline exceeded while waiting for HTTP response from URL: ....

来源如下：

import webapp2
import urllib2
from bs4 import BeautifulSoup

def functionRunning2To5Seconds_1()    
    #Check if the Url could be parsed
    try:
        url         ="http://...someUrl..."
        req         = urllib2.Request(url,headers={'User-Agent': 'Mozilla/5.0'})
        page        = urllib2.urlopen(req)
        htmlSource  = BeautifulSoup(page)
    except Exception  e:
        logging.info("Error : {er}".format(er=str(e)))

    #do some calculation with the data of htmlSource, which takes 2 To 5 Seconds

#and the handler looks like:
class xyHandler(webapp2.RequestHandler):
    def post(self, uurl=None):
        r_data1 = functionRunning2To5Seconds_1()
        r_data2 = functionRunning2To5Seconds_2()
        r_data3 = functionRunning2To5Seconds_3()
        ...
        #show the results in a web page

我找到了这个doc，上面写着：

You can use the Python standard libraries urllib, urllib2 or httplib to make HTTP requests. When running in App Engine, these libraries perform HTTP requests using App Engine's URL fetch service

还有这个：

You can set a deadline for a request, the most amount of time the service will wait for a response. By default, the deadline for a fetch is 5 seconds. The maximum deadline is 60 seconds for HTTP requests and 60 seconds for task queue and cron job requests.

那我该怎么做呢？如何在urllib2上设置超时？在

或者，我必须重写整个应用程序才能使用appengine的URL获取服务吗？在

（注：有人知道并行运行“r\u data1=functionRunning2To5Seconds_…（）”调用的安全方法吗？）在

Answer 1

https://docs.python.org/2/library/urllib2.html

urllib2.urlopen(url[, data][, timeout])

The optional timeout parameter specifies a timeout in seconds for blocking operations like the connection attempt (if not specified, the global default timeout setting will be used).

Answer 2

正如Paul建议的，可以传递timeout参数。在appengine上，它与URL获取绑定，并将其截止时间调整为最长60秒。请记住，如果urlopen花费的时间超过了timeout参数中指定的时间，那么将从谷歌.appengine.api.urlphetch_errors.DeadlineExceededError错误而不是平常套接字超时. 最好捕获此错误并在必要时重试/记录。有关处理DeadlineExceedError的更多信息，请参见[1]。在

[1]-https://developers.google.com/appengine/articles/deadlineexceedederrors