Python10054尝试使用请求库登录航空公司网站时出错

Question

我正在学习python，作为我的第一个项目，我想登录几家航空公司的网站，收集我的飞行里程信息。我已经成功地登陆美国航空公司和美国联合航空公司，但在达美航空公司、美国航空公司和英国航空公司，我都做不到。在

我一直使用的方法是从Fiddler2、Chrome或Firebug观察网络流量。Wireshark目前看来太复杂了。在

为了让我的脚本与American和United scraping一起工作，我所做的只是观察fiddler2上的流量，复制表单数据和请求头数据，然后使用python第三方请求库来访问数据。很简单。很简单。另一家航空公司的网站给了我很多麻烦。在

让我们具体谈谈英国航空公司。下面是我登录虚拟BA帐户时从fiddler获取的表单数据和请求头数据的图片。我还包括了我一直在使用的测试脚本。我写了两个不同的版本。一个使用请求库，一个使用urllib。它们都会产生相同的错误，但我想我会提供这两种方法，以便在没有导入请求库的情况下更容易帮助我。用你喜欢的那个。在

基本上，当我请求.post我得到了

10054，“远程主机强制关闭了现有连接”错误。在

我不知道发生了什么事。找了3天，什么也没找到。我希望有人能帮助我。下面的代码是使用我的虚拟BA帐户信息。用户名：python\u noob密码：p4ssword。请随意使用和测试它。在

这是一些图片的数据

http://i.imgur.com/iOL91.jpg?1

http://i.imgur.com/meLHL.jpg?1

import requests

import urllib


def get_BA_login_using_requests ():
    url_loginSubmit1 = 'https://www.britishairways.com/travel/loginr/public/en_us'

    url_viewaccount1 = 'https://www.britishairways.com/travel/viewaccount/public/en_us?eId=106011'
    url_viewaccount2 = 'https://www.britishairways.com/travel/viewaccount/execclub/_gf/en_us?eId=106011'


    form_data = {
        'Directional_Login':'',
        'eId':'109001',
        'password':'p4ssword',
        'membershipNumber':'python_noob',
        }


    request_headers= {
        'Cache-Control':'max-age=0',
        'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
        'Accept-Charset':'ISO-8859-1,utf-8;q=0.7,*;q=0.3',
        'Accept-Encoding':'gzip,deflate,sdch',
        'Accept-Language':'en-US,en;q=0.8',
        'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.97 Safari/537.11',

        'Cookie': 'BIGipServerba.com-port80=997762723.20480.0000; v1st=EDAB42A278BE913B; BASessionA=kDtBQWGclJymXtlsTXyYtykDLLsy3KQKvd3wMrbygd7JZZPJfJz2!-1893405604!clx42al01-wl01.baplc.com!7001!-1!-407095676!clx43al01-wl01.baplc.com!7001!-1; BIGipServerba.com-port81=997762723.20736.0000; BA_COUNTRY_CHOICE_COOKIE=us; Allow_BA_Cookies=accepted; BA_COUNTRY_CHOICE_COOKIE=US; opvsreferrer=functional/home/home_us.jsp; realreferrer=; __utma=28787695.2144676753.1356203603.1356203603.1356203603.1; __utmb=28787695.1.10.1356203603; __utmc=28787695; __utmz=28787695.1356203603.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); fsr.s={"v":-2,"rid":"d464cf7-82608645-1f31-3926-49807","ru":"http://www.britishairways.com/travel/globalgateway.jsp/global/public/en_","r":"www.britishairways.com","st":"","to":3,"c":"http://www.britishairways.com/travel/home/public/en_us","pv":1,"lc":{"d0":{"v":1,"s":false}},"cd":0}',

        'Content-Length':'78',
        'Content-Type':'application/x-www-form-urlencoded',

        'Origin':'https://www.britishairways.com',
        'Referer':'https://www.britishairways.com/travel/loginr/public/en_us',

        'Connection':'keep-alive',
        'Host':'www.britishairways.com',
        }



    print ('Trying to login to British Airways using Requests Library (takes about 1 minute for error to occur)')


    try:
        r1 = requests.post(url_loginSubmit1, data = form_data, headers = request_headers)
    print ('it worked')
    except Exception as e:
        msg = "An exception of type {0} occured, these were the arguments:\n{1!r}"
        print (msg.format(type(e).__name__, e.args))


    return







def get_BA_login_using_urllib():
    """Tries to request the URL. Returns True if the request was successful; false otherwise.
    https://www.britishairways.com/travel/loginr/public/en_us

    response -- After the function has finished, will possibly contain the response to the request.

    """
    response = None
    print ('Trying to login to British Airways using urllib Library (takes about 1 minute for error to occur)')
    # Create request to URL.
    req = urllib.request.Request("https://www.britishairways.com/travel/loginr/public/en_us")

    # Set request headers.
    req.add_header("Connection", "keep-alive")
    req.add_header("Cache-Control", "max-age=0")
    req.add_header("Origin", "https://www.britishairways.com")
    req.add_header("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.97 Safari/537.11")
    req.add_header("Content-Type", "application/x-www-form-urlencoded")
    req.add_header("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
    req.add_header("Referer", "https://www.britishairways.com/travel/home/public/en_us")
    req.add_header("Accept-Encoding", "gzip,deflate,sdch")
    req.add_header("Accept-Language", "en-US,en;q=0.8")
    req.add_header("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.3")
    req.add_header("Cookie", 'BIGipServerba.com-port80=997762723.20480.0000; v1st=EDAB42A278BE913B; BIGipServerba.com-port81=997762723.20736.0000; BA_COUNTRY_CHOICE_COOKIE=us; Allow_BA_Cookies=accepted; BA_COUNTRY_CHOICE_COOKIE=US; BAAUTHKEY=BA4760A2434L; BA_ENROLMENT_APPLICATION_COOKIE=1356219482491AT; BASessionA=wKG4QWGSTggNGnsLTnrgQnMxGMyzvspGLCYpjdSZgv2pSgYN1YRn!-1893405604!clx42al01-wl01.baplc.com!7001!-1!-407095676!clx43al01-wl01.baplc.com!7001!-1; HOME_AD_DISPLAY=1; previousCountryInfo=us; opvsreferrer=functional/home/home_us.jsp; realreferrer=; __utma=28787695.2144676753.1356203603.1356216924.1356219076.6; __utmb=28787695.15.10.1356219076; __utmc=28787695; __utmz=28787695.1356203603.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); fsr.s={"v":-2,"rid":"d464cf7-82608645-1f31-3926-49807","ru":"http://www.britishairways.com/travel/globalgateway.jsp/global/public/en_","r":"www.britishairways.com","st":"","to":5,"c":"https://www.britishairways.com/travel/home/public/en_us","pv":31,"lc":{"d0":{"v":31,"s":true}},"cd":0,"f":1356219889982,"sd":0}')

    # Set request body.
    body = b"Directional_Login=&eId=109001&password=p4ssword&membershipNumber=python_noob"

    # Get response to request.


    try:
        response = urllib.request.urlopen(req, body)
        print ('it worked')
    except Exception as e:
        msg = "An exception of type {0} occured, these were the arguments:\n{1!r}"
        print (msg.format(type(e).__name__, e.args))

    return



def main():
    get_BA_login_using_urllib()
    print()
    get_BA_login_using_requests()
    return


main()

Answer 1

顺便说一句，我想说你成功地创建了一个格式错误或非法的请求，而另一方的服务器（甚至代理）只是拒绝处理它。在

1. 使用requests库。太棒了。Urllib已经过时了（而且，使用起来一点也不好玩）

2. 去掉几乎所有的自定义标题。尤其是Content-Length、Keep-Alive、Connection和{}。前三个应该由请求库处理，因为它们是http1.1协议的一部分。关于Cookie：这也将由requests库处理，这取决于您如何使用会话。（您可能需要查阅那里的文档。）如果没有以前的cookies，当您尝试访问站点时，您可能会得到类似401的信息，或者您将（透明地）重定向到登录页面。登录将设置正确的cookies，之后您应该可以重新尝试原始请求。

3. 如果对post数据使用dict，则也不需要Content-Type头。你可能想尝试在said dict中使用unicode值，我发现这有时会产生影响。

换言之：尽可能多地删除，然后从那里构建它。做这样的事情通常不会花费太多的线。现在，刮网页，那是另一回事：试试“beauthulsoup”。在

注意：千万不要在公共论坛上发布cookie数据：它们可能包含一些可疑人物可能会滥用的个人或其他敏感数据。在

Answer 2

在python3.3的windows版本中似乎有一个bug是我问题的原因。我从这里得到了答案

HTTPS request results in reset connection in Windows with Python 3

在我的脚本的urllib版本上取得进展。我想使用请求，所以我需要弄清楚如何使用该模块进行SSL降级工作。我会把那条线分开。如果有人对此有答案，你也可以在这里发表。谢谢。在