了解提升远程断开连接（“远程端闭合连接”

Question

我正在抓取twitter，试图得到twitter用户列表中的好友/用户。我在OSX10.13上使用tweepy和Python3.6.5。缩写代码块：

def get_friends_for_each_twitter_user(UserL=None, Name=None):
   .
   . # Auth keys and such
   .
   for user in UserL:  ### This is a list of USER class with the below fields ###
        ### Handle protected users ###
        if(user.protected == True):
            user.friendsL = "protected"
            continue
        screenNameL=[]
        friendIDL=[]
        friendL=[]
        friendScreenNameL=[]
        ### Get IDs of people that this user follows (i.e. 'friends') ###
        for page in tweepy.Cursor(api.friends_ids, screen_name=user.screenName).pages():
            friendIDL.extend(page)
            time.sleep(60)
        ## Loop through IDs, get user profile, keep only friends' screen name ###
        for i in range(0, len(friendIDL), 100):
            friendL.extend(api.lookup_users(user_ids=friendIDL[i:i+100]))
        ### Keep only screen name ###
        for friend in friendL:
            friendScreenNameL.append(friend._json['screen_name'])
        user.friendsL = friendScreenNameL

当我这样做时，在收集了大约12个用户的friends（即用户遵循的配置文件）之后，我得到以下错误：

^{pr2}$

错误似乎发生在 get_friends_for_each_twitter_user()函数

问题：

1. 为什么会出现这种错误？

2. 我如何避免/绕过它？

Answer 1

在搞乱了一段时间后，我相信这个问题是由我的网络连接引起的。当这一切发生时，我连接到了我的5GHz无线网络。当我连接到我的2.4GHz无线网络时，这些错误就不那么频繁了。在这种情况下，正确的做法是处理异常，等待几秒钟，然后重试。下面是适当的代码片段：

def get_friends_for_each_twitter_user(UserL=None, Name=None):
    consumerKey =  #your value here
    consumerSecret = #your value here
    auth = tweepy.AppAuthHandler(consumerKey, consumerSecret)  ### Supposedly faster
    api = tweepy.API(auth, wait_on_rate_limit=True, wait_on_rate_limit_notify=True)  ## Now I don't have to handle rate limiting myself

    for user in UserL:
        accountStatus = 'active'
        if(user.protected == True):
            user.friendsL = "protected"
            continue
        screenNameL=[]
        friendIDL=[]
        friendL=[]
        friendScreenNameL=[]
        #### TWITTER LIMITS US #####
        try :
            for page in tweepy.Cursor(api.friends_ids, screen_name=user.screenName).pages():
                friendIDL.extend(page)
        except tweepy.TweepError as error :
            if(error.__dict__['api_code'] == 34):
                accountStatus = 'dead'
                print("...{} is dead".format(user.screenName))
                continue
            else:
                raise

        for i in range(0, len(friendIDL), 100):
            ### This handles when exception occurs (probably due to connection issues)
            ### When exception occurs, sleeps then retries. I don't notice this error
            ### when I'm running on corporate Wifi, maybe my router just sucks
            while True:
                try :
                    friendL.extend(api.lookup_users(user_ids=friendIDL[i:i+100]))
                except tweepy.TweepError as error :
                    print("...Exception for {} : api_code {}".format(user.screenName,
                          error.__dict__['api_code']))
                    time.sleep(5)
                    continue
                break

Answer 2

任何事情都可能导致出现错误，但如果原因不是永久性的，那么重试偶尔失败的API调用可以使脚本正常工作。在

根据Tweepy docs，API客户机构造函数接受一个默认为0的retry_count参数。尝试将retry_count设置为0以上的值，然后查看脚本是否能够成功完成，如下所示：

api = tweepy.api.API(..., retry_count=3)