Get All Follower IDs in Twitter by Tweepy

Question

Is it possible to get the full follower list of an account who has more than one million followers, like McDonald's?

I use Tweepy and follow the code:

c = tweepy.Cursor(api.followers_ids, id = 'McDonalds')
ids = []
for page in c.pages():
     ids.append(page)

I also try this:

for id in c.items():
    ids.append(id)

But I always got the 'Rate limit exceeded' error and there were only 5000 follower ids.

Answer 1

The answer from alecxe is good, however no one has referred to the docs. The correct information and explanation to answer the question lives in the Twitter API documentation. From the documentation :

Results are given in groups of 5,000 user IDs and multiple “pages” of results can be navigated through using the next_cursor value in subsequent requests.

Answer 2

In order to avoid rate limit, you can/should wait before the next follower page request. Looks hacky, but works:

import time
import tweepy

auth = tweepy.OAuthHandler(..., ...)
auth.set_access_token(..., ...)

api = tweepy.API(auth)

ids = []
for page in tweepy.Cursor(api.followers_ids, screen_name="McDonalds").pages():
    ids.extend(page)
    time.sleep(60)

print len(ids)

Hope that helps.

Answer 3

Use the rate limiting arguments when making the connection. The api will self control within the rate limit.

The sleep pause is not bad, I use that to simulate a human and to spread out activity over a time frame with the api rate limiting as a final control.

api = tweepy.API(auth, wait_on_rate_limit=True, wait_on_rate_limit_notify=True, compression=True)

also add try/except to capture and control errors.

example code https://github.com/aspiringguru/twitterDataAnalyse/blob/master/sample_rate_limit_w_cursor.py

I put my keys in an external file to make management easier.

https://github.com/aspiringguru/twitterDataAnalyse/blob/master/keys.py