How to limit download rate of HTTP requests in req

Question

Is it possible to limit the download rate of GET requests using the requests Python library? For instance, with a command like this:

r = requests.get('https://stackoverflow.com/')

...is it possible to limit the download rate? I'm hoping for something similar to this wget command:

wget --limit-rate=20k https://stackoverflow.com/

I know it's possible with urllib2. I'm asking specifically about the requests library.

Answer 1

There are several approaches to rate limiting; one of them is token bucket, for which you can find a recipe here and another one here.

Usually you would want to do throttling or rate limiting on socket.send() and socket.recv(). You could play with socket-throttle and see if it does what you need.

---

This is not to be confused with x-ratelimit rate limiting response headers, which are related to a number of requests rather than a download / transfer rate.