I am using python v2.7 and Requests HTTP library. I am using windows 7 OS.
I have failed to understand why the following code consumes more and more memory upon execution? I have observed the memory consumption using task manager. It would be great if some one can point out what could be the possible reason and how to avoid it? Also it would be great if some one can test this on his/her system and confirm that it is not only me or my system who is facing this problem. :) Although the memory consumption increases with a small amount, I think at some point my app will surely crash when there is no more free memory left.
Code:
import requests
def getName():
url = 'https://stackoverflow.com/users/2382792/pss'
r = requests.get(url)
print r
while True:
getName()
The reason for this simple question is provided in detail below. Also I have a similar problem with urllib2 module too. So I thought of using Requests HTTP library. But the same problem regarding the memory consumption persists.
Real world scenario: My real world app is built with python and wxPython. The wxPython code doesn't causes this problem. The python code which is connecting to the URLs to fetch some values from the server is causing the problems. To keep the values updated the values are fetched after every 2 seconds. These fetched values are then displayed on the GUI. The code shown above is the a sample of my original code. More over in above code sample there is only one instance of connecting to an URL. In my real world app I have 9-10 threads those are doing this fetching stuff by connecting to different URLs! Which means in this case the memory consumption increases more rapidly as compared to the above sample code.
I hope now it is more clear how severe this problem is. Also, just for the clarification I am not scrapping any website.
Comparison: Why the following code doesn't have the same problem as the code above:
import random
def getValue():
value = random.randrange(0,11)
print value
while True:
getValue()
Update: I tested the code snippet for half an hour. I have finally concluded that the memory consumption increases in the beginning and later it gets in a state where it decreases as well as increases. Finally the memory consumption will remain in a range. Here is a graph created from processExplorer on Windows 8 OS. The middle one is for the memory(Right click the image and click view to see it clearly):
Thank you for your time!