python program stops in command line

Question

I searched a lot about problem VMware with python, but I didn't find any information about my problem. My problem is that python programs freeze, process is still running but it doesn't use CPU and memory usage doesn't change. Program doesn't return any exception or anything... it just freeze and it never back to execution. It looks that it happens randomly and it is not a problem with no memory available for my machine, because in the same time I can execute it in another session.

My machine is virtual machine with: Windows Server 2008 64-bit, VMware Tools 9.4.5

I tried python: 2.7, 3.3 and 3.4

my example script: (but not only this script freeze)

print("START")
for i in range(0, 1000): 
  print("step: " + str(i)) 
  file = open("./test_file.csv", "r") #file size is 1.2GB but I have 10GB RAM
  for line in file.readlines(): 
    pass    
  file.close() #close the file
print("END")

example output is:

START
step 0
step 1
step 2
step 3
step 4

and it freeze, it is randomly on which step (sometimes 4, 15, 34...) All what I can do is kill the process and run it again. During the execution I can see that program repeatable use 1,2GB RAM and release... use and release. Freeze is always after release memory and from this time memory usage is stable and CPU usage 0% for this process.

I run the script in IDLE to play with debugger and stack viewer, but when the program freeze, the whole idle is not responding. Also I tried it on others no-virtual machine and there is no problem.

I would be grateful for any suggestions, how to solve or debug that kind o problems.

Answer 1

I solved it, the problem was not with python or VMware.... only with my knowledge about Microsoft products.

I did not mentioned that I execute python scripts using windows command line, and the reason of the "freeze" was cmd. I had no idea that cmd suspends executed task if you mark sth in command line.

Only on this virtual machine I had set up “QuickEdit Mode”. You can change it following steps: right click on panel of cmd -> properties -> tab: “Options” -> section: “QuickEdit Mode”)

In this mode if you click on command line it will mark sth and... suspend executed task… I turned off the “QuickEdit Mode” and program works perfectly. Without QuickEdit Mode you can not mark anything in command line by clicking.

I know... I should shame myself... and I do.

Answer 2

Propably you should use readline() not readlines() that reads the whole file.