Python: Why is IDLE so slow?

Question

IDLE is my favorite Python editor. It offers very nice and intuitive Python shell which is extremely useful for unit-testing and debugging, and a neat debugger.

However, code executed under IDLE is insanely slow. By insanely I mean 3 orders of magnitude slow:

bash

time echo "for i in range(10000): print 'x'," | python

Takes 0.052s,

IDLE

import datetime
start=datetime.datetime.now()
for i in range(10000): print 'x',
end=datetime.datetime.now()
print end-start

Takes:

>>> 0:01:44.853951

Which is roughly 2,000 times slower.

Any thoughts, or ideas how to improve this? I guess it has something to do with the debugger in the background, but I'm not really sure.

Adam

Answer 1

The problem is in the Tkinter Text widget, and its inefficient management of very long lines, and you create one. You'll notice that, while any part of a very long line is visible, all scrolling is devilishly slow.

Answer 2

The problem is the text output not the debugger.

I just tried it on my Q6600 (3GHz overclocked) System and my numbers are even worse. But its easy to see that they are going down the more output text is added.

I tried to run it with

1000 iterations => 7,8 sec 2000 iterations => 28,5 sec 3000 iterations => 70 sec

I did some low level TK stuff in the past and i know that the TkText Widget is keeping the text in a BTree structure. Appending text a character a time is one of the worst ways to do but this seems to be what IDLE is doing. The normal way is to catch more data and append a larger chunk of text.

Amazingly if you write print 'x\n' the output is much faster. 3000 iterations in 7 seconds and your 10000 in 19 sec.

So the problem is definitely with appending single chars to existing lines. The IDLE programmer didn't know how TkText works.

So the advise is to add more newlines into your text or output larger chunks and not only a single 'x' character.