I understand the concept of what timeit
does but I am not sure how to implement it in my code.
How can I compare two functions, say insertion_sort
and tim_sort
, with timeit
?
I understand the concept of what timeit
does but I am not sure how to implement it in my code.
How can I compare two functions, say insertion_sort
and tim_sort
, with timeit
?
If you want to use
timeit
in an interactive Python session, there are two convenient options:Use the IPython shell. It features the convenient
%timeit
special function:In a standard Python interpreter, you can access functions and other names you defined earlier during the interactive session by importing them from
__main__
in the setup statement:I'll let you in on a secret: the best way to use
timeit
is on the command line.On the command line,
timeit
does proper statistical analysis: it tells you how long the shortest run took. This is good because all error in timing is positive. So the shortest time has the least error in it. There's no way to get negative error because a computer can't ever compute faster than it can compute!So, the command-line interface:
That's quite simple, eh?
You can set stuff up:
which is useful, too!
If you want multiple lines, you can either use the shell's automatic continuation or use separate arguments:
That gives a setup of
and times
If you want to have longer scripts you might be tempted to move to
timeit
inside a Python script. I suggest avoiding that because the analysis and timing is simply better on the command line. Instead, I tend to make shell scripts:This can take a bit longer due to the multiple initialisations, but normally that's not a big deal.
But what if you want to use
timeit
inside your module?Well, the simple way is to do:
and that gives you cumulative (not minimum!) time to run that number of times.
To get a good analysis, use
.repeat
and take the minimum:You should normally combine this with
functools.partial
instead oflambda: ...
to lower overhead. Thus you could have something like:You can also do:
which would give you something closer to the interface from the command-line, but in a much less cool manner. The
"from __main__ import ..."
lets you use code from your main module inside the artificial environment created bytimeit
.It's worth noting that this is a convenience wrapper for
Timer(...).timeit(...)
and so isn't particularly good at timing. I personally far prefer usingTimer(...).repeat(...)
as I've shown above.Warnings
There are a few caveats with
timeit
that hold everywhere.Overhead is not accounted for. Say you want to time
x += 1
, to find out how long addition takes:Well, it's not 0.0476 µs. You only know that it's less than that. All error is positive.
So try and find pure overhead:
That's a good 30% overhead just from timing! This can massively skew relative timings. But you only really cared about the adding timings; the look-up timings for
x
also need to be included in overhead:The difference isn't much larger, but it's there.
Mutating methods are dangerous.
But that's completely wrong!
x
is the empty list after the first iteration. You'll need to reinitialize:But then you have lots of overhead. Account for that separately.
Note that subtracting the overhead is reasonable here only because the overhead is a small-ish fraction of the time.
For your example, it's worth noting that both Insertion Sort and Tim Sort have completely unusual timing behaviours for already-sorted lists. This means you will require a
random.shuffle
between sorts if you want to avoid wrecking your timings.Example of how to use Python REPL interpreter with function that accepts parameters.
If you want to compare two blocks of code / functions quickly you could do:
lets setup the same dictionary in each of the following and test the execution time.
The setup argument is basically setting up the dictionary
Number is to run the code 1000000 times. Not the setup but the stmt
When you run this you can see that index is way faster than get. You can run it multiple times to see.
The code basically tries to get the value of c in the dictionary.
Here are my results, yours will differ.
by index: 0.20900007452246427
by get: 0.54841166886888
simply pass your entire code as an argument of timeit: