How to benchmark Matlab processes?

Question

Searching for an idea how to avoid using loop in my Matlab code, I found following comments under one question on SE:

The statement "for loops are slow in Matlab" is no longer generally true since Matlab...euhm, R2008a?

and

Have you tried to benchmark a for loop vs what you already have? sometimes it is faster than vectorized code...

So I would like to ask, is there commonly used way to test the speed of a process in Matlab? Can user see somewhere how much time the process takes or the only way is to extend the processes for several minutes in order to compare the times between each other?

Answer 1

I think that I am right to state that many of us time Matlab by wrapping the block of code we're interested in between tic and toc. Furthermore, we take care to ensure that the total time is of the order of 10s of seconds (rather than 1s of seconds or 100s of seconds) and repeat it 3 - 5 times and take some measure of central tendency (such as the mean) and draw our conclusions from that.

If the piece of code takes less than, say 10s, then repeat it as many times as necessary to bring it into the range, being careful to avoid any impact of one iteration on the next. And if the code naturally takes 100s of seconds or longer, either spend longer on the testing or try it with artificially small input data to run more quickly.

In my experience it's not necessary to run programs for minutes to get data on average run time with acceptably low variance. If I run a program 5 times and one (or two) of the results is wildly different from the mean I'll re-run it.

Of course, if the code has any features which make its run time non-deterministic then it's a different matter.

Answer 2

You can use the profiler to assess how much time your functions, and the blocks of code within them, are taking.

>> profile on; % Starts the profiler
>> myfunctiontorun( ); % This can be a function, script or block of code
>> profile viewer; % Opens the viewer showing you how much time everything took

Viewer also clears the current profile data for next time.

Bear in mind, profile does tend to slow execution a bit, but I believe it does so in a uniform way across everything.

Obviously if your function is very quick, you might find you don't get reliable results so if you can run it many times or extend the computation that would improve matters.

If it's really simple stuff you're testing, you can also just time it using tic and toc:

>> tic; % Start the timer
>> myfunctionname( );
>> toc; % End the timer and display elapsed time

Also if you want multiple timers, you can assign them to variables:

>> mytimer = tic;
>> myfunctionname( );
>> toc(mytimer);

Finally, if you want to store the elapsed time instead of display it:

>> myresult = toc;

Answer 3

The best tool for testing the performance of MATLAB code is Steve Eddins' timeit function, available here from the MATLAB Central File Exchange.

It handles many subtle issues related to benchmarking MATLAB code for you, such as:

• ensuring that JIT compilation is used by wrapping the benchmarked code in a function
• warming up the code
• running the code several times and averaging

---

Update: As of release R2013b, timeit is part of core MATLAB.

---

Update: As of release R2016a, MATLAB also includes a performance testing framework that handles the above issues for you in a similar way to timeit.