Does effective Cython cProfiling imply writing man

Question

I am trying to optimize some code with Cython, but cProfile is not providing enough information.

To do a good job at profiling, should I create many sub-routines func2, func3,... , func40 ?

Note below that i have a function func1 in mycython.pyx, but it has many for loops and internal manipulations. But cProfile does not tell me stats for those loops .

     2009 function calls in 81.254 CPU seconds

Ordered by: standard name

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    1    0.000    0.000   81.254   81.254 <string>:1(<module>)
    2    0.000    0.000    0.021    0.010 blah.py:1495(len)
 2000    0.000    0.000    0.000    0.000 blah.py:1498(__getitem__)
    1    0.214    0.214    0.214    0.214 mycython.pyx:718(func2)
    1   80.981   80.981   81.216   81.216 mycython.pyx:743(func1)
    1    0.038    0.038   81.254   81.254 {mycython.func1}
    2    0.021    0.010    0.021    0.010 {len}
    1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

Answer 1

You need to enable profiling support for your Cython code. Use

# cython: profile=True

http://docs.cython.org/src/tutorial/profiling_tutorial.html

Answer 2

Yes, it does. The finest granularity available to cProfile is a function call. You must split up func1 into multiple functions. (Note that you can make them functions defined inside func1 and thus only available to func1.)

If you want finer-grained profiling (line-level), then you need a different profiler. Take a look at this line-level profiler, but I don't think it works for Cython.