Speed-up nested cross-validation

Question

In order to speed-up nested cross-validation with sklearn, is it better to fix n_jobs=-1 in inner or outer loop, since nested parallelism is not allowed ?

Answer 1

This seems to be an an open question, see e.g. this open issue on scikit-learn's github page.

Another approach is to use a Message Passing Interface (MPI) to exploit multiple processors, see e.g. this blogpost using MPI4PY.