PyQt5 run sklearn calculations on a separate QThre

问题:

I am creating a PyQt5 application and want to run some ML code in a separate thread when a button is clicked. I am unable to do so. This is my threading code:

newthread = QtCore.QThread()
runthread = RunThread()
runthread.moveToThread(newthread)
runthread.finished.connect(newthread.quit)
newthread.started.connect(runthread.long_run)
newthread.start()

And the class RunThread is as follows:

class RunThread(QtCore.QObject):

finished = QtCore.pyqtSignal()

def __init__(self):
    super().__init__()

@pyqtSlot()
def long_run(self):
    #ML code called in the line below
    simulation.start_simulation()
    self.finished.emit()

Running it normally doesnt work. Pycharm quits with the following error:

process finished with exit code -1073740791 (0xc0000409)

Running it in debug mode throws thousands of warnings thrown by sklearn saying:

Multiprocessing-backed parallel loops cannot be nested below threads,
setting n_jobs=1

The code runs eventually, but it takes a lot more time (at least 4x times) to do so.

Could someone please let me know what the issue here is?

回答1:

The warning from sklearn is pretty explicit here. Basically you cannot train sklearn models with hyperparameter n_jobs set to anything greater than 1 when executing from a nested thread.

That being said, without seeing what is going on inside of simulation.start_simulation() it's hard to conjecture.

My best guess would be to look for anything in start_simulation() that is using multiple jobs and see what happens when you set that to n_jobs=1.

Alternatively, you may try writing your ML code as a standalone script and execute it using QProcess. That may allow you to utilize n_jobs greater than 1.