Pyinstaller numpy “Intel MKL FATAL ERROR: Cannot l

Question

I'm new with python apps. I'm trying to build my python GUI app with pyinstaller. My app depends on the following packages: PyQt4, numpy, pyqtgraph, h5py. I'm working with WinPython-32bit-3.4.4.1.

I build the app with this command:

pyinstaller --hidden-import=h5py.defs --hidden-import=h5py.utils --hidden-import=h5py.h5ac --hidden-import=h5py._proxy VOGE.py

I launch my app with the exe file in the dist directory created by pyinstaller and it seems work fine until the program call numpy and crash with this error:

Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll

The mkl_intel_thread.dll is not present in the software directory; but with the file copied in the root dir of the program I got the same error

Thanks for your help

Answer 1

I had the same issue using Pyinstaller and Numpy. By default pyinstaller seems to not take into account numpy binaries so you have to specify it manually. You can add the files editing the ".spec" file "binaries" variable, but that will only work for your current program. If you want it working for all the programs your freeze you should make a "hook" and save it in C:\Python3*\Lib\site-packages\PyInstaller\hooks.

I had to adapt LeonidR's code to make the numpy-hook working. I rewrited it using a more modern, pythonic approach using list comprehensions:

from PyInstaller import log as logging 
from PyInstaller import compat
from os import listdir

mkldir = compat.base_prefix + "/Lib/site-packages/numpy/core" 
logger = logging.getLogger(__name__)
logger.info("MKL installed as part of numpy, importing that!")
binaries = [(mkldir + "/" + mkl, '') for mkl in listdir(mkldir) if mkl.startswith('mkl_')] 

"Binaries" is a list of tuples. The second item of the tuple corresponds to the folder where you want to place the 'dlls'. In this case is empty so it copies them directly in the main folder where your '.exe' is.

Answer 2

I just ran into the same problem. As a workaround I copied the DLL's manually, as described in https://stackoverflow.com/a/34893933/4089081

I'm trying to find a better solution though.

Answer 3

I updated the snippet to get it working with my anaconda installation (the path is different):

from PyInstaller import log as logging
from PyInstaller import compat
from os import listdir
from os.path import join

mkldir = join(compat.base_prefix, "Library", "bin")
binaries = [(join(mkldir, mkl), '') for mkl in listdir(mkldir) if mkl.startswith('mkl_')]

Update: This is just working for windows. I also saw that there is a PR on github with a more accurate fix. See this Commit

Answer 4

j4n7's answer was very helpful, however, it may or may not be buggy. compat.base_prefix uses backslashes (at least for me) but they then concatenate with "/Lib/site-packages/numpy/core" (forward slashes).

>>> from PyInstaller import compat
>>> compat.base_prefix
'C:\\Python34'
>>> mkldir = compat.base_prefix + "/Lib/site-packages/numpy/core"
>>> mkldir
'C:\\Python34/Lib/site-packages/numpy/core'

As you can see, it produces both forward and backward slashes in a path.

Here are my steps that allowed me to bundle the numpy mkl files into onefile. Note that my particular app uses matplotlib and the problem I was experiencing was everytime I clicked a button (tkinter) to execute the plot, the app crashed.

Steps

First: Make a build of your app using:

pyinstaller --onefile --windowed yourpythonappnamehere.py

Second: Open the .spec file and add this to it. Obviously make sure the below files actually exist first. You may not have Python34 so just a friendly warning not to copy blindly.

mkl_dlls = [('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_avx.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_avx2.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_avx512.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_core.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_def.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_intel_thread.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_mc.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_mc3.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_rt.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_sequential.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_tbb_thread.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_avx.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_avx2.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_avx512.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_cmpt.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_def.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_mc.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_mc2.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_mc3.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\libiomp5md.dll', '')]

Third: where it says binaries=None, change to binaries=mkl_dlls.

a = Analysis(['yourpythonappnamehere.py'],
             pathex=['C:\\Users\\...\\Documents\\...'],
             binaries=mkl_dlls,
             datas=None,
             ....

Fourth: Re-run the first step. When your app is built, go into the dist folder and launch your app. I hope it works for you!

UPDATE: If you get Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll but you can clearly see that mkl_intel_thread.dll IS IN your program directory, go to numpy/core and literally copy all the files with .dll extensions that you don't have and paste them into your program's directory and re-run. If it works, great, but you might want to delete one at a time to figure out which ones you need and which ones you don't.

Answer 5

I had 2 versions of Python installed. One basic version of Python and one installed by Anaconda. When running my machine learning script it threw the same error. When I uninstalled one of the two versions (in my case I removed Anaconda) everything works fine again. I use deeplearning4j as library and apparently it has difficulties to resolve this dll, cause there were multiple Python installations installed.

Answer 6

I created a hook-numpy.py to deal with this problem:

from PyInstaller import log as logging 
from PyInstaller import compat
from os import listdir

libdir = compat.base_prefix + "/lib"
mkllib = filter(lambda x : x.startswith('libmkl_'), listdir(libdir))
if mkllib <> []: 
   logger = logging.getLogger(__name__)
   logger.info("MKL installed as part of numpy, importing that!")
   binaries = map(lambda l: (libdir + "/" + l, ''), mkllib)

In my case, conda is installing the mkl libraries to speed up numpy and scipy.