Build Cython-compiled modules and python code into

Question

I am trying to package my project code into a an executable binary using Cython and PyInstaller libraries. My code directory looks like this:

The main.py is the main code which imports the logic from program_a.py and program_b.py.

I am successfully able to convert my program_a and program_b files into .so files which can be imported by any python code. I did this by executing the following script.

from distutils.core import setup
from Cython.Build import cythonize

sourcefiles = ['program_a.py', 'program_b.py']

setup(
    name = "Hello World",
    ext_modules = cythonize(sourcefiles), 
)

By executing >python setup.py build_ext --inplace I get .so files as shown below

When I run python main.py it runs perfectly with .so files. Which shows that I can import them as a module.

Now, I want to package binaries (.so) files and main.py into single binary file. For that I used the following command provided by pyInstaller

pyinstaller "main.py" --onefile

It actually gives a binary in dist/ folder but I cannot able to import some modules and getting the following error:

Traceback (most recent call last):
  File "main.py", line 1, in <module>
    import program_a as lisence_checker
  File "program_a.py", line 1, in init program_a
ModuleNotFoundError: No module named 'licensing'
[18032] Failed to execute script main

How can I link libraries with the pyinstaller or embed library information into my binaries?

What I found yet:

1. Building Cython-compiled python code with PyInstaller

2. https://riptutorial.com/cython/example/21982/bundling-a-cython-program-using-pyinstaller

But all of these above links are not using any external package inside there python code examples. I am able to compile the code without external modules

Answer 1

After getting familiar with PyInstaller package I am able to figure out the issue. I followed the following steps to make it work for me at the end.

Now, posting my answer to help others :)

## Build *.so files from python modules 
    1. Execute "setup.py" file
       > python setup.py build
    2. It will generate "*.so" modules inside "build/lib.linux-x86_64-3.6" dir.

## Created binary from cython modules
    1. Copy the binaries (i.e. *.so) files into binary folder
    2. Get inside the binary folder 'cd binary'
    3. Run Pyinstaller command inside binary directory: `python -O -m PyInstaller --clean --onefile idps.spec`
    4. Your binary will be inside dist folder 'binary/dist/'
    5. Execute the binary in linux using './dist/sample_app'
    6. Your app is ready :)

Here is spec file to make it work for me:

# -*- mode: python -*-

block_cipher = None


a = Analysis(['main.py'],
             pathex=['cython_pyinstaller_sample/binary'],
             binaries=[('program_a.cpython-36m-x86_64-linux-gnu.so', '.'),('program_b.cpython-36m-x86_64-linux-gnu.so', '.')],
             datas=[('config_file.txt', '.')],
             hiddenimports=['licensing', 'licensing.methods', 'pandas'],
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher,
             noarchive=False) pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher) exe = EXE(pyz,
          a.scripts,
          a.binaries,
          a.zipfiles,
          a.datas,
          [],
          name='sample_app',
          debug=True,
          bootloader_ignore_signals=False,
          strip=False,
          upx=True,
          runtime_tmpdir=None,
          console=True )