cx_Freeze - Preventing including unneeded packages

Question

I have coded a tiny python program using PyQt4. Now, I want to use cx_Freeze to create a standalone application. Everything works fine - cx_Freeze includes automatically all necessary modules; the resulting exe works.

The only problem is that cx_Freeze packs plenty of unneeded modules into the standalone. Even though I only use QtCore and QtGui, also modules like sqlite3, QtNetwork, or QtScript are included. Surprisingly, I find also PyQt5 dlls in the resulting folder. It seems to me as if cx_Freeze uses all PyQt packages that I have installed. The result is a 200Mb program - albeit I only wrote a tiny script.

How can I prevent this behaviour?

I use the following setup.py:

import sys
from cx_Freeze import setup, Executable

setup(
    name="MyProgram",
    version="0.1",
    description="MyDescription",
    executables=[Executable("MyProgram.py", base = "Win32GUI")],
)

I tried explicitely excluding some packages (although it is quite messy to exclude all unused Qt modules) adding this code:

build_exe_options = {"excludes": ["tkinter", "PyQt4.sqlite3",
                              "PyQt4.QtOpenGL4", "PyQt4.QtSql"]}

but the upper modules were still used. I also tried

build_exe_options = {"excludes": ["tkinter", "PyQt4.sqlite3",
                              "QtOpenGL4", "QtSql"]}

with the same result.

In addition to the nedless Qt packages I find also unneded folders with names like "imageformats", "tcl", and "tk". How can I include only needed files in order to keep the standalone folder and installer as small as possible?

I googled this problem for hours but only found this thread which did not help me.

I am running python 3.4.2 amd64 on windows 8.

I am happy about every solution that gives me the desired result "standalone" with a reasonable size. I tried also pyqtdeploy but ran into the error: Unknown module(s) in QT (but this is a different question).

Edit:

I am using two modules. One is the GUI class created by uic, "MyProgramGUIPreset". In this file there are the following import commands:

from PyQt4 import QtCore, QtGui
from matplotlibwidget import MatplotlibWidget

In the main module I do the following imports:

import MyProgramGUIPreset
import numpy as np
from PyQt4.QtGui import QApplication, QMainWindow, QMessageBox
import sys
from math import *

Maybe this helps to figuring out where the issue is.

Answer 1

The reason for the not working "excludes" command was that I forgot to include the build options into the setup. After adding the respective line into the code excluding works:

from cx_Freeze import setup, Executable
import sys

# exclude unneeded packages. More could be added. Has to be changed for
# other programs.
build_exe_options = {"excludes": ["tkinter", "PyQt4.QtSql", "sqlite3", 
                                  "scipy.lib.lapack.flapack",
                                  "PyQt4.QtNetwork",
                                  "PyQt4.QtScript",
                                  "numpy.core._dotblas", 
                                  "PyQt5"],
                     "optimize": 2}

# Information about the program and build command. Has to be adjusted for
# other programs
setup(
    name="MyProgram",                           # Name of the program
    version="0.1",                              # Version number
    description="MyDescription",                # Description
    options = {"build_exe": build_exe_options}, # <-- the missing line
    executables=[Executable("MyProgram.py",     # Executable python file
                            base = ("Win32GUI" if sys.platform == "win32" 
                            else None))],
)

This decreased the program size from 230MB to 120MB. Nevertheless, I did not find a nice way of excluding all unneeded packages. By trial and error (deleting the biggest files in the build folder test-wise) I figured out which classes I can exclude.

I tried whether the matplotlib backends cause the problem and finally figured out that this is not the case. Nontheless, if anybody needs code to exclude all modules of a certain name scheme in a particular folder except some special ones, he may adjust the following to his needs:

mplBackendsPath = os.path.join(os.path.split(sys.executable)[0],
                        "Lib/site-packages/matplotlib/backends/backend_*")

fileList = glob.glob(mplBackendsPath)

moduleList = []

for mod in fileList:
    modules = os.path.splitext(os.path.basename(mod))[0]
    if not module == "backend_qt4agg":
        moduleList.append("matplotlib.backends." + modules)

build_exe_options = {"excludes": ["tkinter"] + moduleList, "optimize": 2}

I would be happy about more elegant solutions. Further ideas are still welcome. Nevertheless, I regard the problem as solved for me.

Answer 2

I was having a similar problem on a very simple PyQt4 Gui for a small database where the program was 58Mb for a small amount of code, the problem being that the entire PyQt4 folder was being included in the program.

The article here refers to using zip_include_packages in your options to exclude files or to compress them to reduce the file size.

I excluded the entire PyQt4 folder and then included the bits I needed as shown below and it reduced the whole package to 16Mb automatically

options = {
'build_exe': {
    'packages':packages,
    'zip_include_packages':'PyQt4',
    'includes':['PyQt4.QtCore','PyQt4.QtGui','sqlite3','sys','os'],
},

Not sure it is the right way to do it but seems to have no negative impact on my program as of yet

Answer 3

This is how I optimized my executable to the minimum file size

from cx_Freeze import setup, Executable
import subprocess
import sys


NAME = 'EXE NAME'
VERSION = '1.0'
PACKAGES = ['pygame', ('import_name', 'package_name')]
# if names are same just have a string not a tuple
installed_packages = subprocess.check_output([sys.executable, '-m', 'pip', 'freeze']).decode('utf-8')
installed_packages = installed_packages.split('\r\n')
EXCLUDES = {pkg.split('==')[0] for pkg in installed_packages if pkg != ''}
EXCLUDES.add('tkinter')
for pkg in PACKAGES:
    if type(pkg) == str: EXCLUDES.remove(pkg)
    else: EXCLUDES.remove(pkg[1])


executables = [Executable('main.py', base='Win32GUI', icon='Resources/Jungle Climb Icon.ico', targetName=NAME)]

setup(
    name=NAME,
    version=VERSION,
    description=f'{NAME} Copyright 2019 AUTHOR',
    options={'build_exe': {'packages': [pkg for pkg in PACKAGES if type(pkg) == str else pkg[0]],
                           'include_files': ['FOLDER'],
                           'excludes': EXCLUDES,
                           'optimize': 2}},
    executables=executables)