Which standard library modules are required to run

Question

Here's a CPython program that tries to initialize the interpreter with an empty sys.path:

#include <Python.h>

int main(int argc, char** argv)
{
    wchar_t* program = NULL;
    wchar_t* sys_path = NULL;

    Py_NoSiteFlag = 1;

    program = Py_DecodeLocale(argv[0], NULL);
    Py_SetProgramName(program);

    sys_path = Py_DecodeLocale("", NULL);
    Py_SetPath(sys_path);

    Py_Initialize();

    PyMem_RawFree(program);    
    PyMem_RawFree(sys_path);
    Py_Finalize();
}

Executing the program above raises the following error:

Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named 'encodings'

Current thread 0x00007ffff7fc6700 (most recent call first):
Signal: SIGABRT (Aborted)

So which of the packages and modules in the Python 3.5 standard library, besides the encodings package, are absolutely required to run the Python 3.5 interpreter? This information seems to me absent from the documentation.

Answer 1

These are packages/modules that are used during interpreter start-up (as, @Charles Duffy noted in a comment, by looking in sys.modules).

The result depends on whether you have site enabled or not (your Py_NoSiteFlag = 1; implies this isn't the case but anyway, I'll give both options :-)).

site drags a couple of additional modules with it when you use it like _sitebuiltins and stat, in total you could run Python using only the following:

abc.py               encodings       os.py         _sitebuiltins.py  sysconfig.py
codecs.py            genericpath.py  posixpath.py  site.py           _collections_abc.py  
io.py                stat.py         _weakrefset.py

with site disabled, you're stripped down to the following 6:

abc.py  codecs.py  encodings  io.py  os.py  _weakrefset.py

when invoked through C with Py_Initialize() (or through Windows based on your comment) I'm guessing os.py might not be actually needed.

Answer 2

If you run the interpreter as Charles Duffy suggests in his comment, you'll load packages like readline. It has been a decade since I did this but IIRC you don't need that module if you use python as an extension to your C program, as there is no commandline interaction. The same might hold for other modules.

The quickest way to determine what is really needed with only a slight chance of getting in too much is by putting all of lib/python3.5 where your program can find it, and in the program print out sys.modules, that will give you a list of what your program actually loaded, not what the interpreter might need to start up. After that remove everything not on that list.

Answer 3

Here's another approach - asking the Python interpreter what modules are loaded:

$ python3.5 -v -S -c '' |& grep SourceFileLoader | sort 
import 'abc' # <_frozen_importlib_external.SourceFileLoader object at 0x7f4b1d12e860>
import '_bootlocale' # <_frozen_importlib_external.SourceFileLoader object at 0x7f4b1d1367b8>
import 'codecs' # <_frozen_importlib_external.SourceFileLoader object at 0x7f4b1d187fd0>
import 'encodings.aliases' # <_frozen_importlib_external.SourceFileLoader object at 0x7f4b1d11eac8>
import 'encodings' # <_frozen_importlib_external.SourceFileLoader object at 0x7f4b1d187be0>
import 'encodings.latin_1' # <_frozen_importlib_external.SourceFileLoader object at 0x7f4b1d12e3c8>
import 'encodings.utf_8' # <_frozen_importlib_external.SourceFileLoader object at 0x7f4b1d12c898>
import 'io' # <_frozen_importlib_external.SourceFileLoader object at 0x7f4b1d12e5f8>
import '_weakrefset' # <_frozen_importlib_external.SourceFileLoader object at 0x7f4b1d135080>

_bootlocale is not required but recommended. It's used for initializing the best encoding for sys.stdin/sys.stdout/sys.stderr. See https://hg.python.org/cpython/rev/fbbf8b160e8d

sys.modules can lie as it's mutable.