Is the itertools C module included somehow in the main Python binary in 3.x?
Assuming that the C module is built and included, which it appears to be:
>>> import inspect
>>> import itertools
>>>
>>> inspect.getsourcefile(itertools)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/inspect.py", line 571, in getsourcefile
filename = getfile(object)
File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/inspect.py", line 518, in getfile
raise TypeError('{!r} is a built-in module'.format(object))
TypeError: <module 'itertools' (built-in)> is a built-in module
I can't find an itertools.so for Python 3.x on my system, but there's one for 2.7.
I noted that some other C modules exist as shared objects (locate '.so' | grep -E '^/usr/local/' | grep '.so' e.g. mmap.so) on disk, so what's the deal with itertools? How can I use it if there's not a shared library?
There are hints in the makefile that's near the Python wrapper of inspect.py:
/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/config-3.4m/Makefile
We can see the build rules for the itertools.c source:
1668 Modules/itertoolsmodule.o: $(srcdir)/Modules/itertoolsmodule.c; $(CC) $(PY_CORE_CFLAGS) -c $(srcdir)/Modules/itertoolsmodule.c -o Modules/itertoolsmodule.o
And then trace it a little to see that it's being bundled in:
24 MODOBJS= .. Modules/itertoolsmodule.o ... Modules/xxsubtype.o
462 # objects that get linked into the Python library
463 LIBRARY_OBJS_OMIT_FROZEN= \
...
470 $(MODOBJS)
471
472 LIBRARY_OBJS= \
473 $(LIBRARY_OBJS_OMIT_FROZEN) \
474 Python/frozen.o
...
553 # Build the interpreter
554 $(BUILDPYTHON): Modules/python.o $(LIBRARY) $(LDLIBRARY) $(PY3LIBRARY)
555 $(LINKCC) $(PY_LDFLAGS) $(LINKFORSHARED) -o $@ Modules/python.o $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST)
556
557 platform: $(BUILDPYTHON) pybuilddir.txt
558 $(RUNSHARED) $(PYTHON_FOR_BUILD) -c 'import sys ; from sysconfig import get_platform ; print(get_platform()+"-"+sys.version[0: 3])' >platform
589 # Build static library
...
598 $(AR) $(ARFLAGS) $@ $(MODOBJS)
599 $(RANLIB) $@
944 $(LIBRARY_OBJS) $(MODOBJS) Modules/python.o: $(PYTHON_HEADERS)
Or if made via distutils, the path will be something like: /usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/_sysconfigdata.py
And assuming that this gets built into a dynamic library:
Ξ ~ → strings /usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/libpython3.4.dylib | grep itertools
itertools
itertools._tee_dataobject
itertools._tee
itertools._grouper
itertools.groupby
itertools.repeat
itertools.product
...
Which means that at build time, the itertools.c module gets included in the libpython dynamic library, meaning that it's now part of the standard library.
In Python 3, the itertools extension is compiled into the main Python binary:
>>> import sys
>>> 'itertools' in sys.builtin_module_names
True
See the sys.builtin_module_names documentation:
A tuple of strings giving the names of all modules that are compiled into this Python interpreter.
The module was added to the Python binary because it is used very widely in the Python standard library.
The list of modules to include is taken from the Modules/Setup.dist file in the Python distribution; itertools was added together with _collections, as it is a transient dependency of that module. See issue #9545.
