Why has the cyclical import issue disappeared?

Question

There is an obvious cyclical import error when importing this package:

File __init__.py:

from . import modules

File forward.py:

from .modules import ext_modules

def forward(dest):
    if dest in ext_modules:
        print("forwarding to {}".format(ext_modules[dest]))

File modules.py:

from . import forward

ext_modules = {}

def main():
    ext_modules['test'] = 'TEST'
    forward.forward('test')

This import problem can be solved e.g. by exchanging lines 1 and 3 in the modules.py file. So far I think I understand what is going on.

What I really do not understand is this. When I add another import to the top of the __init__.py file:

from . import forward
from . import modules

the problem is gone. The package can be imported and the main function works. However the cycle dependency between modules and forward is still there. These files are left unmodified. Could you please explain me what is going on there? (Python version 3.5)

Answer 1

The first thing that happens when you import a module, is that an empty module object is added to the sys.modules mapping. Subsequent import statements for that same module will re-use that object rather than load the file into memory.

Python then proceeds to execute the module contents and add the global names this produces to that module object.

In your case, it matters what order your modules are imported. forward directly depends on the module contents of modules, while modules only depends on the module forward to exist, not the contents of that module (the forward.forward reference dependency is postponed until main() is called).

So if forward is imported first, the empty forward module object is created, the first line from .modules import ext_modules is executed, triggering the the modules.py file to be loaded, which then can safely use from . import forward because that empty module object now exists. The rest of the modules.py file can then continue to run without further incident.

But if you import modules first, then there is no forward module object yet, so then the forward.py file is run second, and the from .modules import ext_modules line fails because the modules module object is still empty and has no ext_modules attribute.