Lazy loading module imports in an __init__.py file

Question

I was wondering if anyone had any suggestions for lazy loading imports in an init file? I currently have the following folder structure:

/mypackage
    __init__.py
    /core
        __init__.py
        mymodule.py
        mymodule2.py

The init.py file in the core folder with the following imports:

from mymodule import MyModule
from mymodule2 import MyModule2

This way I can just do:

from mypackage.core import MyModule, MyModule2

However, in the package init.py file, I have another import:

from core.exc import MyModuleException

This has the effect that whenever I import my package in python, MyModule and MyModule2 get imported by default because the core init.py file has already been run.

What I want to do, is only import these modules when the following code is run and not before:

from mypackage.core import MyModule, MyModule2

Any ideas?

Many thanks.

Answer 1

Not sure if it applies here but in general lazy loading of modules can be done using the Importing package.

Works like this:

from peak.util.imports import lazyModule
my_module = lazyModule('my_module')

Now my module is only really imported when you use it the first time.

Answer 2

You can't. Remember that when python imports it executes the code in the module. The module itself doesn't know how it is imported hence it cannot know whether it has to import MyModule(2) or not.

You have to choose: allow from mypackage.core import A, B and from core.exc import E does the non-needed imports (x)or do not import A and B in core/__init__.py, hence not allowing from mypackage.core import A, B.

Note: Personally I would not import MyModule(2) in core/__init__.py, but I'd add an all.py module that does this, so the user can do from mypackage.core.all import A, B and still have from mypackage.core.exc import TheException not loading the unnecessary classes.

(Actually: the all module could even modify mypackage.core and add the classes to it, so that following imports of the kind from mypackage.core import MyModule, MyModule2 work, but I think this would be quite obscure and should be avoided).

Answer 3

You may use follow code in __init__ in module:

import apipkg
apipkg.initpkg(__name__, {
    'org': {
        'Class1': "secure._mypkg:Class1",
        'Class2': "secure._mypkg2:Class2",
    }
})

Answer 4

Unless I'm mistaking your intentions, this is actually possible but requires some magic.

Basically, subclass types.ModuleType and override __getattr__ to import on demand.

Check out the Werkzeug init.py for an example.

Answer 5

If your modules structure is like:

/mypackage
  __init__.py
  /core
    __init__.py
    MyModule.py
    MyModule2.py

or:

/mypackage
  __init__.py
  /core
    __init__.py
    /MyModule
      __init__.py
    /MyModule2
      __init__.py

then feel free to use

from mypackage.core import MyModule, MyModule2

without importing them in __init__.py under mypackage/core