Proper way to dynamically import a module with rel

Question

I need to dynamically import modules into my project from another package.

The structure is like:

project_folder/
    project/
        __init__.py
        __main__.py
    plugins/
        __init__.py
        plugin1/
            __init__.py
            ...
        plugin2/
            __init__.py
            ...

I made this function to load a module:

import os

from importlib.util import spec_from_file_location, module_from_spec


def load_module(path, name=""):
    """ loads a module by path """
    try:
        name = name if name != "" else path.split(os.sep)[-1]  # take the module name by default
        spec = spec_from_file_location(name, os.path.join(path, "__init__.py"))
        plugin_module = module_from_spec(spec)
        spec.loader.exec_module(plugin_module)
        return plugin_module
    except Exception as e:
        print("failed to load module", path, "-->", e)

It works, unless the module uses relative imports:

failed to load module /path/to/plugins/plugin1 --> Parent module 'plugin1' not loaded, cannot perform relative import

What am I doing wrong?

Answer 1

I had a similar problem not long ago. I added the path of the project folder to the sys.path using the module's absolute path like this:

import sys
import os
sys.path.append(os.path.dirname(os.path.realpath(__file__))+'/..')

This adds the project_folder to the sys.path thus allowing the import statement to find the plugin modules.

Answer 2

How we can see in the official doc of importlib:

importlib.import_module(name, package=None) Import a module. The name argument specifies what module to import in absolute or relative terms (e.g. either pkg.mod or ..mod). If the name is specified in relative terms, then the package argument must be specified to the package which is to act as the anchor for resolving the package name (e.g. import_module('..mod', 'pkg.subpkg') will import pkg.mod). The specified module will be inserted into sys.modules and returned.

Why don't you try it?

Answer 3

I managed to solve my own issue after a LOT of googling. Turns out I needed to import using relative paths:

>>> from importlib import import_module
>>> config = import_module("plugins.config")
>>> config
<module 'plugins.config' from '/path/to/plugins/config/__init__.py'>
>>>