How to load all modules in a folder?

问题:

Could someone provide me with a good way of importing a whole directory of modules?
I have a structure like this:

/Foo
    bar.py
    spam.py
    eggs.py

I tried just converting it to a package by adding __init__.py and doing from Foo import * but it didn\'t work the way I had hoped.

回答1:

Add the __all__ Variable to __init__.py containing:

__all__ = [\"bar\", \"spam\", \"eggs\"]

See also http://docs.python.org/tutorial/modules.html

回答2:

List all python (.py) files in the current folder and put them as __all__ variable in __init__.py

from os.path import dirname, basename, isfile
import glob
modules = glob.glob(dirname(__file__)+\"/*.py\")
__all__ = [ basename(f)[:-3] for f in modules if isfile(f) and not f.endswith(\'__init__.py\')]

回答3:

Update: Today you probably want to use importlib instead.

Make the Foo directory a package by adding an __init__.py. In that __init__.py add:

import bar
import eggs
import spam

Since you want it dynamic (which may or may not be a good idea), list all py-files with list dir and import them with something like this:

import os
for module in os.listdir(os.path.dirname(__file__)):
    if module == \'__init__.py\' or module[-3:] != \'.py\':
        continue
    __import__(module[:-3], locals(), globals())
del module

Then, from your code do this:

import Foo

You can now access the modules with

Foo.bar
Foo.eggs
Foo.spam

etc. from Foo import * is not a good idea for several reasons, including name clashes and making it hard to analyze the code.

回答4:

Expanding on Mihail\'s answer, I believe the non-hackish way (as in, not handling the file paths directly) is the following:

1. create an empty __init__.py file under Foo/
2. Execute

import pkgutil
import sys


def load_all_modules_from_dir(dirname):
    for importer, package_name, _ in pkgutil.iter_modules([dirname]):
        full_package_name = \'%s.%s\' % (dirname, package_name)
        if full_package_name not in sys.modules:
            module = importer.find_module(package_name
                        ).load_module(full_package_name)
            print module


load_all_modules_from_dir(\'Foo\')

You\'ll get:

<module \'Foo.bar\' from \'/home/.../Foo/bar.pyc\'>
<module \'Foo.spam\' from \'/home/.../Foo/spam.pyc\'>

回答5:

Python, include all files under a directory:

For newbies who just can\'t get it to work who need their hands held.

1. Make a folder /home/el/foo and make a file main.py under /home/el/foo Put this code in there:

   from hellokitty import *
   spam.spamfunc()
   ham.hamfunc()
   

2. Make a directory /home/el/foo/hellokitty

3. Make a file __init__.py under /home/el/foo/hellokitty and put this code in there:

   __all__ = [\"spam\", \"ham\"]
   

4. Make two python files: spam.py and ham.py under /home/el/foo/hellokitty

5. Define a function inside spam.py:

   def spamfunc():
     print \"Spammity spam\"
   

6. Define a function inside ham.py:

   def hamfunc():
     print \"Upgrade from baloney\"
   

7. Run it:

   el@apollo:/home/el/foo$ python main.py 
   spammity spam
   Upgrade from baloney
   

回答6:

I got tired of this problem myself, so I wrote a package called automodinit to fix it. You can get it from http://pypi.python.org/pypi/automodinit/.

Usage is like this:

1. Include the automodinit package into your setup.py dependencies.
2. Replace all __init__.py files like this:

__all__ = [\"I will get rewritten\"]
# Don\'t modify the line above, or this line!
import automodinit
automodinit.automodinit(__name__, __file__, globals())
del automodinit
# Anything else you want can go after here, it won\'t get modified.

That\'s it! From now on importing a module will set __all__ to a list of .py[co] files in the module and will also import each of those files as though you had typed:

for x in __all__: import x

Therefore the effect of \"from M import *\" matches exactly \"import M\".

automodinit is happy running from inside ZIP archives and is therefore ZIP safe.

Niall

回答7:

I know I\'m updating a quite old post, and I tried using automodinit, but found out it\'s setup process is broken for python3. So, based on Luca\'s answer, I came up with a simpler answer - which might not work with .zip - to this issue, so I figured I should share it here:

within the __init__.py module from yourpackage:

#!/usr/bin/env python
import os, pkgutil
__all__ = list(module for _, module, _ in pkgutil.iter_modules([os.path.dirname(__file__)]))

and within another package below yourpackage:

from yourpackage import *

Then you\'ll have all the modules that are placed within the package loaded, and if you write a new module, it\'ll be automagically imported as well. Of course, use that kind of things with care, with great powers comes great responsibilities.

回答8:

I have also encountered this problem and this was my solution:

import os

def loadImports(path):
    files = os.listdir(path)
    imps = []

    for i in range(len(files)):
        name = files[i].split(\'.\')
        if len(name) > 1:
            if name[1] == \'py\' and name[0] != \'__init__\':
               name = name[0]
               imps.append(name)

    file = open(path+\'__init__.py\',\'w\')

    toWrite = \'__all__ = \'+str(imps)

    file.write(toWrite)
    file.close()

This function creates a file (in the provided folder) named __init__.py, which contains an __all__ variable that holds every module in the folder.

For example, I have a folder named Test which contains:

Foo.py
Bar.py

So in the script I want the modules to be imported into I will write:

loadImports(\'Test/\')
from Test import *

This will import everything from Test and the __init__.py file in Test will now contain:

__all__ = [\'Foo\',\'Bar\']

回答9:

Anurag\'s example with a couple of corrections:

import os, glob

modules = glob.glob(os.path.join(os.path.dirname(__file__), \"*.py\"))
__all__ = [os.path.basename(f)[:-3] for f in modules if not f.endswith(\"__init__.py\")]

回答10:

Anurag Uniyal answer with suggested improvements!

#!/usr/bin/python
# -*- encoding: utf-8 -*-

import os
import glob

all_list = list()
for f in glob.glob(os.path.dirname(__file__)+\"/*.py\"):
    if os.path.isfile(f) and not os.path.basename(f).startswith(\'_\'):
        all_list.append(os.path.basename(f)[:-3])

__all__ = all_list  

回答11:

See that your __init__.py defines __all__. The modules - packages doc says

The __init__.py files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path. In the simplest case, __init__.py can just be an empty file, but it can also execute initialization code for the package or set the __all__ variable, described later.

...

The only solution is for the package author to provide an explicit index of the package. The import statement uses the following convention: if a package’s __init__.py code defines a list named __all__, it is taken to be the list of module names that should be imported when from package import * is encountered. It is up to the package author to keep this list up-to-date when a new version of the package is released. Package authors may also decide not to support it, if they don’t see a use for importing * from their package. For example, the file sounds/effects/__init__.py could contain the following code:

__all__ = [\"echo\", \"surround\", \"reverse\"]

This would mean that from sound.effects import * would import the three named submodules of the sound package.

回答12:

This is the best way i\'ve found so far:

from os.path import dirname, join, isdir, abspath, basename
from glob import glob
pwd = dirname(__file__)
for x in glob(join(pwd, \'*.py\')):
    if not x.startswith(\'__\'):
        __import__(basename(x)[:-3], globals(), locals())

回答13:

import pkgutil
__path__ = pkgutil.extend_path(__path__, __name__)
for imp, module, ispackage in pkgutil.walk_packages(path=__path__, prefix=__name__+\'.\'):
  __import__(module)

回答14:

Look at the pkgutil module from the standard library. It will let you do exactly what you want as long as you have an __init__.py file in the directory. The __init__.py file can be empty.

回答15:

I\'ve created a module for that, which doesn\'t rely on __init__.py (or any other auxiliary file) and makes me type only the following two lines:

import importdir
importdir.do(\"Foo\", globals())

Feel free to re-use or contribute: http://gitlab.com/aurelien-lourot/importdir

回答16:

Just import them by importlib and add them to __all__ (add action is optional) in recurse in the __init__.py of package.

/Foo
    bar.py
    spam.py
    eggs.py
    __init__.py

# __init__.py
import os
import importlib
pyfile_extes = [\'py\', ]
__all__ = [importlib.import_module(\'.%s\' % filename, __package__) for filename in [os.path.splitext(i)[0] for i in os.listdir(os.path.dirname(__file__)) if os.path.splitext(i)[1] in pyfile_extes] if not filename.startswith(\'__\')]
del os, importlib, pyfile_extes