PEP8 – import not at top of file with sys.path

Question

Problem

PEP8 has a rule about putting imports at the top of a file:

Imports are always put at the top of the file, just after any module comments and docstrings, and before module globals and constants.

However, in certain cases, I might want to do something like:

import sys
sys.path.insert("..", 0)

import my_module

In this case, the pep8 command line utility flags my code:

E402 module level import not at top of file

What is the best way to achieve PEP8 compliance with sys.path modifications?

Why

I have this code because I'm following the project structure given in The Hitchhiker's Guide to Python.

That guide suggests that I have a my_module folder, separate from a tests folder, both of which are in the same directory. If I want to access my_module from tests, I think I need to add .. to the sys.path

Answer 1

Often I have multiple files with tests in a subdirectory foo/tests of my project, while the modules I'm testing are in foo/src. To run the tests from foo/tests without import errors I create a file foo/tests/pathmagic.py that looks like this;

"""Path hack to make tests work."""

import os
import sys

bp = os.path.dirname(os.path.realpath('.')).split(os.sep)
modpath = os.sep.join(bp + ['src'])
sys.path.insert(0, modpath)

In every test file, I then use

import pathmagic  # noqa

as the first import. The "noqa" comment prevents pycodestyle/pep8 from complaining about an unused import.

Answer 2

To comply with the pep8, you should include your project path to the python path in order to perform relative / absolute imports.

To do so, you can have a look at this answer: Permanently add a directory to PYTHONPATH

Answer 3

There is another workaround.

import sys
... all your other imports...

sys.path.insert("..", 0)
try:
    import my_module
except:
    raise

Answer 4

If there are just a few imports, you can just ignore PEP8 on those import lines:

import sys
sys.path.insert("..", 0)
import my_module  # noqa: E402

Answer 5

I've just struggled with a similar question, and I think I found a slightly nicer solution than the accepted answer.

Create a pathmagic module that does the actual sys.path manipulation, but make the change within a context manager:

"""Path hack to make tests work."""

import os
import sys

class context:
    def __enter__(self):
        bp = os.path.dirname(os.path.realpath('.')).split(os.sep)
        modpath = os.sep.join(bp + ['src'])
        sys.path.insert(0, modpath)

    def __exit__(self, *args):
        pass

Then, in your test files (or wherever you need this), you do:

import pathmagic

with pathmagic.context():
    import my_module
    # ...

This way you don't get any complaints from flake8/pycodestyle, you don't need special comments, and the structure seems to make sense.

For extra neatness, consider actually reverting the path in the __exit__ block, though this may cause problems with lazy imports (if you put the module code outside of the context), so maybe not worth the trouble.

---

EDIT: Just saw a much simpler trick in an answer to a different question: add assert pathmagic under your imports to avoid the noqa comment.