pytest cannot import module while python can

问题:

I am working on a package in Python. I use virtualenv. I set the path to the root of the module in a .pth path in my virtualenv, so that I can import modules of the package while developing the code and do testing (Question 1: is it a good way to do?). This works fine (here is an example, this is the behavior I want):

(VEnvTestRc) zz@zz:~/Desktop/GitFolders/rc$ python
Python 2.7.12 (default, Jul  1 2016, 15:12:24) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from rc import ns
>>> exit()
(VEnvTestRc) zz@zz:~/Desktop/GitFolders/rc$ python tests/test_ns.py 
issued command: echo hello
command output: hello

However, if I try to use PyTest, I get some import error messages:

(VEnvTestRc) zz@zz:~/Desktop/GitFolders/rc$ pytest
=========================================== test session starts ============================================
platform linux2 -- Python 2.7.12, pytest-3.0.5, py-1.4.31, pluggy-0.4.0
rootdir: /home/zz/Desktop/GitFolders/rc, inifile: 
collected 0 items / 1 errors 

================================================== ERRORS ==================================================
________________________________ ERROR collecting tests/test_ns.py ________________________________
ImportError while importing test module '/home/zz/Desktop/GitFolders/rc/tests/test_ns.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_ns.py:2: in <module>
    from rc import ns
E   ImportError: cannot import name ns
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================= 1 error in 0.09 seconds ==========================================
(VEnvTestRc) zz@zz:~/Desktop/GitFolders/rc$ which pytest
/home/zz/Desktop/VirtualEnvs/VEnvTestRc/bin/pytest

I am a bit puzzled, it looks like this indicates an import error, but Python does it fine so why is there a problem specifically with PyTest? Any suggestion to the reason / remedy (Question 2)? I googled and stack-overflowed the 'ImportError: cannot import' error for PyTest, but the hits I got were related to missing python path and remedy to this, which does not seem to be the problem here. Any suggestions?

回答1:

Found the answer:

DO NOT put a __init__.py file in a folder containing TESTS if you plan on using pytest. I had one such file, deleting it solved the problem.

This was actually buried in the comments to the second answer of PATH issue with pytest 'ImportError: No module named YadaYadaYada' so I did not see it, hope it gets more visibility here.

回答2:

I can't say I understand why this works, but I had the same problem and the tests work fine if I run python -m pytest.

I'm in a virtualenv, with pytest also available globally:

(proj)tom@neon ~/dev/proj$ type -a python
python is /home/tom/.virtualenvs/proj/bin/python
python is /usr/bin/python

(proj)tom@neon ~/dev/proj$ python -V
Python 3.5.2

(proj)tom@neon ~/dev/proj$ type -a pytest
pytest is /home/tom/.virtualenvs/proj/bin/pytest
pytest is /usr/bin/pytest

(proj)tom@neon ~/dev/proj$ pytest --version
This is pytest version 3.5.0, imported from /home/tom/.virtualenvs/proj/lib/python3.5/site-packages/pytest.py

回答3:

I just solved this by removing the __init__.py in my project root:

.
├── __init__.py <--- removed
├── models
│   ├── __init__.py
│   ├── address.py
│   ├── appointment.py
│   └── client.py
├── requirements.txt
├── setup.cfg
├── tests
│   ├── __init__.py
│   ├── models
│   │   ├── __init__.py
│   │   ├── appointment_test.py
│   │   └── client_test.py
│   └── other_test.py
└── script.py

回答4:

I had the same problem but for another reason than the ones mentioned:

I had py.test installed globally, while the packages were installed in a virtual environment.

The solution was to install pytest in the virtual environment. (In case your shell hashes executables, as Bash does, use hash -r, or use the full path to py.test)

回答5:

This problem will happen if you have a tests.py file and a tests folder with tests/__init__.py.

During the collection pytest finds the folder, but when it tries to import the test files from the folder, tests.py file will cause the import problem.

To fix simply remove the tests.py file and put all your tests inside the tests/ folder.

For your specific case the fix will be precisely:

• Remove the file /home/zz/Desktop/GitFolders/rc/tests.py
• Make sure /home/zz/Desktop/GitFolders/rc/tests/__init__.py is present

回答6:

I had a similar issue, exact same error, but different cause. I was running the test code just fine, but against an old version of the module. In the previous version of my code one class existed, while the other did not. After updating my code, I should have run the following to install it.

sudo pip install ./ --upgrade

Upon installing the updated module running pytest produced the correct results (because i was using the correct code base).

回答7:

Install the packages into Your virtual environment.
Then start a new shell and source Your virtual environment again.

回答8:

In my case, the import error occurred because the package is pointing to another package/directory with the same name and its path is one level above the folder I actually wanted. I think this also explains why some people need to remove _ init _.py while others need to add back.

I just put print(the_root_package.__path__) (after import the_root_package) in both python console and pytest scripts to compare the difference

BOTTOM LINE: When you do python, the package you import may be different from the package when you run pytest.

回答9:

The answer above not work for me. I just solved it by appending the absolute path of the module which not found to the sys.path at top of the test_xxx.py (your test module), like:

import sys
sys.path.append('path')

回答10:

Another special case:

I had the problem using tox. So my program ran fine, but unittests via tox kept complaining. After installing packages (needed for the program) you need to additionally specify the packages used in the unittests in the tox.ini

[testenv]
deps =
    package1
    package2 
...

回答11:

Had a similar issue and it worked when I added init.py file under tests directory.

回答12:

If it is related to python code that was originally developed in python 2.7 and now migrated into python 3.x than the problem is probably related to an import issue.

e.g. when importing an object from a file: base that is located in the same directory this will work in python 2.x:

from base import MyClass

in python 3.x you should replace with base full path or .base not doing so will cause the above problem. so try:

from .base import MyClass

回答13:

I was getting this using VSCode. I have a conda environment. I don't think the VScode python extension could see the updates I was making.

python c:\Users\brig\.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\testing_tools\run_adapter.py discover pytest -- -s --cache-clear test
Test Discovery failed:

I had to run pip install ./ --upgrade

回答14:

Edit your conftest.py and add following lines of code:

import os, sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(file), '..')))

And if trying to run the test case through terminal, use following ex:

python -m pytest test_some_step_file_steps.py --html=HTML_step_file_output.html --self-contained-html

回答15:

[Resolved] Before directly jumping into the solution of removing/ adding __init__.py, we might also want to look at how the imports are been done in your classes. Actually, I lost a day playing around with just __init__.py thinking that might be the problem :) However, that was quite informative.

In my case, it was the wrong way of calling classes from one python class to another python class which was throwing ImportError. Fixed the way classes/ modules to be called and it worked like charm. Hope, this helps others as well.

And yes, for similar error, we might have different solutions depending on how the code is written. Better to spend more time on self debugging. Lesson Learnt :) Happy coding!!!

回答16:

I was experiencing this issue today and solved it by calling python -m pytest from the root of my project directory.

Calling pytest from the same location still caused issues.

My Project dir is organized as:

api/
 - server/
  - tests/
      - test_routes.py
  - routes/
      - routes.py
 - app.py

The module routes was imported in my test_routes.py as: from server.routes.routes import Routes

Hope that helps!

回答17:

Yet another massive win for Python's import system. I think the reason there is no consensus is that what works probably depends on your environment and the tools you are using on top of it.

I'm using this from VS Code, in the test explorer under Windows in a conda environment, Python 3.8.

The setup I have got to work is:

mypkg/
    __init__.py
    app.py
    view.py
tests/
    test_app.py
    test_view.py

Under this setup intellisense works and so does test discovery.

Note that I originally tried the following, as recommended here.

src/
    mypkg/
        __init__.py
        app.py
        view.py
tests/
    test_app.py
    test_view.py

I could find no way of getting this to work from VS Code because the src folder just blew the mind of the import system. I can imagine there is a way of getting this to work from the command line. As a relatively new convert to Python programming it gives me a nostalgic feeling of working with COM, but being slightly less fun.

回答18:

It could be that Pytest is not reading the package as a Python module while Python is (likely due to path issues). Try changing the directory of the pytest script or adding the module explicitly to your PYTHONPATH.

Or it could be that you have two versions of Python installed on your machine. Check your Python source for pytest and for the python shell that you run. If they are different (i.e. Python 2 vs 3), use source activate to make sure that you are running the pytest installed for the same python that the module is installed in.

回答19:

For anyone who tried everything and still getting error,I have a work around.

In the folder where pytest is installed,go to pytest-env folder.

Open pyvenv.cfg file.

In the file change include-system-site-packages from false to true.

home = /usr/bin
include-system-site-packages = true
version = 3.6.6

Hope it works .Don't forget to up vote.

回答20:

If you already have .pyc files, try to delete them.

Today, I encounter this problem, here is what happend:

first I run pytest in mac (this will generate pyc files) then I launch a docker container (the os is alpine), with project dir mounted, and then when I try to run pytest in container, ImportError happens. after cleaning all pyc files, no error any more.

Hope this may be helpful.

回答21:

if you need a init.py file in your folder make a copy of the folder and delete init.py in that one to run your tests it works for local projects. If you need to run test regularly see if you can move your init.py to a separate file.

回答22:

My 2 cents on this: pytest will fail at chance if you are not using virtual environments. Sometimes it will just work, sometimes not.

Therefore, the solution is:

• remove pytest with pip uninstall
• create your venv
• activate your venv
• pip install your project path in editable mode, so it will be treated by pytest as a module (otherwise, pytest wont find your internal imports). You will need a setup.py file for that
• install your packages, including pytest
• finally, run your tests

The code, using windows PowerShell:

pip uninstall pytest
python.exe -m venv my_env
.\my_env\Scripts\activate
(my_env) pip install -e .
(my_env) pip install pytest pytest-html pandas numpy

Then finally

(my_env) pytest --html="my_testing_report.html"

An example of setup.py, for pip install -e:

import setuptools

setuptools.setup(
    name='my_package',
    version='devel',
    author='erickfis',
    author_email='[e-mail]',
    description='My package',
    long_description='My gooood package',
    packages=setuptools.find_packages(),
    classifiers=[
        'Programming Language :: Python :: 3',
        'Operating System :: OS Independent',
    ],
    include_package_data=True
)

回答23:

I had placed all my tests in a tests folder and was getting the same error. I solved this by adding an init.py in that folder like so:

. |-- Pipfile |-- Pipfile.lock |-- README.md |-- api |-- app.py |-- config.py |-- migrations |-- pull_request_template.md |-- settings.py -- tests |-- __init__.py <------ |-- conftest.py -- test_sample.py