How to add my module to travis-ci pythonpath

Question

I'm setting up Travis-CI for my project, and oddly, I can't import my project:

$ python tests/tests.py
Traceback (most recent call last):
  File "tests/tests.py", line 11, in <module>
    from my_module.lib.importer import build_module_list
ImportError: No module named my_module.lib.importer

In production, I just create a symlink like so:

sudo ln -s /usr/local/my_module /usr/lib/python2.7/dist-packages/my_module

But I don't know -- or want to know, really -- Travis-CI's folder structure.

This seems like a solved problem, but I'm new to Travis-CI. What's the best way to make this work, so my code is added as an importable module?

Answer 1

In complement of @Brian-Cain answer, you can also use setuptools instead of distutils. As of writing, distutils is being phased out, and setuptools is being used as a replacement, even though setuptools is not yet in standard library.

from setuptools import setup, find_packages

setup(name='Foo',
      version='0.0.1',
      description='Python Distribution Utilities',
      author='',
      author_email='',
      url='',
      packages=find_packages(exclude=['contrib', 'docs', 'tests*']),
     )

For a quick tutorial on making a setup.py with setuptools: https://packaging.python.org/tutorials/distributing-packages/

For a quick real example: https://github.com/pypa/sampleproject/blob/master/setup.py

Answer 2

To be quick, you can fix the problem more elegantly by adding the following to the before_script stage:

export PYTHONPATH=$PYTHONPATH:$(pwd)

The better way(but with a little more effort) is what Brian Cain has suggested, namely, write a setup.py file and add pip install . to the install stage.

Or if you're using a makefile you can have command that does it as the following one.

test:
    $(shell export PYTHONPATH=$PYTHONPATH:$(pwd))
    python setup.py test

Answer 3

It's more likely necessary to add /home/travis/.local/lib/python2.7/site-packages/ to PYTHONPATH in before_script using export PYTHONPATH=$PYTHONPATH:/home/travis/.local/lib/python2.7/site-packages/.

Answer 4

This is certainly not optimal, but it worked. In my .travis.yml file, I added the following line to the install attribute:

 - ln -s `pwd` $(dirname `which python`)/../lib/python2.7/site-packages/my_module

This basically finds the directory where Python is installed and then adds my_module as a symlink in there. Happy to hear a better answer, cause this one feels super fragile.

Update: See the answer by @Brian Cain for a much better solution.

Answer 5

What's the best way to make this work, so my code is added as an importable module?

The answer is unequivocally to use distutils (and definitely not ln).

In production, I just create a symlink ...

B-b-but why? The complexity to do it the Right Way™ is so low! It even fits in a few lines:

From The Fine Manual -- just create a setup.py like this:

from distutils.core import setup

setup(name='Distutils',
      version='1.0',
      description='Python Distribution Utilities',
      author='Greg Ward',
      author_email='[e-mail]',
      url='https://www.python.org/sigs/distutils-sig/',
      packages=['distutils', 'distutils.command'],
     )

Now you can do fantastic things like python setup.py install, python setup.py bdist_rpm or pip install . -- and not just in the Travis environment but for your project in general.