Change the priority of python sys.path

Question

How can I change the priority of the path in sys.path in python 2.7? I know that I can use PYTHONPATH environment variable, but it is what I will get:

$ PYTHONPATH=/tmp python
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> for i in sys.path:
...   print i
... 

/usr/local/lib/python2.7/dist-packages/pycuda-2014.1-py2.7-linux-x86_64.egg
/usr/local/lib/python2.7/dist-packages/pytest-2.6.2-py2.7.egg
/usr/local/lib/python2.7/dist-packages/pytools-2014.3-py2.7.egg
/usr/local/lib/python2.7/dist-packages/py-1.4.24-py2.7.egg
/usr/lib/python2.7/dist-packages
/tmp
/usr/lib/python2.7
/usr/lib/python2.7/plat-x86_64-linux-gnu
/usr/lib/python2.7/lib-tk
/usr/lib/python2.7/lib-old
/usr/lib/python2.7/lib-dynload
/usr/local/lib/python2.7/dist-packages
/usr/lib/python2.7/dist-packages/PILcompat
/usr/lib/python2.7/dist-packages/gtk-2.0
/usr/lib/python2.7/dist-packages/ubuntu-sso-client
>>> 

/tmp is added between /usr/lib/python2.7/dist-packages and /usr/lib/python2.7.

My goal is to make python to load packages from /usr/local/lib/python2.7/dist-packages first.

Here is what I want:

$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.version
<module 'numpy.version' from '/usr/local/lib/python2.7/dist-packages/numpy/version.pyc'>
>>> 

If I install python-numpy by apt-get install python-numpy. Python will try to load from /usr/lib/python2.7 and not the one I compiled.

Answer 1

As you may know, sys.path is initialized from:

• the current directory
• your PYTHONPATH
• an installation-dependent default

However unfortunately that is only part of the story: setuptools creates easy-install.pth files, which also modify sys.path and worst of all they prepend packages and therefore totally mess up the order of directories.

In particular (at least on my system), there is /usr/local/lib/python2.7/dist-packages/easy-install.pth with the following contents:

import sys; sys.__plen = len(sys.path)
/usr/lib/python2.7/dist-packages
import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,'__egginsert',0); sys.path[p:p]=new; sys.__egginsert = p+len(new)

This causes /usr/lib/python2.7/dist-packages to be prepended even before your PYTHONPATH!

What you could do is simply change the 2nd line in this file to

/usr/local/lib/python2.7/dist-packages

and you will get your desired priority.

However beware this file might be overwritten or changed again by a future setuptools invocation!

Answer 2

We encountered an almost identical situation and wanted to expand upon @kynan's response which is spot-on. In the case where you have such an easy-install.pth that you want to overcome, but which you cannot modify it (say you are a user with no root/admin access), you can do the following:

• Set up an alternate python installation scheme

  • e.g. we use a PYTHON HOME install (setting PYTHONUSERBASE)

• Create a user/home site-packages

  • You can do this by installing a package into the user env: pip install <package> --user

• Create a pth to set sys.__egginsert to workaround the system/distribution easy-install.pth

  • Create a $PYTHONUSERBASE/lib/python2.7/site-packages/fix_easy_install.pth
  • Containing: import sys; sys.__egginsert = len(sys.path);

This will set sys.__egginsert to point to the end of your sys.path including your usersite paths. When the nefarious system/dist easy-install.pth will then insert its items at the end of the system path.