PATH issues with homebrew-installed Python 2 and P

Question

I'm relatively new to programming and have searched 'til my fingertips were blue, but can't seem to find a solution to the problem I'm having.

I have homebrew-installed versions of Python 2 and Python 3 on OSX and I can't seem to get the proper PATH/PYTHONPATH in my .bash_profile in order to be able to import modules properly in both versions in IDLE. I can, however, import modules when running Python 2 or Python 3 directly in a shell window. I am launching IDLE via terminal so it should properly initialize the paths.

Here is my .bash_profile:

export PATH=/bin:/usr/local/bin:$PATH
export PYTHONPATH=/Users/maverett/Documents/PyModules:/Users/maverett/Dropbox/matrix/:$PYTHONPATH

Here's what happens in all four cases, using numpy as an example module.

Importing when running python2 in terminal works:

$ python2
Python 2.7.5 (default, Jun 28 2013, 19:06:25) 
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> 

Launching IDLE for Python 2 from terminal, I can also import numpy. However, when I compare sys.path in IDLE vs sys.path in terminal, they are different. I compared the lists to generate the differences and found:

>>> InIdleNotInTerm 
['/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/bin']
>>> InTermNotInIdle
[]

So there is one extra directory in the IDLE path when running Python 2.

The story is quite different for Python 3.

$ python3
Python 3.3.2 (default, Jul  1 2013, 10:53:26) 
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> 

However, if I launch IDLE for Python 3 (by typing idle3 in terminal) and then try to import numpy, I get

>>> import numpy as np
Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    import numpy as np
ImportError: No module named 'numpy'
>>> 

Again, I compared sys.path in terminal and in IDLE and this time there are major differences:

>>> pp(InIdleNotInTerm)
['/Library/Frameworks/Python.framework/Versions/3.3/bin',
 '/Library/Frameworks/Python.framework/Versions/3.3/lib/python33.zip',
 '/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3',
 '/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/plat-darwin',
 '/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/lib-dynload',
 '/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages']
>>> pp(InTermNotInIdle)
['/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/distribute-0.6.45-py3.3.egg',
 '/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pip-1.3.1-py3.3.egg',
 '/usr/local/lib/python3.3/site-packages/distribute-0.6.45-py3.3.egg',
 '/usr/local/lib/python3.3/site-packages/pip-1.3.1-py3.3.egg',
 '/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python33.zip',
 '/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3',
 '/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/plat-darwin',
 '/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/lib-dynload',
 '/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages',
 '/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/setuptools-0.6c11-py3.3.egg-info',
 '/usr/local/lib/python3.3/site-packages',
 '/usr/local/lib/python3.3/site-packages/setuptools-0.6c11-py3.3.egg-info']

Any idea what's going on? A few other things I've tried:

Changing .bash_profile to

export PATH=/bin:/usr/local/bin:$PATH
export PYTHONPATH=/Users/maverett/Documents/PyModules:/Users/maverett/Dropbox/matrix/:/usr/local/lib/python3.3/site-packages/:$PYTHONPATH

results in being able to import modules in Python 3 in terminal and in IDLE, but breaks imports for Python 2 (because it tries to import the Python 3 versions!)

Launching IDLE for Python 3 via terminal from python3.3/site-packages/ results in being able to properly import everything, but I don't want to do this every time I launch Python 3.

Any thoughts or ideas you have would be greatly appreciated! Thanks :)

Answer 1

Ok, here's what I'd like you to do:

Stop using easy_install, if you're still using it for package management. Use pip instead.

$> easy_install pip

Next, get virtual environments.

$> pip install virtualenv
$> mkdir ~/venvs
$> virtualenv ~/venvs/numpy_project --python=python2.7 --no-site-packages

Make sure to pass in a well-named directory to virtualenv. The standard usage is to organize your virtual environments by project, so I named this virtual environment "numpy_project". You should probably come up with a better name. I also told it to use python 2.7, but you can choose to use 3.3 if you want.

Basically, this is going to create a well-insulated bubble for a dedicated copy of python to live for one specific purpose. I also told it to use no-site-packages, as to ensure a clean slate. It makes it easier to get set up for work on this project, which can be done by running:

$> source ~/venvs/numpy_project/bin/activate

This will switch your python environment from the global "main" python to this protected copy. You can then run:

$> pip install numpy

And it will install it just for that copy of python.

Be sure to run

$> pip freeze > requirements.txt

In the root of your project, where your README.md and stuff would go, so that others can simply run:

$> pip install -r PROJECT_ROOT/requirements.txt

And it will grab all the things you've put in your virtual environment (i.e., numpy). If they're using virtual environments as well, you can be certain they have an exact match to your working environment. There should be no overlap or confusion from other packages and versions of python.

Try this and see if you get better results. Remember to never run pip with sudo!

If this works, just uninstall your "global" install of numpy, and just use it in your virtual environments.