How do I access packages installed by `pip --user`

Question

I realized I had an outdated numpy version:

$ python
Python 2.7.10 (default, Oct 23 2015, 18:05:06)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.version.full_version
'1.8.0rc1'

I sought to update it, but I couldn't install on the entire machine for some reason:

$ sudo pip install -U numpy
Password:
Downloading/unpacking numpy from https://pypi.python.org/packages/dd/9f/cd0ec9c50e4ed8650901ad4afde164e5252b6182a9e0c7bff5f8b4441960/numpy-1.11.1.zip#md5=5caa3428b24aaa07e72c79d115140e46
  Downloading numpy-1.11.1.zip (4.7MB): 4.7MB downloaded
  ...
  Found existing installation: numpy 1.8.0rc1
    Uninstalling numpy:
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-1.4.1-py2.7.egg/pip/basecommand.py", line 134, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-1.4.1-py2.7.egg/pip/commands/install.py", line 241, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/Library/Python/2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 1294, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 525, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 1639, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip-1.4.1-py2.7.egg/pip/util.py", line 294, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
    copystat(src, dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
    os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/tmp/pip-fajcj_-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy-1.8.0rc1-py2.7.egg-info'

Storing complete log in /Users/csaftoiu/Library/Logs/pip.log

Ok, I'll just install to --user then:

$ pip install -U --user numpy
...
Successfully installed numpy

But the version is not updated!

$ python
Python 2.7.10 (default, Oct 23 2015, 18:05:06)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.version.full_version
'1.8.0rc1'

Where did the installed version go?

Answer 1

I installed packages with sudo pip install --user option and I had to do sudo python to get it working.

vishnu$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

With sudo:

vishnu$ sudo python
Python 2.7.10 (default, Jul 15 2017, 17:16:57) 
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import rasa_core
>>> 

Without sudo:

vishnu$ python
Python 2.7.10 (default, Jul 15 2017, 17:16:57) 
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import rasa_core
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named rasa_core

PS: I am running these on Mac OS High Sierra and my issue was with rasa_core and dependent packages.

Answer 2

As per the Python docs, this is installing using the "user scheme":

Files will be installed into subdirectories of site.USER_BASE (written as userbase hereafter).

You can see your USER_BASE value like this:

$ python -c "import site; print(site.USER_BASE)"
/Users/csaftoiu/Library/Python/2.7

I found that on my machine, this was on sys.path, but it came after the global install directories.

I solved it by adding this to my ~/.bash_profile:

# add user base to python path
export PYTHONPATH=$(python -c "import site, os; print(os.path.join(site.USER_BASE, 'lib', 'python', 'site-packages'))"):$PYTHONPATH

Now the latest version is indeed loaded:

$ python
Python 2.7.10 (default, Oct 23 2015, 18:05:06)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy
<module 'numpy' from '/Users/csaftoiu/Library/Python/2.7/lib/python/site-packages/numpy/__init__.pyc'>
>>> numpy.version.full_version
'1.11.1'
>>>