Permission denied when trying to install inside a

Question

I have this play in my ansible playbook

- name: Setup virtualenv
  pip: 
    requirements="/vagrant/webapp/requirements/{{ requirements_filename }}" 
    virtualenv="$HOME/.envs/{{ project_name }}"

However, I am receiving a Permission denied error. It seems like ansible is trying to install the package into the system path and not into the virtualenv.

Here's the full traceback:

TASK: [deploy | Setup virtualenv] ********************************************* 
failed: [default] => {"cmd": "/usr/bin/pip install -r /vagrant/webapp/requirements/dev.txt", "failed": true}
msg: stdout: New python executable in /home/vagrant/.envs/venv/bin/python
Installing setuptools, pip...done.
Downloading/unpacking django>=1.8 (from -r /vagrant/webapp/requirements/requirements.txt (line 3))
Requirement already satisfied (use --upgrade to upgrade): wsgiref>=0.1.2 in /usr/lib/python2.7 (from -r /vagrant/webapp/requirements/requirements.txt (line 4))
Downloading/unpacking django-allauth>=0.19.1 (from -r /vagrant/webapp/requirements/requirements.txt (line 5))
  Running setup.py (path:/tmp/pip_build_vagrant/django-allauth/setup.py) egg_info for package django-allauth

Downloading/unpacking psycopg2>=2.6 (from -r /vagrant/webapp/requirements/requirements.txt (line 6))
  Running setup.py (path:/tmp/pip_build_vagrant/psycopg2/setup.py) egg_info for package psycopg2

Downloading/unpacking django-currencies>=0.3.3 (from -r /vagrant/webapp/requirements/requirements.txt (line 7))
  Downloading django-currencies-0.3.3.tar.gz
  Running setup.py (path:/tmp/pip_build_vagrant/django-currencies/setup.py) egg_info for package django-currencies

Downloading/unpacking django-countries>=3.2 (from -r /vagrant/webapp/requirements/requirements.txt (line 8))
  Running setup.py (path:/tmp/pip_build_vagrant/django-countries/setup.py) egg_info for package django-countries

Downloading/unpacking stripe>=1.22.1 (from -r /vagrant/webapp/requirements/requirements.txt (line 9))
  Running setup.py (path:/tmp/pip_build_vagrant/stripe/setup.py) egg_info for package stripe

Downloading/unpacking pillow>=2.8.0 (from -r /vagrant/webapp/requirements/requirements.txt (line 10))
  Running setup.py (path:/tmp/pip_build_vagrant/pillow/setup.py) egg_info for package pillow
    Single threaded build, not installing mp_compile: 1 processes

    warning: no files found matching 'LICENSE' under directory 'docs'
Downloading/unpacking django-compressor>=1.4 (from -r /vagrant/webapp/requirements/requirements.txt (line 11))
Downloading/unpacking djangorestframework>=3.1.1 (from -r /vagrant/webapp/requirements/requirements.txt (line 12))
Downloading/unpacking django-filter>=0.9.2 (from -r /vagrant/webapp/requirements/requirements.txt (line 13))
  Downloading django_filter-0.9.2-py2.py3-none-any.whl
Obtaining formtools from git+https://github.com/django/django-formtools.git#egg=formtools (from -r /vagrant/webapp/requirements/requirements.txt (line 14))
  Cloning https://github.com/django/django-formtools.git to ./src/formtools
  Running setup.py (path:/tmp/src/formtools/setup.py) egg_info for package formtools

  Installing extra requirements: 'egg'
Downloading/unpacking django-widget-tweaks>=1.3 (from -r /vagrant/webapp/requirements/requirements.txt (line 15))
  Downloading django-widget-tweaks-1.3.tar.gz
  Running setup.py (path:/tmp/pip_build_vagrant/django-widget-tweaks/setup.py) egg_info for package django-widget-tweaks

Downloading/unpacking django-braces>=1.4.0 (from -r /vagrant/webapp/requirements/requirements.txt (line 16))
  Downloading django_braces-1.4.0-py2.py3-none-any.whl
Downloading/unpacking django-debug-toolbar>=1.3.0 (from -r /vagrant/webapp/requirements/dev.txt (line 4))
Downloading/unpacking python-openid>=2.2.5 (from django-allauth>=0.19.1->-r /vagrant/webapp/requirements/requirements.txt (line 5))
  Running setup.py (path:/tmp/pip_build_vagrant/python-openid/setup.py) egg_info for package python-openid

    warning: no files found matching 'CHANGELOG'
    warning: no files found matching '*.css' under directory 'doc'
    warning: no files found matching '*.html' under directory 'doc'
Downloading/unpacking requests-oauthlib>=0.3.0 (from django-allauth>=0.19.1->-r /vagrant/webapp/requirements/requirements.txt (line 5))
  Downloading requests_oauthlib-0.4.2-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): requests>=1.0.3 in /usr/lib/python2.7/dist-packages (from django-allauth>=0.19.1->-r /vagrant/webapp/requirements/requirements.txt (line 5))
Downloading/unpacking django-appconf>=0.4 (from django-compressor>=1.4->-r /vagrant/webapp/requirements/requirements.txt (line 11))
  Downloading django_appconf-1.0.1-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): six in /usr/local/lib/python2.7/dist-packages (from django-braces>=1.4.0->-r /vagrant/webapp/requirements/requirements.txt (line 16))
Downloading/unpacking sqlparse (from django-debug-toolbar>=1.3.0->-r /vagrant/webapp/requirements/dev.txt (line 4))
  Running setup.py (path:/tmp/pip_build_vagrant/sqlparse/setup.py) egg_info for package sqlparse

Downloading/unpacking oauthlib>=0.6.2 (from requests-oauthlib>=0.3.0->django-allauth>=0.19.1->-r /vagrant/webapp/requirements/requirements.txt (line 5))
  Running setup.py (path:/tmp/pip_build_vagrant/oauthlib/setup.py) egg_info for package oauthlib

Installing collected packages: django, django-allauth, psycopg2, django-currencies, django-countries, stripe, pillow, django-compressor, djangorestframework, django-filter, formtools, django-widget-tweaks, django-braces, django-debug-toolbar, python-openid, requests-oauthlib, django-appconf, sqlparse, oauthlib
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1435, in install
    requirement.install(install_options, global_options, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 671, in install
    self.move_wheel_files(self.source_dir, root=root)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 901, in move_wheel_files
    pycompile=self.pycompile,
  File "/usr/lib/python2.7/dist-packages/pip/wheel.py", line 206, in move_wheel_files
    clobber(source, lib_dir, True)
  File "/usr/lib/python2.7/dist-packages/pip/wheel.py", line 193, in clobber
    os.makedirs(destsubdir)
  File "/usr/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/Django-1.8.dist-info'

Storing debug log for failure in /home/vagrant/.pip/pip.log


FATAL: all hosts have already failed -- aborting

Answer 1

I solved the problem by setting the path explicitly and not using the $HOME variable.

- name: Setup virtualenv
  pip:
    virtualenv="/home/vagrant/.envs/{{ project_name }}"
    requirements="/vagrant/webapp/requirements/{{ requirements_filename }}"

Answer 2

It seems the pip module is using the system Python so it will try and install the package on the expected system path to site-packages and since you're not running this as root or using sudo it can't write to '/usr/local/lib/python2.7/...'

I'm assuming you want to install the pip package under the virtualenv, correct?

If so beware that Ansible modules always use, by default, the system default Python interpreter unless you set ansible_python_interpreter. An example of how to do this can be found on Best way to always run ansible inside a virtualenv on remote machines?

A link to a short discussion about the subject and why Ansible people don't want to change #!/usr/bin/python for the more virtualenv-friendly #!/usr/bin/env python can be found here.

As a note you'll have the same problem if you're running Ansible from a local virtualenv and try and use say ec2 module without having boto installed at the system-level (but installed at the virtualenv level).

Answer 3

Looks like it is using /usr/bin/pip, not pip from virtualenv. Not sure why this happens - a bug in Ansible? Can you manually check your virtualenv has pip command? One thing I can think out from my head pip is created for root (sudo) and you try to run the command as a normal user and it doesn't have read access to virtualenv'ed pip.

You can probably use pip manually like this for workaround.

- name: Create virtualenv for app using chosen Python
  shell: virtualenv --python=python3.4 {{deploy_location}}/venv
  sudo: yes

- name: Install Python packages
  sudo: yes
  command: {{deploy_location}}/venv/bin/pip -r {{deploy_location}}/requirements.txt 

Answer 4

Try specifying a pip executable path as well:

executable="$HOME/.envs/{{ project_name }}/bin/pip"

Didn't search for an exact reason TBH, but had to do the same to get stuff installed into virtual environments. Seemed like it tried to install into a virtual environment using global pip. Maybe that's possible to do under some circumstances, wasn't my case.