Buildout and Virtualenv

Question

I am messing around with the combination of buildout and virtualenv to setup an isolated development environment in python that allows to do reproducible builds.

There is a recipe for buildout that let's you integrate virtualenv into buildout:

 tl.buildout_virtual_python

With this my buildout.cfg looks like this:

[buildout]
develop = .
parts = script
        virtualpython


[virtualpython]
recipe = tl.buildout_virtual_python
headers = true
executable-name = vp
site-packages = false

[script]
recipe = zc.recipe.egg:scripts
eggs = foo
python = virtualpython

This will deploy two executables into ./bin/:

vp
script

When I execute vp, I get an interactive, isolated python dialog, as expected (can't load any packages from the system). What I would expect now, is that if I run

./bin/script 

that the isolated python interpreter is used. But it doesn't, it's not isolated as "vp" is (meaning I can import libraries from system level). However I can run:

./bin/vp ./bin/script

Which will run the script in an isolated environment as I wished. But there must be a way to specify this to do so without chaining commands otherwise buildout only solves half of the problems I hoped :)

Thanks for your help! Patrick

Answer 1

zc.buildout 2.0 and later does not provide the isolated environment anymore.

But virtualenv 1.9 and later provides complete isolation (including to not install setuptools).

Thus the easiest way to get a buildout in a complete controlled environment is to run the following steps (here i.e. for Python 2.7):

cd /path/to/buildout
rm ./bin/python
/path/to/virtualenv-2.7 --no-setuptools --no-site-packages --clear .
./bin/python2.7 bootstrap.py
./bin/buildout

Preconditions:

• bootstrap.py has to be a recent one matching the buildout version you are using. You'll find the latest at http://downloads.buildout.org/2/

• if there are any version pins in your buildout, ensure they do not pin buildout itself or recipes/ extensions to versions not compatible with zc.buildout 2 or later.

Answer 2

You don't need virtualenv: buildout already provides an isolated environment, just like virtualenv.

As an example, look at files buildout generates in the bin directory. They'll have something like:

import sys
sys.path[0:0] = [
     '/some/thing1.egg',
     # and other things
     ]

So the sys.path gets completely replaced with what buildout wants to have on the path: the same isolation method as virtualenv.

Answer 3

I've never used that recipe before, but the first thing I would try is this:

[buildout]
develop = .
parts = script
        virtualpython


[virtualpython]
recipe = tl.buildout_virtual_python
headers = true
executable-name = vp
site-packages = false

[script]
recipe = zc.recipe.egg:scripts
eggs = foo
python = virtualpython
interpreter = vp

If that doesn't work, you can usually open up the scripts (in this case vp and script) in a text editor and see the Python paths that they're using. If you're on windows there will usually be a file called <script_name>-script.py. In this case, that would be vp-script.py and script-script.py.

Answer 4

Had issue running buildout using bootstrap on ubuntu server, from then I use virtualenv and buildout together. Simply create virualenv and install buildout in it. This way only virtualenv has to be installed into system (in theory¹).

$ virtualenv [options_you_might_need] virtual
$ source virtual/bin/activate
$ pip install zc.buildout
$ buildout -c <buildout.cfg>

Also tell buildout to put its scripts in to virtual/bin/ directory, that way scripts appear on $PATH.

[buildout]
bin-directory = ${buildout:directory}/virtual/bin
...

---

1: In practice you probably will need to eggs what require compilation to system level that require compilation. Eggs like mysql or memcache.