How to install distutils packages using distutils

Question

I'm working on a buildout script that needs to install a distutils package on remote server.

On PyPi there are 2 recipes for doing this collective.recipe.distutils 0.1 and zerokspot.recipe.distutils 0.1.1.

The later module a derivative of the former, and is a little more convenient then the first, but the both suffer from the same problem, which I will describe now.

When bootstrap.py is executed, it downloads zc.buildout package and puts it into buildout's eggs directory. This gives ./bin/buildout access to zc.buildout code, but /usr/local/python does not know anything about zc.buildout at this point.

Buildout attepts to install the package by running 'python setup.py install' inside of a subprocess. This produces an ImportError because zc.buildout is not installed for /usr/local/python.

So, I have several solutions.

1. Install zc.buildout using easy_install on the remote server. I don't like this option at all, it makes a special case for a module that is very insignificant.

2. Modify zerokspot.recipe.distutils to put try block around 'import zc.buildout' this way, it will install even if zc.buildout is not installed. It's an ok solution, but somewhat hackish.

3. Replace subprocess with code that will install the package using distutils api or setuptools api. This would be the best solution in my opinion.

The question is how would i do #3?

Thank you, Taras

PS: I solved the problem by creating another package that does not have dependancy on zc.buildout. My package is called taras.recipe.distutils and it's available on pypi.

Answer 1

Are you sure you don't want to just generate a bdist?

Answer 2

zerokspot.recipe.distutils is fundamentally broken in that it adds a dependency on zc.buildout in it's setup.py, as follows:

1. setup.py imports get_version from zerokspot.recipe.distutils
2. All of zerokspot.recipe.distutils is defined in it's __init__.py, including get_version
3. __init__.py in zerokspot.recipe.distutils imports zc.buildout

Why the author defines get_version is a mystery to me; best practice keeps a simple version string in setup.py itself and lets setuptools deal with dev versions (through setup.cfg), and distutils for version metadata extraction.

Generally it is not a good idea to import the whole package in setup.py as that would require all the package dependencies to be present at install time. Obviously the author of the package has zc.buildout installed as a site-wide package and didn't notice his oversight.

Your best bet is to fork the package on github, remove the get_version dependency, and propose the change to the original author while you use your fork instead.

Answer 3

You can call a command line program within your Python program using the subprocess module:

import subprocess
subprocess.call('python setup.py install')

However, how much control do you have over the environment that this install will be run? If it is a package that you are distributing, you will likely have issues no matter what solution people propose. How will you handle cases of needing root access (e.g. sudo python setup.py install)?

You may consider looking into Paver since it provides an API that is in some ways an extension of setuptools.