Automatic version number both in setup.py (setupto

Question

SITUATION:

I have a python library, which is controlled by git, and bundled with distutils/setuptools. And I want to automatically generate version number based on git tags, both for setup.py sdist and alike commands, and for the library itself.

For the first task I can use git describe or alike solutions (see How can I get the version defined in setup.py (setuptools) in my package?).

And when, for example, I am in a tag '0.1' and call for 'setup.py sdist', I get 'mylib-0.1.tar.gz'; or 'mylib-0.1-3-abcd.tar.gz' if I altered the code after tagging. This is fine.

THE PROBLEM IS:

The problem comes when I want to have this version number available for the library itself, so it could send it in User-Agent HTTP header as 'mylib/0.1-3-adcd'.

If I add setup.py version command as in How can I get the version defined in setup.py (setuptools) in my package?, then this version.py is generated AFTER the tag is made, since it uses the tag as a value. But in this case I need to make one more commit after the version tag is made to make the code consistent. Which, in turns, requires a new tag for further bundling.

THE QUESTION IS:

How to break this circle of dependencies (generate-commit-tag-generate-commit-tag-...)?

Answer 1

Following OGHaza's solution in a similar SO question I keep a file _version.py that I parse in setup.py. With the version string from there, I git tag in setup.py. Then I set the setup version variable to a combination of version string plus the git commit hash. So here is the relevant part of setup.py:

from setuptools import setup, find_packages
from codecs import open
from os import path
import subprocess

here = path.abspath(path.dirname(__file__))

import re, os
VERSIONFILE=os.path.join(here,"_version.py")
verstrline = open(VERSIONFILE, "rt").read()
VSRE = r"^__version__ = ['\"]([^'\"]*)['\"]"
mo = re.search(VSRE, verstrline, re.M)
if mo:
    verstr = mo.group(1)
else:
    raise RuntimeError("Unable to find version string in %s." % (VERSIONFILE,))
if os.path.exists(os.path.join(here, '.git')):
    cmd = 'git rev-parse --verify --short HEAD'
    git_hash = subprocess.check_output(cmd)
    # tag git
    gitverstr = 'v' + verstr
    tags =  subprocess.check_output('git tag')
    if not gitverstr in tags:
        cmd = 'git tag -a %s %s -m "tagged by setup.py to %s"' % (gitverstr, git_hash, verstr)        
        subprocess.check_output(cmd)
    # use the git hash in the setup
    verstr += ', git hash: %s' % git_hash

setup(
    name='a_package',
    version = verstr,
    ....

Answer 2

Eric's idea was the simple way to go, just in case this is useful here is the code I used (Flask's team did it this way):

import re
import ast

_version_re = re.compile(r'__version__\s+=\s+(.*)')

with open('app_name/__init__.py', 'rb') as f:
    version = str(ast.literal_eval(_version_re.search(
        f.read().decode('utf-8')).group(1)))

setup(
    name='app-name',
    version=version,
 .....
)

Answer 3

You could also reverse the dependency: put the version in mylib/__init__.py, parse that file in setup.py to get the version parameter, and use git tag $(setup.py --version) on the command line to create your tag.

git tag -a v$(python setup.py --version) -m 'description of version'

Is there anything more complicated you want to do that I haven’t understood?

Answer 4

A classic issue when toying with keyword expansion ;)

The key is to realize that your tag is part of the release management process, not part of the development (and its version control) process.

In other word, you cannot include a release management data in a development repository, because of the loop you illustrates in your question.

You need, when generating the package (which is the "release management part"), to write that information in a file that your library will look for and use (if said file exists) for its User-Agent HTTP header.

Answer 5

Since this topic is still alive and sometimes gets to search results, I would like to mention another solution which first appeared in 2012 and now is more or less usable:

https://github.com/warner/python-versioneer

It works in different way than all mentioned solutions: you add git tags manually, and the library (and setup.py) reads the tags, and builds the version string dynamically.

The version string includes the latest tag, distance from that tag, current commit hash, "dirtiness", and some other info. It has few different version formats.

But it still has no branch name for so called "custom builds"; and commit distance can be confusing sometimes when two branches are based on the same commit, so it is better to tag & release only one selected branch (master).