automake and project dependencies

Question

I have a project that I want to build using automake. The project consists of different components or modules, and there are inter module dependencies which require the project to be built in a specific order.

For example:

project dir/
  module1 (core C shared lib)
  module2 (C++ shared lib wrapper around module 1)
  module3 (C++ application with dependency on module2)
  module4 (C library with dependency on module1)
  module5 (C application with dependency on module4)

I am relatively new to automake, but I (just about) know how to use it to successfully build a single project.

I would like to have a 'master' project file (if that's possible), which specifies the build order of the projects modules, runs unit tests and fails the entire build process if either:

• One of the modules fails to build
• One of the modules fails a unit test

How would I go about writing such a 'master project' file (or invoking any other mechanism) to build projects that have a lot of inter-modular dependencies?

Answer 1

If you're using autotools, then you might as well use automake. The top level Makefile.am can provide a list of subdirectories that are descended in order, e.g:

SUBDIRS = module1 module2 module3 module4 module5

The subdirectory Makefile.am can add targets for tests, invoked with 'make check', which will force the build if necessary:

check_PROGRAMS = t_module1

t_module1_SOURCES = t_module1.c
t_module1_LDADD = ./libmodule1.la

There's a lot to learn, and best current practice can be difficult to determine, but if you're using autotools, you'll be used to this situation.

EDIT:

info automake provides the reference documentation - but it makes a poor tutorial. One of the best guides I've come across can be found here.

Answer 2

I've encountered the same issue and found that a pure autotools solution is very hard to get running, because the configure script e.g. for module4 depends on the installation of module1.

A hand-rolled Makefile and configure script for this situation is fairly easy to generate. I've pasted below the rapidSTORM project Makefile. It is used for out-of-tree build (source directory and a build directory).

TARGETS=any_iterator libb64 readsif cs_units dStorm-doc simparm andorcamd rapidSTORM plugin-andorsif fitter master

all:

# Project dependencies: Any project whose configure run depends upon other projects has a line here
andorcamd.prerequisites-installed : $(addsuffix .installed-stamp,libb64 simparm cs_units)
rapidSTORM.prerequisites-installed : $(addsuffix .installed-stamp,simparm cs_units libb64 any_iterator)
plugin-andorsif.prerequisites-installed : $(addsuffix .installed-stamp,rapidSTORM readsif)
master.prerequisites-installed fitter.prerequisites-installed : $(addsuffix .installed-stamp,rapidSTORM)

# [Autoconf substitutions snipped here]
# The .options files control configuration of subdirectories. They are used in %.configured-stamp 
vpath %.options $(srcdir)/options:$(builddir)

RULES = all check install installcheck dist distcheck 

# All standard rules have a simple template: Execute them for each
# subdirectory after configuring it and installing all prerequisite
# packages, and re-execute them whenever
# the source files changed. install and distcheck are special and
# treated further below.   
define recursive_rule_template
 $(1) : $(foreach target,$(TARGETS),$(target).$(1)ed-stamp)
endef
define standard_rule_template
 %.$(1)ed-stamp : %.source-change-stamp %.configured-stamp %.prerequisites-installed
    make -j 4 -C $$* $(1) && touch $$@
endef

$(foreach rule,$(RULES),$(eval $(call recursive_rule_template,$(rule))))
$(foreach rule,$(filter-out install distcheck,$(RULES)),$(eval $(call standard_rule_template,$(rule))))

%.installed-stamp : %.alled-stamp 
    make -C $* install && touch $@

# This rule is probably the most complex. It collects option files named after a
# number of options and generates configure flags from them; this rule could be 
# shortened considerably when you don't need project-specific configure/CFLAGS
# configuration.
%.configured-stamp : $(foreach i, all $(host_config) $(tag) $(host_config)-$(tag), global-$i.options) \
    $(foreach i, all $(host_config) $(tag) $(host_config)-$(tag),%-$i.options) | %.prerequisites-installed
    prefix="$(prefix)"; abs_builddir=$(abs_builddir); \
    for i in $(filter %.options,$^); do . ./$$i; done; \
        mkdir -p $* && cd $* \
        && echo "Configuring with $$OPTIONS CPPFLAGS=$$CPPFLAGS CFLAGS=$$CFLAGS CXXFLAGS=$$CXXFLAGS LDFLAGS=$$LDFLAGS PKG_CONFIG_PATH=$$PKG_CONFIG_PATH" INSTALL="$(INSTALL)" \
        && /bin/sh ../$(srcdir)/$*/configure --build=$(build_alias) --host=$(host_alias) --target=$(target_alias) --config-cache $$OPTIONS \
        CPPFLAGS="$$CPPFLAGS" CFLAGS="$$CFLAGS" CXXFLAGS="$$CXXFLAGS" PKG_CONFIG_PATH="$$PKG_CONFIG_PATH" \
        LDFLAGS="$$LDFLAGS" $(if $(CC),CC=$(CC),) $(if $(CXX),CXX=$(CXX),) \
        INSTALL="$(INSTALL)"
    touch $@

# The source change stamp is updated whenever a file in the source directory changes.
# It is used to prevent non-necessary sub-make invocations.
%.source-change-stamp : always-renew
    { test -e $@ && find $(srcdir)/$* -newer $@ -and -not -ipath '*/.svn*' -and -not -path '*/.libs*' | wc -l | grep -q '^0$$'; } \
        || touch $@

%.prerequisites-installed :
    @true

%.distchecked-stamp : %.source-change-stamp %.configured-stamp %.prerequisites-installed
    DISTCHECK_CONFIGURE_FLAGS=`./$*/config.status --config | sed -e "s/'--prefix=[^']*' //"` \
        $(MAKE) -j 4 -C $* distcheck && touch $@

Makefile : $(srcdir)/Makefile.in config.status
    ./config.status $@

installcheck : dejagnu-tests-ran-stamp

dejagnu-tests-ran-stamp : $(foreach target,$(TARGETS),$(target).installed-stamp) testsuite.configured-stamp
    make -C testsuite check
    touch $@

always-renew :
    @true

clean :
    rm -rf *-stamp $(foreach target,$(TARGETS),$(target)/*.la $(target)/config.cache) deploy

realclean : clean
    rm -rf $(TARGETS) 

%.options : 
    touch $@

world : $(foreach target,$(TARGETS),$(foreach rule,$(RULES),$(target).$(rule)ed-stamp))

.PHONY : always-renew
.SECONDARY :
.DELETE_ON_ERROR :