make include directive and dependency generation w

Question

I want a build rule to be triggered by an include directive if the target of the include is out of date or doesn't exist.

Currently the makefile looks like this:

program_NAME := wget++
program_H_SRCS := $(wildcard *.h)
program_CXX_SRCS := $(wildcard *.cpp)
program_CXX_OBJS := ${program_CXX_SRCS:.cpp=.o}
program_OBJS := $(program_CXX_OBJS)

DEPS = make.deps

.PHONY: all clean distclean

all: $(program_NAME) $(DEPS)

$(program_NAME): $(program_OBJS)
    $(LINK.cc) $(program_OBJS) -o $(program_NAME)

clean:
    @- $(RM) $(program_NAME)
    @- $(RM) $(program_OBJS)
    @- $(RM) make.deps

distclean: clean

make.deps: $(program_CXX_SRCS) $(program_H_SRCS)
    $(CXX) $(CPPFLAGS) -MM $(program_CXX_SRCS) > make.deps

include $(DEPS)

The problem is that it seems like the include directive is executing before the rule to build make.deps which effectively means that make is either getting no dependency list if make.deps doesn't exist or always getting the make.deps from the previous build and not the current one.

For example:

$ make clean 
$ make
makefile:32: make.deps: No such file or directory
g++  -MM addrCache.cpp connCache.cpp httpClient.cpp wget++.cpp > make.deps
g++    -c -o addrCache.o addrCache.cpp
g++    -c -o connCache.o connCache.cpp
g++    -c -o httpClient.o httpClient.cpp
g++    -c -o wget++.o wget++.cpp
g++      addrCache.o connCache.o httpClient.o wget++.o -o wget++

Edit

I read the docs for the include directive, and it sounds like if the include target doesn't exist it will continue processing the parent makefile try and build the target, but it's not completely clear to me how this works:

If an included makefile cannot be found in any of these directories, a warning message is generated, but it is not an immediately fatal error; processing of the makefile containing the include continues. Once it has finished reading makefiles, make will try to remake any that are out of date or don't exist. See section How Makefiles Are Remade. Only after it has tried to find a way to remake a makefile and failed, will make diagnose the missing makefile as a fatal error.

ANSWER

This is a modification of the answer I accepted. The one thing missing was that the dependency files also depend on the sources, and won't get regenerated unless they are added to the deps files which are being included:

%.d: $(program_CXX_SRCS)
    @ $(CXX) $(CPPFLAGS) -MM $*.cpp | sed -e 's@^\(.*\)\.o:@\1.d \1.o:@' > $@

sed adds the name of the .d file to the beginning of each dependency line like so:

foo.d foo.o: foo.cpp foo.h bar.h baz.h

I got the idea from this amazing paper on the dangers of recursive make:

Recursive Make Considered Harmful

I also add the following to the makefile:

clean_list += ${program_SRCS:.c=.d}

# At the end of the makefile
# Include the list of dependancies generated for each object file
# unless make was called with target clean
ifneq "$(MAKECMDGOALS)" "clean"
-include ${program_SRCS:.c=.d}
endif

Answer 1

An important point that it took me a while to grasp is that the make.deps from the previous build are good enough. Think about it: for a given object file, the only way the list of dependency files can change is if... one of the old dependency files has been altered. And if that's the case, then the old make.deps will cause that object file to be rebuilt, and if rebuilding the object file also rebuilds make.deps, then everything will be up to date. You don't have to rebuild make.deps before checking to see which objects must be rebuilt.

Answer 2

You are relying on an implicit rule to compile your .cpp files. You have to redefine it to use the -MM and -MF flags that will create the dependency file.

%.o: %.cpp
    $(CXX) $(CPPFLAGS) $(CXXFLAGS) -c $< -o $@ -MM -MF $@.d

Then, you have to include these dependencies files in the Makefile, using -include that will not error when the dependencies files do not exist yet (on the first time, or after a clean).

program_DEPS := $(program_OBJS:.o=.o.d)
-include $(program_DEPS)

And remember to add the rm command for the dependencies files in the clean rule.

Answer 3

The include directives work like they do in C and C++ - they are processed before anything else happens, to build the "real" makefile that make then processes. Specifically, they are processed before any rules are fired.