Autodependencies with GNU make Scott McPeak, November 2001

The Problem

.c

.h

.o

make

However, while make tends to do a good job of understanding the dependency of .o files upon .c files, it has no built-in facility for determining, nor convenient way of expressing, the dependencies on .h files. What's more, any solution must deal well with source files which are automatically generated.

This article outlines my solution to this problem, which is actually quite simple. I originally wrote this up because I thought it was original, but it turns out Paul Smith had already documented this solution.

The Situation

OBJS := foo.o bar.o # link proggie: $(OBJS) gcc $(OBJS) -o proggie # compile %.o: %.c gcc -c $(CFLAGS) $*.c -o $*.o # remove compilation products clean: rm -f proggie *.o

foo.c

bar.c

proggie

.o

.c

However, suppose that both foo.c and bar.c include foo.h . That means the respective .o files both depend on the contents of foo.h , but that fact isn't expressed in the Makefile. As a result, if the programmer changes foo.h , the program will likely be inconsistent when rebuilt.

One could of course add more lines like

foo.o: foo.h bar.o: bar.h

The Solution

.o

.d

.o

.d

.d

make

-MM

gcc

green

OBJS := foo.o bar.o # link proggie: $(OBJS) gcc $(OBJS) -o proggie # pull in dependency info for *existing* .o files -include $(OBJS:.o=.d) # compile and generate dependency info %.o: %.c gcc -c $(CFLAGS) $*.c -o $*.o gcc -MM $(CFLAGS) $*.c > $*.d # remove compilation products clean: rm -f proggie *.o *.d

.d

bar.d

bar.o: bar.c foo.h

make

bar.o

Note that .d files exist if and only if the corresponding .o file exists. This makes sense, since if the .o file doesn't exist yet, we don't need a .d file to tell us it has to be rebuilt.

More subtly, we never try to build a .d file until we have the necessary ingredients to build the corresponding .o file. This is important when a project has some source files built automatically (e.g. Bison output), since any attempt to build the .d files prematurely would fail.

The -include $(OBJS:.o=.d) syntax may need some explanation. First, $(OBJS:.o=.d) takes the value of $(OBJS) , and replaces all occurrences of .o at the end of a name with .d . Next, the leading hyphen ("-") means that if some .d files don't exist, make should continue without complaining (again, if the .d file doesn't exist, then neither will the .o file, so the .o file will be properly rebuilt).

An Improvement

foo.h

foo2.h

foo.c

bar.c

make

bar.o

foo.h

make clean

What is needed is a way to say that a particular prerequisite file, if missing, should be treated as changed (so the target will be rebuilt), but not cause an error. GNU make has an obscure feature that does just this: if a file (1) appears as a target in a rule with no prerequisites and no commands, and (2) that file does not exist and cannot be remade, then make will rebuild anything which depends on that file and not report an error. (See Chapter 4 of the make manual, "Rules without Commands or Prerequisites".)

To exploit this feature, dependency generation must rewrite the .d file to list one command-less, prerequisite-less rule for every file named as a dependency. There are several ways to do this; I choose to use a combination of sed and fmt . I've also chosen to prepend the at-sign ("@") to the new commands, so they won't get echoed when make is running (changes in green ):

OBJS := foo.o bar.o # link proggie: $(OBJS) gcc $(OBJS) -o proggie # pull in dependency info for *existing* .o files -include $(OBJS:.o=.d) # compile and generate dependency info; # more complicated dependency computation, so all prereqs listed # will also become command-less, prereq-less targets # sed: strip the target (everything before colon) # sed: remove any continuation backslashes # fmt -1: list words one per line # sed: strip leading spaces # sed: add trailing colons %.o: %.c gcc -c $(CFLAGS) $*.c -o $*.o gcc -MM $(CFLAGS) $*.c > $*.d @cp -f $*.d $*.d.tmp @sed -e 's/.*://' -e 's/\\$$//' < $*.d.tmp | fmt -1 | \ sed -e 's/^ *//' -e 's/$$/:/' >> $*.d @rm -f $*.d.tmp # remove compilation products clean: rm -f proggie *.o *.d

bar.o: bar.c foo.h bar.c: foo.h:

One More Tweak

make

gcc -MM

bar.o

dir/bar.o

For example, the system above might create

bar.o: dir/bar.c dir/foo.h dir/bar.c: dir/foo.h:

Makefile

bar.o

To work around this (arguably a bug in gcc ), one more sed command is needed in the block that builds the dependencies (changes in green again):

%.o: %.c gcc -c $(CFLAGS) $*.c -o $*.o gcc -MM $(CFLAGS) $*.c > $*.d @mv -f $*.d $*.d.tmp @sed -e 's|.*:|$*.o:|' < $*.d.tmp > $*.d @sed -e 's/.*://' -e 's/\\$$//' < $*.d.tmp | fmt -1 | \ sed -e 's/^ *//' -e 's/$$/:/' >> $*.d @rm -f $*.d.tmp

dir/bar.o: dir/bar.c dir/foo.h dir/bar.c: dir/foo.h:

Related Work

.d

The GNU make manual adopts Miller's solution. My hope is the technique described here might also make its way into the manual.

The inspiration for the close coupling between .o and .d file lifetimes is the Borland C++ compiler, which actually puts the dependency information into the .o file itself (and then optionally caches that info in the GUI). At one point, a company called CodeSourcery had a competition of sorts called Software Carpentry, for alternatives to traditional compile tools ( make among others). They have a good list of people's improvements to make on their build tools page (this link is now broken and I don't know of a replacement). You could also try saying "make replacement" to google.

After I originally wrote this, I learned that Paul Smith (the current maintainer of GNU make ) already wrote up this technique. My first solution had overlooked the command-less, prerequisite-less rule feature, and instead relied on a patch to make 's sources.

Links