References / useful Links
- GNU Make Manual.
- Recursive Make Considered Harmful, Peter Miller, 1998.
- Auto-Dependency Generation, Paul D. Smith, 2017.
- Makefile, Anonymous rm command executed at last, StackOverflow
- Writing a large build system with GNU make, by David Rothlisberger.
- Implementing non-recursive make, Emile van Bergen.
A Makefile describes how your program is built by specifying everything that it depends on and building the dependencies first. A Makefile rule looks like this:
target_1 ... target_n: prerequisite_1 ... prerequisite_m <tab> command_1 # NOTE that each command <tab> ... # is executed in its own <tab> command_j # subshell.
- Target is anything the must be made.
- Prerequisite are things that must be built before the target can be built.
- Commands are shell commands used to build the target. NOTE, each command executed in own subshell!
- Rule is what's shown above - it defines how to build a target.
To perform a build, make will construct a directed acyclic graph (DAG) from the rules. It will then do a post order traversal (visit children first), building the leaf nodes first and going back up the graph.
A very simple example is shown below:
The makefile for the above diagram would look something like this:
target: prequisite_1 prequisite_1 prequisite_1 <tab> commands_A prequisite_1: prequisite_4 <tab> commands_B prequisite_2: prequisite_5 <tab> commands_C prequisite_3: prequisite_5 prequisite_7 <tab> commands_D prequisite_4: <tab> commands_E prequisite_5: prequisite_6 <tab> commands_F prequisite_6: <tab> commands_G prequisite_7: <tab> commands_H
The post-order traversal is shown by the orange-background numbers.
- Make looks at
targetand visits the first child,
prerequisit_1. It can see that
prerequisit_4. So now,
prerequisit_1becomes the "target". If make determines that
prerequisit_1is newer than
prerequisit_4, it will build
prerequisit_4. We assume that it does and builds
prerequisit_4is built, all of
prerequisit_1's dependencies have been visited so
prerequisit_1can now be built using
- Make returns back to
targetand examines the next dependency,
prerequisit_5is older than
prerequisit_6is older than
prerequisit_5, the post-order traversal will result in make next trying to build
- All of
prerequisit_5's dependencies have been built so make can now build
prerequisit_2's dependencies have been built so make can now build
- Make returns back to
targetand examines the next dependency
prerequisit_3. There are two dependencies,
prerequisit_7. Make has already built
prerequisit_5so it knows that it does not need to build this again, so it ignores this path... nice! Thus all that is left is to look at
prerequisit_7, assuming it is older than
prerequisit_3. Assume it is, so make builds it.
prerequisit_3's dependencies have been built so it can now be built.
- Make returns back to
targetand because all of its dependencies have been built, it can finally be built.
By doing this kind of traversal make ensures that everything that is need to be built is built, but not more than this. I.e., if some dependencies do not need to be refreshed, they are not rebuilt, helping to produce an efficient build. It is also the case that targets that are as new as all of their dependencies are not rebuilt.
The basic make process can be described like this:
- Make finds files indicated by the targets and prerequisites.
- If a prerequisite has a rule associated with it (i.e., make is looking at the rule
target: prerequisiteand the rule
prerequisite: dependenciesexists), make will try to update the prerequisite first.
- When considering a target, if any prerequisite is newer than the target, make will attempt to make the prequisite(s) first.
A special type of target is a
.PHONY target, which is always out of date and
thus always rebuilt.
How Make Locates Source Files:
When a rule references a file without a path Make looks in the current working directory.
Few projects are housed in a single directory. One solution is to use
VPATH to instruct
Make to look in other directories for files.
VPATH = dir1 dir2 dir3 ...
This works for source files, i.e.
*.c, *.cpp etc but not includes. For those modifdy
Only targets and dependencies are searched using
VPATH. Paths in commands are not!
Beware of same-named files in different directories. Make just uses the first one it sees, so this can cause problems!
VPATH special variable is course-grained. It applies to all files. To be more specific
vpath pattern dir-list
For example, to search for C files only under the
src directory and headers
only under the
include directory one could write the following:
vpath %.h include vpath %.c src
Make does its work in two phases:
- Reads Makefile and all included makefiles. Load vars and rules into internal DB. Simple-expanded vars expanded. Create DAG.
- Analyse DAG, recursively-expanded vars expanded, determined targets to update, update include targets first and possibly restart, then execute commands for all other rules.
Makefile Automatic Variables
This is just a summary of some of the more commonly used automatic variables. You can find a complete list here. The automatic variables do not have very user-friendly names so I have a few memory pegs I try to use to recall what they all mean...
||The file name of the target of the rule. Remember a rule looks like
||The name of the first prerequisite. Memory peg: The chevron looks like it "points" to a target, so think prerequisite.|
||The names of all the prerequisites that are newer than the target. Memory peg: It's a question... what prerequisite is newer?|
||The names of all the prerequisites (duplicated removed). Memory peg: it's pointing up, so all of the above prerequisite.|
Makefile Pattern Rules
In the example makefile in the introduction section, each target was explicitly written down. This would be fine for a small project, but if you have more than a few source files, listing each one independently will soon get tedious and error prone. Most of the time we just want to say "and object file is generated from a c/c++ source file with the same file base name". And luckily we can... to say this we would write something like the following.
%.o: %.cpp <tab> ...Commands... my-target: fileA.o fileB.o ... fileZ.o
In the above example, Make knows that to build
my-target it must build the
fileZ.o. So, how does it construct
these objects? It looks through all of the rules it has encountered so far and tries to
file?.o with a target. It finds
is a wild card. The
% matches the portion
file? of the
prerequisite (this portion is called the stem), so Make can use this rule to
construct the object file.
fileA.o as an example. Make will search for the file. If the file is
as new or newer than the target it knows that this prerequisite does not need to be rebuilt.
However, if it does, Make searches for a rule that will tell it how to build the prerequisite.
Make finds the rule
%.o: %.c; commands:
%.o, where the
fileA. The prerequisite
for this rule specified
%.c, which substituting in the match will become
fileA.c. As long as
fileA.c is just a file and not generated
by some other tool, Make will find no rule to create this file, so if the file doesn't
exist, then Make reports an error, otherwise it will rebuild
fileA.c if it
is newer than the target
If two or more rules match Make will prefer more specific rules over more generic ones.
Limiting Scope Of Pattern Rules
To limit the scope of a pattern rule that includes wild cards to a set of files use the following:
$(OBJECTS) : %.o : %.c
In the above the
%.o will only match object filesnames in the variable
OBJECTSand thus the pre-requisites will only match the corresponding objects
Make includes a set of default rules for commonly required types of dependencies. For example, you will generally not need to tell make how to compile a C or C++ program as its default rule database already has this information.
See the default rule database by typing (you can ommit the pipe to vim if you want, but the output is many screens long):
make -p -q | vim - ^ ^ ^ Don't execute any commands Print rule database
If you search the output for C/C++ rules you will find the following rules, amougst others, which are presented here possibly out-of-order wrt to the Make db print out:
OUTPUT_OPTION = -o $@ LINK.c = $(CC) $(CFLAGS) $(CPPFLAGS) $(LDFLAGS) $(TARGET_ARCH) LINK.o = $(CC) $(LDFLAGS) $(TARGET_ARCH) LINK.cpp = $(LINK.cc) LINK.cc = $(CXX) $(CXXFLAGS) $(CPPFLAGS) $(LDFLAGS) $(TARGET_ARCH) COMPILE.cpp = $(COMPILE.cc) COMPILE.cc = $(CXX) $(CXXFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c COMPILE.c = $(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c %: %.o $(LINK.o) $^ $(LOADLIBES) $(LDLIBS) -o $@ %: %.c $(LINK.c) $^ $(LOADLIBES) $(LDLIBS) -o $@ %.o: %.c $(COMPILE.c) $(OUTPUT_OPTION) $<
So, if a Makefile has a rule
Make will find the rule
%.o: %.c and execute the rule
$(COMPILE.c) $(OUTPUT_OPTION) $<.
This expands to
$(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c -o $@ $<, which expands to
$(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c -o file1.o file1.c.
So, what do the variables such as
CPPFLAGS expand to? These must be
set by the makefile creator to meet the needs of their compilation!
||By default this is set to
||Flags/Switches that should be added to the C compiler command line only, i.e., not shared with C++ compiler.|
||Flags that should be passed to either or both of the C/C++ compilers. This generally means that they are preprocessor flags as they are generally the only thing that would be passed to bothcompilers.|
||Flags to pass to the linker. Normall add the library search pathes here:
||Itsa deprecated, but still supported, alternative to
||Specifies the libraries to link against and possibly other flags.|
||Flags/Switches that should be passed to the C++ compiler command line only, i.e., not shared with C compiler.|
||Use to select a specific architecture being compiled for [Ref]. For example, default PC comilation produces code that can run on any x86 architectire. So here one might add
Constants or environment variables in upper snake case.
Variables internam to the make file, lower snake case.
Variables are either simple expanded or recursively expanded.
Simple expanded variables are declared like so:
SIMPLE_EXPANDED := some-value # ^^ # Note the colon before the equals! # some-value is expanded IMMEDIATELY upon reading this line
Simple expanded variables have their RHS expression expanded immediately upon being encountered, during the 1st make phase, and have the value of the expansion assigned to the variable.
Recursively expanded variables are declared like so:
RECUSIVE_EXPANDED = some-value # ^^ # Note ONLY the equals! # some-value is not expanded. just stored as-is in the variable without evaluating it!
A recusively expanded variable just has the RHS stored as-is in the variable without any evaluation what so ever. The value is only ever expanded when the variable is USED.
When Expansion Occurs
- LHS of variable assignments immediately expanded
- RHS of
?=expansion deferred until second phase
- RHS of
- RHS of
+=immediately expanded for variables declared with
:=, otherwise expansion/evaluation deferred until use
- Rule targets and prerequisites immediately expanded. Commands deferred.
Target Specific Variables
Can append to or create variables specifically for a given target:
target: my_variable (=|:=|?=|+=) value target: prerequisites <tab> commands # ^^ # my_variable now has a value specific to this target.
Including Other Make Files Into The Main Makefile
Can include functionality from other Makefiles into the main Makefile using the
Importantly if any include file is updated by a file, make clears its internal database and re-reads the
entire makefile! This is especially important for dependency generation.
The flow of logic that dictates when a Makefile is re-processed due to a rule-based update to an include target is shown below:
This process is especially important for dependency generation which is covered in its own section.
Notes taken from this article.
DEPDIR := .deps DEPFLAGS = -MT $@ -MMD -MP -MF $(DEPDIR)/$*.d # ^^^^^^ ^^^^ ^^^ ^^^^^^^^^^^^^^^^^^ # ^^^^^^ ^^^^ ^^^ Specify files to write the dependencies to # ^^^^^^ ^^^^ ^^^ # ^^^^^^ ^^^^ Instructs CPP to add a phony target for each dependency # ^^^^^^ ^^^^ other than the main file, causing each to depend on nothing. # ^^^^^^ ^^^^ These dummy rules work around errors make gives if you remove # ^^^^^^ ^^^^ header files without updating the Makefile to match. # ^^^^^^ ^^^^ # ^^^^^^ Instead of outputting the result of preprocessing, output a rule # ^^^^^^ suitable for make describing the dependencies of the main source # ^^^^^^ file. Do NOT include system header files as dependencues. # ^^^^^^ AND # ^^^^^^ Generate dependency information as a side-effect of compilation, # ^^^^^^ not instead of compilation. # ^^^^^^ # Change target of generated rule to $@ rather than the default that would # otherwise be the name of the main input file with directories deleted and # suffix replaced with platform specific object suffix. COMPILE.c = $(CC) $(DEPFLAGS) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c %.o : %.c %.o : %.c $(DEPDIR)/%.d | $(DEPDIR) $(COMPILE.c) $(OUTPUT_OPTION) $< $(DEPDIR): ; @mkdir -p $@ DEPFILES := $(SRCS:%.c=$(DEPDIR)/%.d) $(DEPFILES): include $(wildcard $(DEPFILES)) # ^^ # As seen in prev section if any of the include targets (i.e., the DEPFILES) were # updated by the rules the makefile is reloaded with a cleared db.
Separate Source And Binary
CMake calls this sort of the out-of-source-builds. Need these to cope with multiple targets and multiple builds per target, for example. To accomplish need to have source and output binaries in separate locations!