=================== More about GNU make =================== When you are using an __integrated development environment__ (IDE) for software development you initially have to select whether your project is an application or a library. Roughly spoken writing an application means that you write a program for a user to solve a specific problem. Whereas writing a library means to produce software that other developers can use for an implementation of an application or another (eventually more high-level) library. In this class the focus is rather on developing libraries than applications. However, you can consider that as the more general case. Every library should have tests for its implementation, and as tests are executable programs we also develop applications. The difference is that our applications are more limited in there scope (each test just checks some small portion of the library) and we have many tests. Why you should cope with makefiles ================================== Dealing with GNU make in more detail can be motivated as giving you an idea what happens in an IDE when you click a run or build button. And also giving you a principal understanding what kind of preferences each IDE provides for configuring the underlying build system. In my opinion teaching the general concepts make more sense than teaching a particular IDE. While there are many IDEs out there details differ and will change. Once you learned and understood the underlying concept you easily can adapt to the special case. We need some pet project ======================== For developing some useful software for the ULM we need some kind of a __C standard library__. In its initial stage it contains just three functions that are implemented in separate translation units: - The startup code that initializes the stack, calls `main` and halts the ULM with the return value of `main`. :import: session11/mini-lib/crt0.s [fold] - Procedure `puts` prints a string. :import: session11/mini-lib/puts.s [fold] - Procedure `putui` prints an unsigned integer. :import: session11/mini-lib/putui.s [fold] Whatever the library implements needs to be tested. For that we initially have two programs: - `xhello` is testing `puts` by printing "hello, world". ---- SHELL (path=session11/mini-lib) ----------------------------------------- xhello ------------------------------------------------------------------------------ :import: session11/mini-lib/xhello.s [fold] - `xanswer` is testing `putui` by printing 42. ---- SHELL (path=session11/mini-lib) ----------------------------------------- xanswer ------------------------------------------------------------------------------ :import: session11/mini-lib/xanswer.s [fold] Simplifications ~~~~~~~~~~~~~~~ It certainly would be a good idea to separate source files that are part of the library form those that are used for testing. But this will make things in the beginning unnecessarily complicated (it in general requires __hierarchical makefiles__, i.e. makefiles that call makefiles etc.). Still we somehow have to distinguish in the build system between these two categories. A practical solution is using naming conventions. If a source file begins with the character 'x' then it's assumed to be a test program and otherwise part of the library. For now we also make the oversimplified assumption that all tests depend on all object files of the library part. So all our tests will contain code from all the object files even if some of the code is not needed. We take care of this unnecessarily large executables later by using bigger guns (in form of __static libraries__). ---- TIKZ ---------------------------------------------------------------------- \begin{tikzpicture} \input{flowchart.tex} \renewcommand\BoxWidth {2.3} \renewcommand\FlowColDist {3.6} \renewcommand\BoxHeight {1.2} \renewcommand\BoxDistance {2.0} \SetMargin{1}{12}{1}{3} \Group{0}{0}{1}{3}{Executables } \Group{2}{0}{4}{3}{Library} \renewcommand\FlowCol {0} \PutStatement{0}{xanswer.s } \PutStatement{1}{xanswer.o} \PutStatement{3}{xanswer} \AddDepPath{0}{1}{0}{0}{} \AddDepPath{0}{3}{0}{1}{} \renewcommand\FlowCol {1} \PutStatement{0}{xhello.s } \PutStatement{1}{xhello.o} \PutStatement{3}{xhello} \AddDepPath{1}{1}{1}{0}{} \AddDepPath{1}{3}{1}{1}{} \renewcommand\FlowCol {2} \PutStatement{0}{crt0.s } \PutStatement{1}{crt0.o} \AddDepPath{2}{1}{2}{0}{} \renewcommand\FlowCol {3} \PutStatement{0}{puts.s } \PutStatement{1}{puts.o} \AddDepPath{3}{1}{3}{0}{} \renewcommand\FlowCol {4} \PutStatement{0}{putui.s } \PutStatement{1}{putui.o} \AddDepPath{4}{1}{4}{0}{} \AddDepPath{1}{3}{2}{1}{} \AddDepPath{1}{3}{3}{1}{} \AddDepPath[red,right=1.9cm]{1}{3}{4}{1}{simplification: actually xhello does not depend on putui.o} \AddDepPath{0}{3}{2}{1}{} \AddDepPath{0}{3}{3}{1}{} \AddDepPath{0}{3}{4}{1}{} \end{tikzpicture} -------------------------------------------------------------------------------- Translating and linking manually -------------------------------- ---- SHELL (path=session11/mini-lib) ------------------------------------------- ulmas -o xhello.o xhello.s ulmas -o xanswer.o xanswer.s ulmas -o crt0.o crt0.s ulmas -o puts.o puts.s ulmas -o putui.o putui.s ulmld -o xhello xhello.o crt0.o puts.o putui.o ulmld -o xanswer xanswer.o crt0.o puts.o putui.o -------------------------------------------------------------------------------- Motivation to consider static libraries next -------------------------------------------- Check that `xhello` contains the object code `putui.o` which is never used. Yes, look into the executable! Typical use-cases for developing a library ------------------------------------------ We want a build system that allows to simply add new source files to the library, and also allows to add new tests. You don't want to write makefiles like this =========================================== This is a handwritten makefile in the style of Quiz 13 from the previous page. It is certainly not suitable when you want to add new source files to your project. But that is what we start with. Because it is easy to see that it satisfies the dependencies stated in the above graph: :import: session11/make01/Makefile [fold] Check the following things: - If you modify `puts.s` then `puts.o` gets updated, i.e. the assembler gets called. Also `xhello` and `xanswer` gets update by using the linker. Hence the following needs to be done: ---- CODE (type=txt) ---------------- ulmas -o puts.o puts.s ulmld -o xhello xhello.o crt0.o puts.o putui.o ulmld -o xanswer xanswer.o crt0.o puts.o putui.o ------------------------------------- - If you modify `xhello.s` only `xhello.o` get updated with the assembler and the used to update `xhello`. Hence the following needs to be done: ---- CODE (type=txt) ---------------- ulmas -o xhello.o xhello.s ulmld -o xhello xhello.o crt0.o puts.o putui.o ------------------------------------- Now check that the makefile is doing the right thing. Use the `touch` command to update the modification time of a file, i.e. let make think a file was modified: ---- SHELL (path=session11/make01) --------------------------------------------- make touch puts.s make touch xhello.s make make clean -------------------------------------------------------------------------------- Using variables =============== In a next step we make only minor improvements to the makefile. We use __variables__ to define what command gets used to translate assembly code and to link object files: ---- CODE (type=txt) ----------------------------------------------------------- AS := ulmas LD := ulmld TestTargets := xhello xanswer -------------------------------------------------------------------------------- With `$(AS)` and `$(LD)` we can refer to the values of these variables in the rest of the makefile, in particular the rules. Note that GNU make support different __flavors of variables__. Here we use the flavor you get when you define the variable with `:=`. Which simply means that it has a fixed value, and you can use the variable _after_ its definition. The other flavor will be used (and motivated) on the next page when we build a static library. Selecting in one place what tools get used to do the job is certainly a good ides. In an IDE you usually can select somewhere in the preference what tool for building a project can be used, e.g. which C compiler (Gnu C compiler, Intel compiler, Microsoft compiler, Sun Compiler, etc.). But there is more about it. The identifiers `AS` and `LD` where not chosen arbitrary. In an upcoming step we will exploit that these are __variables used by implicit rules__. For now just observe that the rules will be changed to something like this: ---- CODE (type=txt) ----------------------------------------------------------- xanswer: xanswer.o crt0.o puts.o putui.o $(LD) -o $@ $^ -------------------------------------------------------------------------------- Variables `$@` and `$^` are so called __automatic variables__. In the command you can refer with `$@` to the target of the rule (i.e. the left hand side of the colon) , and with `$^` you can refer to the list of dependencies (i.e. the right hand side of the colon). So altogether this is equivalent to ---- CODE (type=txt) ----------------------------------------------------------- xanswer: xanswer.o crt0.o puts.o putui.o ulmld -o xanswer xanswer.o crt0.o puts.o putui.o -------------------------------------------------------------------------------- Now check that you understand the rest of the makefile: :import: session11/make02/Makefile Also check that it is acting as before: ---- SHELL (path=session11/make02, fold) --------------------------------------- make touch puts.s make touch xhello.s make make clean -------------------------------------------------------------------------------- Using pattern rules =================== Using __pattern rules__ we can reduces the size of the makefile. In the makefile we define ---- CODE (type=txt) ----------------------------------------------------------- %.o: %.s $(AS) -o $@ $^ -------------------------------------------------------------------------------- Other rules like ---- CODE (type=txt) ----------------------------------------------------------- xanswer: xanswer.o crt0.o puts.o putui.o $(LD) -o $@ $^ -------------------------------------------------------------------------------- will trigger the above pattern rule for checking the dependencies of its object files, so altogether this makefile is equivalent to before: :import: session11/make03/Makefile ---- SHELL (path=session11/make03, fold) --------------------------------------- make touch puts.s make touch xhello.s make make clean -------------------------------------------------------------------------------- Using static pattern rules ========================== We also want to generalize the rules for building the tests. In a first attempt we would probably try the following pattern rule: ---- CODE (type=txt) ----------------------------------------------------------- %: %.o crt0.o puts.o putui.o $(LD) -o $@ $^ -------------------------------------------------------------------------------- This should work because the first rule ---- CODE (type=txt) ----------------------------------------------------------- all: $(TestTargets) -------------------------------------------------------------------------------- has dependencies `xhello` and `xanswer` (these are the values of `TestTargets`). Hence the pattern rule would match when `make` substitutes `%` accordingly. However, there are so called __implicit rules__ (see below) built-in, and one of them is ---- CODE (type=txt) ----------------------------------------------------------- %: %.s # recipe to execute (built-in): $(LINK.s) $^ $(LOADLIBES) $(LDLIBS) -o $@ -------------------------------------------------------------------------------- If more than one rule matches the one with fewer dependencies is preferred. To work around this issue we could use the more restrictive pattern rule ---- CODE (type=txt) ----------------------------------------------------------- x%: x%.o crt0.o puts.o putui.o $(LD) -o $@ $^ -------------------------------------------------------------------------------- This works for us because only source files beginning with the character are test programs. A more general solution can be achieved with __static pattern rules__. With ---- CODE (type=txt) ----------------------------------------------------------- $(TestTargets): %: %.o crt0.o puts.o putui.o $(LD) -o $@ $^ -------------------------------------------------------------------------------- we define an dedicated rule for the targets listed in $(TestTargets). In our case this is just `xhello` and `xanswer` but overall we have a solution that works for an arbitrary list of targets: :import: session11/make04/Makefile ---- SHELL (path=session11/make04, fold) --------------------------------------- make touch puts.s make touch xhello.s make make clean -------------------------------------------------------------------------------- Using wildcards and text functions ================================== So far we explicitly list the test programs in the makefile with ---- CODE (type=txt) ----------------------------------------------------------- TestTargets := xhello xanswer -------------------------------------------------------------------------------- And also the objects files for the library in this rule: ---- CODE (type=txt) ----------------------------------------------------------- $(TestTargets): % : %.o crt0.o puts.o putui.o $(LD) -o $@ $^ -------------------------------------------------------------------------------- Using __wildcards__ and __text functions__ we can make let create this list automatically. In the following makefile `$(wildcard x*.s)` expands to the list all files that begin with 'x' and have the extension '.s'. Currently that is `xhello.s` and `xanswer.s`. When new test programs are added the list automatically adapts. With ---- CODE (type=txt) ----------------------------------------------------------- LibSources := $(filter-out x%.s,$(wildcard *.s)) -------------------------------------------------------------------------------- We get a list of all source files for the library. The `wildcard` function is first used to get all files with extension '.s' and the `filter-out` function to remove files that begin with 'x'. With that the variable `LibSources` is set to the space-separated list with entries `crt0..s`, `puts.s` and `putui.s`. Finally the `patsubst` (pattern substitution) function is used to replace the extension '.s' with '.o'. With that the variable `LibObjects` is set to the list of object files for the library. Hence with the following makefile allows that new files for library of test programs simply can be added to the directory: :import: session11/make05/Makefile ---- SHELL (path=session11/make05, fold) --------------------------------------- make touch puts.s make touch xhello.s make make clean -------------------------------------------------------------------------------- Using implicit rules ==================== Above __implicit rules__ where already mentioned above. With `make -p` you get a (long) list of buit-in rules: ---- SHELL (path=session11/make05, fold) --------------------------------------- make -p -------------------------------------------------------------------------------- This contains ---- CODE (type=txt) ----------------------------------------------------------- %.o: %.s # recipe to execute (built-in): $(COMPILE.s) -o $@ $< -------------------------------------------------------------------------------- and the definition ---- CODE (type=txt) ----------------------------------------------------------- COMPILE.s = $(AS) $(ASFLAGS) $(TARGET_MACH) -------------------------------------------------------------------------------- The variables `ASFLAGS` and `TARGET_MACH` are in our case not defined and therefore will be expanded to empty strings. Hence this default rule is equivalent to our rule ---- CODE (type=txt) ----------------------------------------------------------- %.o: %.s $(AS) -o $@ $^ -------------------------------------------------------------------------------- and we simply remove it: :import: session11/make06/Makefile ---- SHELL (path=session11/make06, fold) --------------------------------------- make touch puts.s make touch xhello.s make make clean -------------------------------------------------------------------------------- Phony targets ============= A __phony target__ is one that is not really the name of a file; rather it is just a name for a recipe to be executed when you make an explicit request. In our case these are the targets `all` and `clean`: :import: session11/make07/Makefile ---- SHELL (path=session11/make07, fold) --------------------------------------- make touch puts.s make touch xhello.s make make clean -------------------------------------------------------------------------------- :links: pattern rules -> https://www.gnu.org/software/make/manual/html_node/Pattern-Rules.html variables -> https://www.gnu.org/software/make/manual/html_node/Using-Variables.html#Using-Variables automatic variables -> https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html#Automatic-Variables variables used by implicit rules -> https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html#Implicit-Variables flavors of variables -> https://www.gnu.org/software/make/manual/html_node/Flavors.html#Flavors static pattern rules -> https://www.gnu.org/software/make/manual/html_node/Static-Pattern.html#Static-Pattern wildcards -> https://www.gnu.org/software/make/manual/html_node/Wildcards.html#Wildcards text functions -> https://www.gnu.org/software/make/manual/html_node/Text-Functions.html#Text-Functions implicit rules -> https://www.gnu.org/software/make/manual/html_node/Implicit-Rules.html#Implicit-Rules phony target -> https://www.gnu.org/software/make/manual/html_node/Phony-Targets.html#Phony-Targets integrated development environment -> https://en.wikipedia.org/wiki/Integrated_development_environment C standard library -> https://en.wikipedia.org/wiki/C_standard_library hierarchical makefiles -> https://www.gnu.org/software/make/manual/html_node/Recursion.html static libraries -> https://en.wikipedia.org/wiki/Static_library