Building multiple executables with similar rules

build, build-automation, c++, scons

Solution

You can actually do this with a few lines of GNU Make.

Below are two makefiles that allow building and cleaning from `all_lessons` directory and individual project directories. It assumes that all C++ sources in that directory comprise an executable file which gets named after its directory. When building and cleaning from the top level source directory (`all_lessons`) it builds and cleans all the projects. When building and cleaning from a project's directory it only builds and cleans the project's binaries.

These makefiles also automatically generate dependencies and are fully parallelizable (`make -j` friendly).

For the following example I used the same source file structure as you have:

$ find all_lessons
all_lessons
all_lessons/even_or_odd
all_lessons/even_or_odd/main.cpp
all_lessons/Makefile
all_lessons/helloworld
all_lessons/helloworld/lesson.cpp
all_lessons/helloworld/main.cpp
all_lessons/project.mk
all_lessons/calculator
all_lessons/calculator/lesson.cpp
all_lessons/calculator/user_created_add.cpp
all_lessons/calculator/main.cpp

To be able to build from individial project directories `project.mk` must be symlinked as `project/Makefile` first

[all_lessons]$ cd all_lessons/calculator/
[calculator]$ ln -s ../project.mk Makefile
[helloworld]$ cd ../helloworld/
[helloworld]$ ln -s ../project.mk Makefile
[even_or_odd]$ cd ../even_or_odd/
[even_or_odd]$ ln -s ../project.mk Makefile

Let's build one project:

[even_or_odd]$ make
make -C .. project_dirs=even_or_odd all
make[1]: Entering directory `/home/max/src/all_lessons'
g++ -c -o even_or_odd/main.o -Wall -Wextra   -MD -MP -MF even_or_odd/main.d even_or_odd/main.cpp
g++ -o even_or_odd/even_or_odd even_or_odd/main.o  
make[1]: Leaving directory `/home/max/src/all_lessons'
[even_or_odd]$ ./even_or_odd
hello, even_or_odd

Now build all projects:

[even_or_odd]$ cd ..
[all_lessons]$ make
g++ -c -o calculator/lesson.o -Wall -Wextra   -MD -MP -MF calculator/lesson.d calculator/lesson.cpp
g++ -c -o calculator/user_created_add.o -Wall -Wextra   -MD -MP -MF calculator/user_created_add.d calculator/user_created_add.cpp
g++ -c -o calculator/main.o -Wall -Wextra   -MD -MP -MF calculator/main.d calculator/main.cpp
g++ -o calculator/calculator calculator/lesson.o calculator/user_created_add.o calculator/main.o  
g++ -c -o helloworld/lesson.o -Wall -Wextra   -MD -MP -MF helloworld/lesson.d helloworld/lesson.cpp
g++ -c -o helloworld/main.o -Wall -Wextra   -MD -MP -MF helloworld/main.d helloworld/main.cpp
g++ -o helloworld/helloworld helloworld/lesson.o helloworld/main.o  
[all_lessons]$ calculator/calculator 
hello, calculator
[all_lessons]$ helloworld/helloworld
hello, world

Clean one project:

[all_lessons]$ cd helloworld/
[helloworld]$ make clean
make -C .. project_dirs=helloworld clean
make[1]: Entering directory `/home/max/src/all_lessons'
rm -f helloworld/lesson.o helloworld/main.o helloworld/main.d helloworld/lesson.d helloworld/helloworld
make[1]: Leaving directory `/home/max/src/all_lessons'

Clean all projects:

[helloworld]$ cd ..
[all_lessons]$ make clean
rm -f calculator/lesson.o calculator/user_created_add.o calculator/main.o even_or_odd/main.o helloworld/lesson.o helloworld/main.o calculator/user_created_add.d calculator/main.d calculator/lesson.d even_or_odd/main.d  calculator/calculator even_or_odd/even_or_odd helloworld/helloworld

The makefiles:

[all_lessons]$ cat project.mk 
all :
% : forward_ # build any target by forwarding to the main makefile
    $(MAKE) -C .. project_dirs=$(notdir ${CURDIR}) $@
.PHONY : forward_

[all_lessons]$ cat Makefile 
# one directory per project, one executable per directory
project_dirs := $(shell find * -maxdepth 0 -type d )

# executables are named after its directory and go into the same directory
exes := $(foreach dir,${project_dirs},${dir}/${dir})

all : ${exes}

#  the rules

.SECONDEXPANSION:

objects = $(patsubst %.cpp,%.o,$(wildcard $(dir ${1})*.cpp))

# link
${exes} : % : $$(call objects,$$*) Makefile
    g++ -o $@ $(filter-out Makefile,$^) ${LDFLAGS} ${LDLIBS}

# compile .o and generate dependencies
%.o : %.cpp Makefile
    g++ -c -o $@ -Wall -Wextra ${CPPFLAGS} ${CXXFLAGS} -MD -MP -MF ${@:.o=.d} $<

.PHONY: clean

clean :
    rm -f $(foreach exe,${exes},$(call objects,${exe})) $(foreach dir,${project_dirs},$(wildcard ${dir}/*.d)) ${exes}

# include auto-generated dependency files
-include $(foreach dir,${project_dirs},$(wildcard ${dir}/*.d))

Problem

I am writing something like an interactive tutorial for C++. The tutorial will consist of two parts: one is compiled into a library (I'm using Scons to build that), and the other (the lessons) is shipped with the tutorial to be compiled by the end user. I'm currently looking for a good, easy way for people to build these lessons. Basically, the second part is a directory with all the lessons in it, each in its own directory. Each lesson will have at least a `lesson.cpp` and a `main.cpp` file, there may be also other files, the existence of which I will not know until after it is shipped -- the end user will create these. It will look something like this: ``` all_lessons/ helloworld/ lesson.cpp main.cpp even_or_odd/ lesson.cpp main.cpp calculator/ lesson.cpp main.cpp user_created_add.cpp ``` Each of these will need to be compiled according to almost the same rules, and the command for compiling should be possible to run from one of the lesson directories (`helloworld/`, etc.). Seeing as the rest of the project is built with Scons, it would make sense to use it for this part, too. However, Scons searches for the `SConstruct` file in the directory it is run from: would it be acceptable to put a `SConstruct` file in each lesson directory, plus a `SConscript` in the `all_lessons/` directory that gives the general rules? This seems to go against the typical way Scons expects projects to be organised: what are the potential pitfalls of this approach? Could I put a SConstruct file instead of the SConscript one, and thereby make it possible to build from either directory (using exports to avoid endless recursion, I'm guessing)? Also, I may at some point want to replace the `lesson.cpp` with a `lesson.py` that generates the necessary files; will Scons allow me to do this easily with builders, or is there a framework that would be more convenient? In the end, I want to end up with the following (or equivalent with different build systems): ``` all_lessons/ SConstruct helloworld/ SConstruct lesson.cpp main.cpp even_or_odd/ SConstruct lesson.py main.cpp calculator/ SConstruct lesson.cpp main.cpp user_created_add.cpp ``` Running `scons all` in the `all_lessons` directory would need to: - Run `even_or_odd/lesson.py` to generate `even_or_odd/lesson.cpp`. - Realise that `user_created_add.cpp` also needs to be compiled. - Produce an executable for each lesson. Running `scons` in `even_or_odd/`, or `scons even_or_odd` in `all_lessons/` should produce an executable identical to the one above (same compile flags). Summary: - Is Scons suitable for/capable of this? - Does Scons work well when `SConscript` files are above `SConstruct` files? - Does Scons work well with multiple `SConstrcut` files for one project, SConscripting each other? - Is the Scons builder system suitable for using Python scripts to generate C++ files? - Is there any advantage of using a different build system/writing my own build framework that I'm missing? Any further comments are, of course, welcome. Thanks.

Original source