Makefile w. Protocol Buffer and Automatic Dependency

makefile, protocol-buffers

Solution

Another solution is to make your dependency files depend on the results of the protocol buffer generation. The following snippet contains all steps to do that, since it is hard to explain them one by one, with an explanation of some items at the bottom:

CXX_FLAGS      := $(shell pkg-config --cflags protobuf) -xc++
LD_FLAGS       := $(shell pkg-config --libs protobuf)

PROTOS         := $(wildcard *.proto)
PROTO_OBJS     := $(PROTOS:.proto=.pb.o)

BINS     := my_binary

SRCS     := $(wildcard *.cu)
OBJS     := $(SRCS:.cu=.o)
DEPS     := $(SRCS:.cu=.d)

PBSRCS   := $(wildcard *.proto)
PBOBJS   := $(PROTOS:.proto=.pb.o)
PBGENS   := $(PROTOS:.proto=.pb.cc) $(PROTOS:.proto=.pb.h)

all: $(BINS)

clean:
        rm -f $(BINS) $(OBJS) $(DEPS) $(PBOBJS) $(PBGENS)

$(BINS): $(OBJS)
$(OBJS): $(DEPS)
$(DEPS): $(PBOBJS)

.PRECIOUS: $(PBGENS)

%.d: %.cu
        $(CXX) -M $(CXX_FLAGS) $< > $@

%.pb.cc: %.proto
        protoc --cpp_out=. $<

%.pb.o : %.pb.cc
        $(CXX) $(CXX_FLAGS) -c -o $@ $<

%.o: %.cu
        $(CXX) $(CXX_FLAGS) -c -o $@ $<

$(BINS): %: %.o
        $(CXX) $(LD_FLAGS) -o $@ $(PROTO_OBJS) $^

ifneq ($(MAKECMDGOALS),clean)
-include $(DEPS)
endif

The `pkg-config` command is not required, but is convenient if you want to automatically obtain compilation and linking flags relevant for the protobuf files. Of course, you have to add your own flags to this variable.

The `-xc++` is probably useless for you, but used here in order to be able to work with `.cu` files and have them interpreted as C++; even for compilers different than `nvcc`.

The line `$(DEPS): $(PBOBJS)` is the indication that the protobuf files should be created and compiled before the dependencies are created. There a several ways to achieve this, so this is just an example of how to do it.

The `.PRECIOUS` line instructs `make` to keep the generated protobuf files. In this example snippet, those files are considered intermediate and as such would be deleted without this line.

I posted this as a separate answer, because the previous one and this one do not have much in common.

Problem

I have a Makefile something like this: ``` .SECONDARY: NVCC = nvcc NVCCFLAGS = --gpu-architecture compute_20 CXX = g++ CXXFLAGS = -O3 -std=c++0x -Wall CXXLINT = python cpplint.py CXXLINTFLAGS = --filter=-build/include,-readability/streams,-runtime/sizeof,-whitespace/parens PROTOC = protoc PROTOCFLAGS = --cpp_out=. BINS = my_binary LIBS = -lcublas -lcusparse PROTOS = $(wildcard *.proto) SOURCES = $(wildcard *.cu) HEADERS = $(wildcard *.cuh) PBS = $(PROTOS:%.proto=%.pb) DEPS = $(SOURCES:%.cu=%.d) TESTS = my_test all: lint protos all: deps all: bins protos: ${PBS} deps: ${DEPS} bins: ${BINS} clean: rm -rf *.o *.d *.pb.cc *.pb.h ${BINS} ${TESTS} lint: ${CXXLINT} ${CXXLINTFLAGS} ${SOURCES} ${CXXLINT} ${CXXLINTFLAGS} ${HEADERS} tests: lint protos tests: deps tests: ${TESTS} tests: tests-run tests-run: ${TESTS} for f in $^; do eval "/usr/bin/time -f \"$$f runtime: %E\" ./$$f"; done %: %.o ${NVCC} ${NVCCFLAGS} -o $@ $^ ${LIBS} %.d: %.cu # ${CXXLINT} ${CXXLINTFLAGS} $*.cu ${NVCC} -M -o $*.d $*.cu %.o: %.cu ${NVCC} ${NVCCFLAGS} -c -o $@ $*.cu rm $*.d %.pb: %.proto ${PROTOC} ${PROTOCFLAGS} $*.proto ${CXX} ${CXXFLAGS} -c -o $*.pb.o $*.pb.cc ifneq ($(MAKECMDGOALS),clean) -include ${DEPS} endif ``` The problem occurs since I can not generate my DEPS until my proto target is built. Because building the protocol buffers will add a new header file to the tree, if this isn't done first before the DEPS, the nvcc -M (make dependency) will fail since it can not find the *.pb.h that is generated. Any ideas how to fix this?

Original source