CMake and FindProtobuf

Question

I'm using the FindProtobuf module in a project where the protocol buffer files are in a sub-directory. I want the CMakeLists.txt file in that sub-directory to invoke protoc to generate the CPP Files. My project folder structure is like this:

cammy/CMakeLists.txt   # Root CMakeLists.txt
cammy/protofiles/test.proto # protofile
cammy/protofiles/CMakeLists.txt

I have the include(FindProtobuf), the find_package invocation and the call to PROTOBUF_GENERATE_CPP in the CMakeLists.txt file in protobuf folder.

The executable build step is in the Root CMakeLists.txt file and I add the generated files to the target executable in this file ie.

add_executable( ${EXEC} ${SRCS} ${PROTO_SRC} ${PROTO_HEADER} )
target_link_libraries( ${EXEC} ${PROTOBUF_LIBRARIES} )

are both defined in the root CMakeLists.txt

When I run cmake, it does not run protoc to generate the Source files even though I expilicitly tie generated sources to the executable thereby creating a dependency.

When I move all the contents of CMakeLists.txt in the protofiles folder into the root CMakeLists.txt, the proto files are compiled.

Can anyone help me with this? I want all the protocol buffer building stuff to go in the CMakeLists.txt file created in the protofiles folder.

I also noticed that variables generated in the inner CMakeLists.txt ( like PROTO_SRC ) are defined in the inner file when printed ( ie I get the correct generated CPP filename ) but when I print the same variable in the root file.. it is empty. Its almost as though I need to "export" (if there were a way in cmake ) the variables out to the root folder.

Any help would be much appreciated.

Thanks Kartik

Answer 1

I think FindProtobuf isn't really meant to be used this way. From its docs:

NOTE: The PROTOBUF_GENERATE_CPP macro & add_executable() or add_library() calls only work properly within the same directory.

You're trying to use the PROTOBUF_GENERATE_CPP macro in a subdirectory, and although the CMake docs don't really make it clear, a subdirectory introduces a new scope for variables. This means that any variables set or modified in the subdir scope don't affect similarly-named variables in the parent scope. Hence the reason for PROTO_SRC being available in your protofiles dir, but not in the parent.

The way to pass variables up a scope is to use set(... PARENT_SCOPE), so in protofiles/CMakeLists.txt you could do:

PROTOBUF_GENERATE_CPP(PROTO_SRC PROTO_HEADER test.proto)

set(PROTOBUF_INCLUDE_DIRS ${PROTOBUF_INCLUDE_DIRS} PARENT_SCOPE)
set(PROTOBUF_LIBRARIES ${PROTOBUF_LIBRARIES} PARENT_SCOPE)
set(PROTO_SRC ${PROTO_SRC} PARENT_SCOPE)
set(PROTO_HEADER ${PROTO_HEADER} PARENT_SCOPE)

However, this still doesn't get us all the way!

CMake doesn't actually invoke the protoc compiler to generate the .pb.h and .pb.cc files - it uses add_custom_command to do this. The custom command specifies the .pb.h and .pb.cc files as outputs, and the custom command is only invoked (i.e. protoc executed) if a subsequent target which depends on these files is built.

So, at configure time (when CMake executes) these files don't exist. This is a problem if you try to add them as sources to an add_library or add_executable command - CMake needs to be told that these files don't exist when it runs, but that they will exist at build time.

The way to do that is to set the GENERATED property to TRUE for these files. The PROTOBUF_GENERATE_CPP macro does that automatically, but as with the variables, the property isn't populated up into the parent scope. So in your top-level CMakeLists.txt, you also need to add:

set_source_files_properties(${PROTO_SRC} ${PROTO_HEADER} PROPERTIES
                            GENERATED TRUE)

As you can see, using PROTOBUF_GENERATE_CPP in a different dir to the corresponding add_library/add_executable commands is a bit fragile. If you can avoid doing it, you probably should.