How to configure portable parallel builds in CMake

Question

Is it somehow possible to be able to have a parallel build no matter which build tool is used?

Under Unix we can add make -jN where N are the number of threads and under Windows I added to the CXX_FLAG "/MP" which is then used in Visual Studio to parallel build...? How can I make my version such that CMAKE_MAKE_PROGRAM is not always extended when I run CMake?

What is a general solution?

I came up with this:

#Add some multithreaded build support
MARK_AS_ADVANCED(MULTITHREADED_BUILD)
set(MULTITHREADED_BUILD 12 CACHE STRING "How many threads are used to build the project")
if(MULTITHREADED_BUILD)
    if(${CMAKE_GENERATOR} MATCHES "Unix Makefiles")
            message(STATUS ${CMAKE_BUILD_TOOL})
            set(CMAKE_MAKE_PROGRAM "${CMAKE_MAKE_PROGRAM} -j${MULTITHREADED_BUILD}")
            message(STATUS "Added arguments to CMAKE_BUILD_TOOL: ${CMAKE_MAKE_PROGRAM}")
    elseif(MSVC)
      set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /MP")
      message(STATUS "Added parallel build arguments to CMAKE_CXX_FLAGS: ${CMAKE_CXX_FLAGS}")
    endif()
endif()

Answer 1

You can't do this cross-platform. The -jN option is a parameter to make, and not part of the generated Makefile. However, you could have CMake generate a bash script that runs make for your project using -jN (where the script looks up the number of cores you have).

Answer 2

As this post is already a bit old:

My approach i have settled down to is writing a parallelmake.sh script for Unix Makefiles based Generators This is done here: https://github.com/gabyx/ApproxMVBB

And the relevant parts in the the cmake file:

https://github.com/gabyx/ApproxMVBB/blob/master/CMakeLists.txt#L89

#Add some multithreaded build support =====================================================================================================
MARK_AS_ADVANCED(MULTITHREADED_BUILD)
SET(MULTITHREADED_BUILD ON CACHE BOOL "Parallel build with as many threads as possible!")
if(MULTITHREADED_BUILD)
    if(${CMAKE_GENERATOR} MATCHES "Unix Makefiles")
            file(COPY ${ApproxMVBB_ROOT_DIR}/cmake/parallelmake.sh DESTINATION ${PROJECT_BINARY_DIR}
                FILE_PERMISSIONS OWNER_READ OWNER_WRITE OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE
                NO_SOURCE_PERMISSIONS
            )
            SET(CMAKE_MAKE_PROGRAM "${PROJECT_BINARY_DIR}/parallelmake.sh")
            MESSAGE(STATUS "Set make program to ${PROJECT_BINARY_DIR}/parallelmake.sh")
    elseif(MSVC)
      SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}" "/MP")
      MESSAGE(STATUS "Added parallel build arguments to CMAKE_CXX_FLAGS: ${CMAKE_CXX_FLAGS}")
    endif()
endif()
# ========================================================================================================================================

Answer 3

If you have cmake v2.8.8 or higher, you may use ninja as an alternative of GNU make

mkdir build
cd    build
cmake -G Ninja ..
ninja              # Parallel build (no need -j12)

or

mkdir build
cd    build
cmake -G Ninja ..
cmake --build .    # Parallel build using ninja

As you can see, no need to use CMAKE_MAKE_PROGRAM, the build is run in parallel by default, optimizing the number of jobs depending on available CPU cores.

ninja is based on a low-level JSON configuration to speed up the startup phase. Therefore its JSON configuration is not easy to write by hand and I always generate it using a high-level tool/IDE:

• CMake v2.8.8 (2012)
• Qt Creator v2.6 (2012)
• KDevelop v4.6 (2013)
• ... see the generators of ninja configuration at https://github.com/ninja-build/ninja/wiki/List-of-generators-producing-ninja-build-files

As C++ build often requires lots of memory, your computer must provide as much memory as the number of CPU cores.

Answer 4

With CMake 3.12 this is possible. From the release notes:

The “cmake(1)” Build Tool Mode (“cmake –build”) gained “– parallel []” and “-j []” options to specify a parallel build level. They map to corresponding options of the native build tool.

Edit: As mentioned by @dkg you can also set the environment variable CMAKE_BUILD_PARALLEL_LEVEL.

Links to CMake's documentation:

• https://cmake.org/cmake/help/latest/manual/cmake.1.html#build-tool-mode
• https://cmake.org/cmake/help/latest/envvar/CMAKE_BUILD_PARALLEL_LEVEL.html