Cmake recompiles everything each time I add a new

(This question is quite similar to this one I asked but is not related to the GLOB subtleties. So separating the two problems would be clearer I think)

Each time I add or remove a folder in include_directory() (even empty) in my CMakeLists.txt, then typing

cmake .
make

triggers the whole project to be recompiled. Why and what can I do about it?

My actual CMakeLists.txt (note there is no GLOB):

cmake_minimum_required(VERSION 2.8)
cmake_policy(VERSION 2.8.12.2)

# Project
get_filename_component(ProjectName MyProject NAME)
project(${ProjectName})

# Compiler and options
set(CMAKE_CXX_COMPILER /usr/bin/clang++)
set(CLANG_COMPILE_FLAGS "-std=c++1y -stdlib=libc++ ")

# Type of build
set(CMAKE_BUILD_TYPE "Release")
# Build tool 
set(CMAKE_GENERATOR "Unix Makefiles")

set(EXECUTABLE_OUTPUT_PATH ../bin/${CMAKE_BUILD_TYPE})
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CLANG_COMPILE_FLAGS}")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${CLANG_LINK_FLAGS}")

include_directories(
    [all folders containing .h files here] # adding anything here triggers the whole project to recompile
)

add_executable(
    ${ProjectName}
    [all .cpp and .h files here]
)

Command include_directories add compiler flags (e.g. -I/path/to/include/dir for gcc). So if you change include_directories => compiler flags changes => target need to be rebuild.

Example

To avoid regular recompilation of whole project you need to not modify include_directories often. As an example you can organize code like this:

CMakeLists.txt
Source/
        - A/foo.hpp
        - A/foo.cpp
        - B/boo.hpp
        - B/boo.cpp

Content of CMakeLists.txt:

include_directories(./Source)
add_executable(myexe A/foo.hpp A/foo.cpp B/boo.hpp B/boo.cpp)

Content of C++ files:

#include "A/foo.hpp"
#include "B/boo.hpp"