How to pass CMAKE_GENERATOR to a CMake external pr

Question

I am trying to compile the TensorFlow CMake build as an external project on Windows. This requires the 64-bit toolchain to not run out of memory during compilation. However, even though the CMAKE_GENERATOR for the top-level project is Visual Studio 14 2015 Win64, and this is passed to the external project via the CMAKE_GENERATOR variable, the 32 bit toolchain is used to compile the external project.

What I observe: When the build is started, a 64 bit MSBuild process spawns to drive the main project as expected. This can be seen in the task manager. When building the TensorFlow external project however, an MSBuild (32 bit) process start for the external project. The compiler later fails with the error: c1060: compiler is out of heap space.

The CMake code for the TensorFlow external project is

ExternalProject_Add(TensorFlow
  GIT_REPOSITORY ${TensorFlow_REPOSITORY}
  SOURCE_DIR ${TensorFlow_SOURCE_DIR}
  SOURCE_SUBDIR tensorflow/contrib/cmake
  BINARY_DIR ${TensorFlow_BINARY_DIR}
  CMAKE_GENERATOR ${CMAKE_GENERATOR}
  CMAKE_ARGS
    --no-warn-unused-cli
    -DBUILD_SHARED_LIBS:BOOL=${BUILD_SHARED_LIBS}
    -DSWIG_EXECUTABLE:FILEPATH=${SWIG_EXECUTABLE}
    -DPYTHON_EXEUTABLE:FILEPATH=${PYTHON_EXECUTABLE}
    -DPYTHON_LIBRARIES:LIST=${PYTHON_LIBRARIES}
    -Dtensorflow_WIN_CPU_SIMD_OPTIONS=/arch:AVX
  UPDATE_COMMAND ""
  DEPENDS ${TensorFlow_DEPENDENCIES}
)

The output from the TensorFlow configuration stage is

Creating directories for 'TensorFlow'
Performing download step (git clone) for 'TensorFlow'
Cloning into 'TensorFlow'...
Checking out files: 100% (7280/7280), done.
Already on 'master'
Your branch is up-to-date with 'origin/master'.
No update step for 'TensorFlow'
No patch step for 'TensorFlow'
Performing configure step for 'TensorFlow'
Not searching for unused variables given on the command line.
-- The C compiler identification is MSVC 19.0.24215.1
-- The CXX compiler identification is MSVC 19.0.24215.1
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe -- wor
ks
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test COMPILER_OPT_ARCH_NATIVE_SUPPORTED
-- Performing Test COMPILER_OPT_ARCH_NATIVE_SUPPORTED - Failed
-- Performing Test COMPILER_OPT_WIN_CPU_SIMD_SUPPORTED
-- Performing Test COMPILER_OPT_WIN_CPU_SIMD_SUPPORTED - Success
-- Found PythonInterp: C:/Users/kasper/Anaconda3/python.exe (found version "3.5.2")
-- Found PythonLibs: C:/Users/kasper/Anaconda3/libs/python35.lib (found version "3.5.2")
-- Found SWIG: C:/Development/build/t/SWIG/swig.exe (found version "3.0.10")
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Development/build/t/TensorFlow-build

Then adding message(STATUS "CMAKE_GENERATOR: ${CMAKE_GENERATOR") to the top-level TensorFlow CMakeLists.txt and running cmake . in the TensorFlow-build directory gives

-- CMAKE_GENERATOR: Visual Studio 14 2015 Win64
-- Configuring done
-- Generating done

The value of CMAKE_GENERATOR in the TensorFlow external project is Visual Studio 14 2015 Win64. I also tried to add -G ${CMAKE_GENERATOR} to CMAKE_ARGS but to no avail. Task manager still shows MSBuild (32 bit) and Microsoft Compiler/Linker Driver (32 bit) when the external project starts.

The code can be found here: https://github.com/kaspermarstal/TensorFlowImageFilter.

What am I missing?