I want to use ctest from the command line to run my tests with memcheck and pass in arguments for the memcheck command.
I can run ctest -R my_test
to run my test, and I can even run ctest -R my_test -T memcheck
to run it through memcheck.
But I can't seem to find a way to pass arguments to that memcheck command, like --leak-check=full
or --suppressions=/path/to/file
.
After reading ctest's documentation I've tried using the -D
option with CTEST_MEMCHECK_COMMAND_OPTIONS
and MEMCHECK_COMMAND_OPTIONS
. I also tried setting these as environment variables. None of my attempts produced any different test command. It's always:
Memory check command: /path/to/valgrind "--log-file=/path/to/build/Testing/Temporary/MemoryChecker.7.log" "-q" "--tool=memcheck" "--leak-check=yes" "--show-reachable=yes" "--num-callers=50"
How can I control the memcheck command from the ctest command line?
TL;DR
ctest --overwrite MemoryCheckCommandOptions="--leak-check=full --error-exitcode=100" \
--overwrite MemoryCheckSuppressionFile=/path/to/valgrind.suppressions \
-T memcheck
Explanation
I finally found the right way to override such variables, but unfortunately it's not easy to understand this from the documentation.
So, to help the next poor soul that needs to deal with this, here is my understanding of the various ways to set options for memcheck
.
In a CTestConfig.cmake
in you top-level source dir, or in a CMakeLists.txt
(before calling include(CTest)
), you can set MEMORCHECK_COMMAND_OPTIONS
or MEMORYCHECK_SUPPRESSIONS_FILE
.
When you include(CTest)
, CMake will generate a DartConfiguration.tcl
in your build directory and setting the aforementioned variables will populate MemoryCheckCommandOptions
and MemoryCheckSuppressionFile
respectively in this file.
This is the file that ctest
parses in your build directory to populate it's internal variables for running the memcheck
step.
So, if you'd like to set you project's options for memcheck during cmake configuration, this is the way to got.
If instead you'd like to modify these options after you already have a properly configured build directory, you can:
- Modify the DartConfiguration.tcl directly, but note that this will be overwritten if cmake runs again, since this file is regenerated each time cmake runs.
- Use the ctest
--overwrite
command-line option to set these memcheck options just for that run.
Notes
- I've seen mentions online of a
CMAKE_MEMORYCHECK_COMMAND_OPTIONS
variable. I have no idea what this variable is and I don't think cmake is aware of it in any way.
- Setting
CTEST_MEMORYCHECK_COMMAND_OPTIONS
(the variable that is actually documented in the cmake docs) in your CTestConfig.cmake
or CMakeLists.txt
has no effect. It seems this variable only works in "CTest Client Scripts", which I have never used.
- Unfortunately, both
MEMORCHECK_COMMAND_OPTIONS
and MEMORYCHECK_SUPPRESSIONS_FILE
aren't documented explicitly in cmake, only indirectly, in ctest documentation and the Testing With CTest tutorial.
When ctest
is run in the build, it parses the file to populate internal variables:
https://cmake.org/cmake/help/latest/manual/ctest.1.html#dashboard-client-via-ctest-command-line
It's not clear to me how this interacts with