As part of my unit testing I want to ensure code coverage of the tests. The aim is to place something like REQUIRE_TEST
macros somewhere in the code and check whether all of these were called.
void foo(bool b) {
if (b) {
REQUIRE_TEST
...
} else {
REQUIRE_TEST
...
}
}
void main() {
foo(true);
output_all_missed_REQUIRE_macros();
}
Ideally the output would include source-file and line of the macro.
My initial idea was to have the macros create static objects that would register themselves in some map and later check whether all of these were called
#define REQUIRE_TEST \
do { \
static ___RequiredTest requiredTest(__func__, __FILE__, __LINE__);\
(void)requiredTest;\
___RequiredTest::increaseCounter(__func__, __FILE__, __LINE__);\
} while(false)
but the static object are only created when the code is called the first time. So the map only contains functions that are also counted in the next line - missing REQUIRE_TEST macros are not found. the __attribute__((used))
is ignored in this case.
The gcc has a nice attribute __attribute__((constructor))
, but apparently chooses to ignore it when placed here (following code instead of the static object)
struct teststruct { \
__attribute__((constructor)) static void bla() {\
___RequiredTest::register(__func__, __FILE__, __LINE__); \
} \
};\
as well as for
[]() __attribute__((constructor)) { \
___RequiredTest::register(__func__, __FILE__, __LINE__); \
};\
The only way out I can think of now is a) manually (or via script) analyse the code outside of the regular compilation (uargh) or b) use the __COUNTER__
macro to count the macros - but then I would not know which specific REQUIRE_TEST macros were not called... (and everything breaks if somebody else decides to use the __COUNTER__
macro as well...)
Are there any decent solutions to this problem? What am I missing? It
would be nice to have a macro that appends the current line and file
so some preprocessor variable whenever it is called - but that is not
possible, right? Are there any other ways to register something to be
executed before main()
that can be done within a function body?
How about this:
#include <iostream>
static size_t cover() { return 1; }
#define COV() do { static size_t cov[2] __attribute__((section("cov"))) = { __LINE__, cover() }; } while(0)
static void dump_cov()
{
extern size_t __start_cov, __stop_cov;
for (size_t* p = &__start_cov; p < &__stop_cov; p += 2)
{
std::cout << p[0] << ": " << p[1] << "\n";
}
}
int main(int argc, char* argv[])
{
COV();
if (argc > 1)
COV();
if (argc > 2)
COV();
dump_cov();
return 0;
}
Results:
$ ./cov_test
19: 1
22: 0
25: 0
and:
$ ./cov_test x
19: 1
22: 1
25: 0
and:
$ ./cov_test x y
19: 1
22: 1
25: 1
Basically, we set up a coverage array in a named memory section (obviously we've used GCC-specific mechanisms for this), which we dump after execution.
We rely on constant initialisation of local statics being performed at startup - which gets the line numbers into the coverage array with coverage flag set to zero - and on initialisation via the function call to cover()
being performed at first execution of the statement, which sets the coverage flags to 1 for executed lines. I'm not 100% sure all this is guaranteed by the standard, nor by which versions of the standard (I compiled with --std=c++11
).
Finally, building with '-O3' also produces correct results (albeit allocated in a different order):
$ ./a
25: 0
22: 0
19: 1
and:
$ ./a x
25: 0
22: 1
19: 1
and
$ ./a x y
25: 1
22: 1
19: 1
An ugly but simple approach is for REQUIRE_TEST
to use __LINE__
or __COUNTER__
to construct the name of a unique file-scope static object which it refers to, and which will cause a compiler error if it hasn't been declared. You then need to manually declare all such objects up front, one for each REQUIRE_TEST
- but at least you get a compile error if you haven't done so.
Hey, I said it was ugly!
Jeremy gave me the right idea to have a closer look at the sections. His answer works - but only without inline
functions. With some more research I was able to find the following solution which is just as much gcc dependent (due to the name of the section) but more flexible (and works in inline functions). The macro is now as follows:
#define REQUIRE_TEST \
do { \
struct ___a{ static void rt() {\
___RequiredTest::register_test(__FILE__, __LINE__);\
} };\
static auto ___rtp __attribute__((section(".init_array"))) = &___a::rt; \
(void) ___rtp; \
___RequiredTest::increase_counter(__FILE__, __LINE__); \
} while(false)
Placing the function pointer in the section .init_array
actually places it in the list of initialization functions that are being called before main. This way it can be assured, that the locally defined function is being called before main.