Data-driven unit tests with google test

Question

I am currently writing unit tests for an embedded application using googles unit test framework. Now my boss got upset that the data I test with (i.e. the values with which I call methods of the class under test) is hard wired in the tests. He requests to have this data read-in from a file. His argument is that it would thus be easier to add another test for a corner case that was previously forgotten. I am not that experienced with unit tests but so far that was not how I did it. So I tried to figure out what would be the best way to do it - even if it is a good idea to do it at all. I quickly came across the DDT (data-driven testing) approach.

The google unit test framework has a feature they call "Value-Parameterized Tests". With that my test fixture becomes a template class and I can pass in parameters. However, I see some problems with this:

• I currently have a fixture per class under test. But I would need a fixture per method under test since each method requires a different set of parameters. This will be a lot of extra work.
• As I see it I can pass in only a single parameter. Since I need several ones for my tests (all parameters for my method plus the expected results) This would require me to pass in something like a vector or map. Again this construction and retrieval sounds like a lot of work.

I would have imagined that something as mature as the google test framework to make it easier. However, they write

value-parameterized tests come handy [when] you want to test your code over various inputs (a.k.a. data-driven testing). This feature is easy to abuse, so please exercise your good sense when doing it!

Additionally there exists this blogpost TotT: Data Driven Traps, also warning me of (abusing) data-driven unit tests.

So my question comes down to:

• Is it a good idea to do data driven unit tests?
• How to do data-driven unit tests with google test framework

I am not really bound to googletest and basically free to choose any framework I'd like, though.

EDIT

I found the following statement in an FAQ entry of the googletest FAQs

Google Test doesn't yet have good support for [...] data-driven tests in general. We hope to be able to make improvements in this area soon.

Answer 1

GTest has support for it - but maybe they do not know...

Use testing::ValuesIn - like in this simplified example:

class SomeTests : public TestWithParam<int>
{
public:

};

TEST_P(SomeTests, shouldBePositive)
{
    ASSERT_GT(GetParam(), 0);
}

And - the way how to get values from input stream:

std::ifstream inputValuesFromFile("input.txt");
INSTANTIATE_TEST_CASE_P(FromFileStream,
                        SomeTests,
                        ValuesIn(std::istream_iterator<int>(inputValuesFromFile), 
                                 std::istream_iterator<int>()));

To use more complicated types than "int" you need to write an operator >> for it - like:

struct A
{
    int a;
    std::vector<int> b;
}
std::istream& operator >> (std::istream& is, A& out)
{
     std::size_t bSize;
     if ((is >> A.a) && (is >> bSize))
     {
         out.b.reserve(bSize);
         while (bSize-- > 0)
         {
             int b;
             if (!(is >> b))
                break;
             out.b.push_back(b);   
         }
     }
     return is;
}

Of course - in more complicated cases - consider to use XMl-like (json?) formats and some more specialized iterators than std::istream_iterator<T>.

---

For XML - like formats - you might consider such scheme (this is very hypothetical code - I do not have any such library in my mind):

SomeLib::File xmlData("input.xml");

class S1BasedTests : public TestWithParam<S1>
{};

TEST_P(S1BasedTests , shouldXxxx)
{
    const S1& s1 = GetParam();
    ...
}
auto s1Entities = file.filterBy<S1>("S1");
INSTANTIATE_TEST_CASE_P(S1,
                        S1BasedTests,
                        ValuesIn(s1Entities.begin(), s1Entities .end()));

// etc for any types S1 you want

---

If there is no such C++-type-friendly library on the market (I searched for 2 minutes and did not find) - then maybe something like this:

SomeLib::File xmlFile("input.xml");

struct S1BasedTests : public TestWithParam<SomeLib::Node*>
{
   struct S1 // xml=<S1 a="1" b="2"/>
   {
       int a;
       int b;
   };
   S1 readNode()
   {
        S1 s1{};
        s1.a = GetParam()->getNode("a").getValue<int>();
        s1.b = GetParam()->getNode("b").getValue<float>();
        return s1;
   }
};

TEST_P(S1BasedTests , shouldXxxx)
{
    const S1& s1 = readNode();
    ...
}
INSTANTIATE_TEST_CASE_P(S1,
                        S1BasedTests ,
                        ValuesIn(xmlFile.getNode("S1").getChildren()));
                        // xml=<S1s> <S1.../> <S1.../> </S1>

// etc for any node types like S1