How does one record test history with TestNg?

问题:

I have a set of java based api tests which I run from Intellij IDEA. Some of the tests generally fail due to the same reason in each test run. Some tests fail less often. I want to run all the tests at least a hundred times to find out which tests failed and what are the unique reasons for the failure of each failed test.

Here is an example of the report I want to make for myself. There is one row for each failing test. The text before each colon is actually the column name.

failingTest: myTest, failureReason: Expected 200 but got 304, timesOccurred: 25/100, stackTrace: text.

failingTest: myTest, failureReason: 404 not found, timesOccurred: 5/100, stackTrace: text.

So, from the above we see that "myTest" failed a total of 30/100 times. What is the easiest way to generate such a report ?

Thanks.

回答1:

The easiest way of doing this would be for you to build a custom reporter by implementing org.testng.IReporter which is backed by a database, within which you add the test results.

If the test result for a particular method already exists, then update its execution count and also its pass/fail count based on the test results for every unique failure reason.