I have a pseudo random number generator (PRNG) class that I want to unit test. There are two approaches:
- write a test case that takes a large amount of samples and test whether they are properly distributed. This approach may lead to a fairly long execution time for the test case;
- calculate a small series of samples 'by hand' and verify if the PRNG algorithm reproduces it. This approach may lead to a not random sequence being generated without being noticed;
I would say that the first approach is not really unit testing because it does not perform a white box test of the generator, but on the other hand it properly tests the responsibility of the class. The second approach is more like a real unit test, focusing on the algorithm, but it does not provide as much evidence as to whether the class fulfills its responsibility.
Which approach do you prefer, and why?
Here's a CodeProject article that includes an implementation of the Kolmogorov-Smirnov test mentioned in Donald Knuth's volume 2, Seminumerical Algorithms. As InSciTek Jeff mentioned above, there are two issues: testing the algorithm and testing your implementation of the algorithm. The K-S test is likely to find bugs in the implementation, and it's a good start toward testing the quality of the algorithm itself.