Unit Testing with functions that return random res

Question

I don't think that this is specific to a language or framework, but I am using xUnit.net and C#.

I have a function that returns a random date in a certain range. I pass in a date, and the returning date is always in range of 1 to 40 years before the given date.

Now I just wonder if there is a good way to unit test this. The best approach seems to be to create a loop and let the function run i.e. 100 times and assert that every of these 100 results are in the desired range, which is my current approach.

I also realize that unless I am able to control my Random generator, there will not be a perfect solution (after all, the result IS random), but I wonder what approaches you take when you have to test functionality that returns a random result in a certain range?

Answer 1

Normaly I use exactly your suggested approach: Control the Random generator. Initialize it for test with a default seed (or replace him by a proxy returning numbers which fit my testcases), so I have deterministic/testable behaviour.

Answer 2

I would recommend overriding the random function. I am unit testing in PHP so I write this code:

// If we are unit testing, then...
if (defined('UNIT_TESTING') && UNIT_TESTING)
{
   // ...make our my_rand() function deterministic to aid testing.
   function my_rand($min, $max)
   {
      return $GLOBALS['random_table'][$min][$max];
   }
}
else
{
   // ...else make our my_rand() function truly random.
   function my_rand($min = 0, $max = PHP_INT_MAX)
   {
      if ($max === PHP_INT_MAX)
      {
         $max = getrandmax();
      }
      return rand($min, $max);
   }
}

I then set the random_table as I require it per test.

Testing the true randomness of a random function is a separate test altogether. I would avoid testing the randomness in unit tests and would instead do separate tests and google the true randomness of the random function in the programming language you are using. Non-deterministic tests (if any at all) should be left out of unit tests. Maybe have a separate suite for those tests, that requires human input or much longer running times to minimise the chances of a fail that is really a pass.

Answer 3

Mock or fake out the random number generator

Do something like this... I didn't compile it so there might be a few syntax errors.

public interface IRandomGenerator
{
    double Generate(double max);
}

public class SomethingThatUsesRandom
{
    private readonly IRandomGenerator _generator;

    private class DefaultRandom : IRandomGenerator
    {
        public double Generate(double max)
        {
            return (new Random()).Next(max);
        }
    }

    public SomethingThatUsesRandom(IRandomGenerator generator)
    {
        _generator = generator;
    }

    public SomethingThatUsesRandom() : this(new DefaultRandom())
    {}

    public double MethodThatUsesRandom()
    {
        return _generator.Generate(40.0);
    }
}

In your test, just fake or mock out the IRandomGenerator to return something canned.

Answer 4

If you want to check the quality of the random numbers (in terms of independance) there are several ways to do this. One good way is the Chi square test.

Answer 5

I don't think Unit testing is meant for this. You can use Unit testing for functions that return a stochastic value, but use a fixed seed, in which case in a way they are not stochastic, so to speak, for random seed, I dont think Unit testing is what you want, for example for RNGs what you mean to have is a system test, in which you run the RNG many many times and look at the distribution or moments of it.

Answer 6

Depending on how your function creates the random date, you may also want to check for illegal dates: impossible leap years, or the 31st day of a 30-day month.