How to test random numbers?

Question

i have written in matlab, a program, which is supossed to generate random numbers between 0 and 1. i have test it only with the runstest in matlab, and te result is that the sequence is random. i have seen the histograms too, and they have a beta distribution. i want to test this rng whith other test, such as diehard, ent, or nist, but i don't know how. can someone explain how to use them, or suggest me some other randomness tests. thank you

Answer 1

@Anna I had the same question as you and have now discovered Diehard thanks to some of the other answers.

The situation with my RNG is that it creates 1's and 0's and stores them in an ASCII file. When trying to upload this file to online randomness tests, it failed - most probably because the data needs to be in binary format.

And that's indeed the case with Diehard. If you install Diehard, you will find a file called DIEHARD.DOC which talks you through the steps of how to convert your ASCII file into the required binary files (along with some other changes you may need to make to your program).

These are my first steps, anyway. Hope this helps someone.

Answer 2

The tests available are:

Dieharder - http://www.phy.duke.edu/~rgb/General/dieharder.php

TestU01 - http://simul.iro.umontreal.ca/testu01/tu01.html

RaBiGeTe - http://cristianopi.altervista.org/RaBiGeTe_MT/

NIST STS - http://csrc.nist.gov/groups/ST/toolkit/rng/documentation_software.html

PractRand - http://pracrand.sourceforge.net/

Any of those can test bits from a file. Some (PractRand, Dieharder, not sure about TestU01) can test data piped in standard input. Some also support linking your PRNG directly to the test suite, dynamically (only RaBiGeTe offers real support for dynamically linking your PRNG to it) or statically.

Quality is not equal. If you have plenty of bits of PRNG output, PractRand can find the widest variety of biases quickest (full diclosure: I wrote PractRand), followed by TestU01. If you don't have plenty of bits, RaBiGeTe might do better. NIST STS and Dieharder generally underperform.

Convenience of interface is also not equal. PractRand and Dieharder are set up for command line automation. PractRand and TestU01 tend to have the easiest output to interpret in my opinion. Dieharder isn't bad in that regard. RaBiGeTe and NIST STS, well... they both promote what seems to me like overcomplicated & useless visualizations of distributions of test results.

Also, NIST STS and Dieharder both have false positive issues.

There's also ENT, can't find a link for it at the moment... it has a fairly convenient interface IIRC but is not very good at finding bias.

Answer 3

With most tests you can supply a large file of random numbers (integer or floating point) and run various tests on that sample file. DIEHARD worked that way, if I remember correctly and some others do, too. If you really want to see your generator fail, you could try using TestU01 by Pierre L'Ecuyer which has enough tests in it to let nearly every generator fail at least one test :-)

Still, for most test suites there is extensive documentation, at least I know this for DIEHARD, the test suite from NIST SP 800-22 as well as DieHarder and TestU01 (links go to the docs). The methods for supplying random numbers to test are usually different but mentioned in the respective documentation.