Generate normal random numbers in float32/16 in-pl

问题:

In Numpy/Scipy, how can I generate random numbers from a normal distribution with a specified (float) dtype? In my case I need float32 and float16.

Since the array is quite large, I don't want to convert the array after the sampling.

For example:

a = np.random.normal(1e7).astype('float16')

does the job but since it need a temporary float64 array it uses 4x the RAM than a direct float16 sampling.

回答1:

I don't know of a random number generator in numpy or scipy that generates 16 or 32 bit floats natively.

To avoid the large temporary, you could generate the values in batches. For example, the following creates an array of 10000000 samples of float16 values.

In [125]: n = 10000000  # Number of samples to generate

In [126]: k = 10000     # Batch size

In [127]: a = np.empty(n, dtype=np.float16)

In [128]: for i in range(0, n, k):
   .....:     a[i:i+k] = np.random.normal(loc=0, scale=1, size=k)
   .....: