I am using numpy and numpy.array2string to probe effects of small changes and numerical stability. I am baffled by the following exercise:
<Get vector data through prior code>
print(data.shape)
print(data.dtype)
variance = np.var(data)
print(variance.dtype)
print(variance)
print(np.array2string(variance, suppress_small=True,formatter={'float': '{: 8.55f}'.format}))
Which gives as output:
torch.Size([1, 64]) # shape of the data
torch.float32 # dtype of the data
float32 # dtype of the variance result
37166410.0
37166408.0000000000000000000000000000000000000000000000000000000
This makes me crosseyed: The array2string isn't just reformatting the value with a (patently absurd) number of digits, it seems to be changing the value itself well above the point where round off errors should matter. (It doesn't seem to be the formatter, as I can remove that and still see changed values.)
A float32 can easily represent either of those integer values directly.
What am I not understanding here? Please reaffirm my faith in objective reality.
UPDATE: No float32 cannot exactly represent 37166410.0, and I need reading glasses.