What is the approximate resolution of a single pre

Question

I am storing many longitudes and latitudes as doubles, I am wondering if I can get away with storing them as floats.

To answer this question, I need to know the approximate resolution of a single precision floating point number when the stored values are longitudes / latitudes (-180 to +180).

Answer 1

The resolution you can count on with single-precision floats is about 360 / (2 ^ 23) or 4 * 10 ^ -5.

More precisely, the largest single-precision float strictly inferior to 360. (which is representable exactly) is about 359.999969. For the whole range -360. .. 360, you will be able to represent differences at least as small as the difference between these two numbers.

Answer 2

Depends, but rather not.

32-bit float stores 7 significant digits. That is normally too little for storing the proper resolution of longitude/latitude. For example, openstreetmap.org uses six digits after the comma, so minimum eight, maximum total of ten digits.

In short, use float64.

Answer 3

Your question may have several interpretations.

If it is just for angles and for storage on a disk or on a device i would suggest you to store your values using a totally different technique: store as 32 bit integer.

int encodedAngle = (int)(value * (0x7FFFFFFF / 180.0));

To recover it, do the contrary.

double angle = (encodedAngle / (0x7FFFFFFF / 180.0));

In this way you have full 31 bit resolution for 180 degrees and 1 bit for the sign.

You can use this way also to keep your values in ram, the cost of this coversion is higher compared to work directly with doubles, but if you want to keep your memory low but resolution high this can work quite well. The cost is not so high, just a conversion to/from integer from/to double and a multiplication, modern processors will do it in a very little amount of time, and since the accessed memory is less, if the list contains a lot of values, your code will be more friendly with processor cache.

Your resolution will be 180 / ((2^31) - 1) = 8.38190318 × 10^-8 degrees, not bad :)

Answer 4

Usually floats are 4 bytes (32 bits) while doubles are double that. However, the exact precision if you're doing calculations is implementation (and hardware) specific. On some systems all floats will be stored as doubles, just to add to the confusion.