Let N(x) be the value of the decimal numeral with the fewest significant digits
such that x is the double
value nearest the value of the numeral.
Given double
values a and b, how can we compute the double
value nearest N(b)-N(a)?
E.g.:
- If a and b are the
double
values nearest .2 and .3,
- the desired result is the
double
value nearest .1,
- 0.1000000000000000055511151231257827021181583404541015625,
- rather than than the result of directly subtracting a and b,
- 0.09999999999999997779553950749686919152736663818359375.
As a baseline: In Java, the Double.toString()
provides the N(x) function described in the question, returning its value as a numeral. One could take the strings for a and b, subtract them with the elementary-school method, and convert the resulting string to double
.
This demonstrates solving the problem is quite feasible using existing library routines. This leaves the task of improving the solution. I suggest exploring:
- Is there a function D(x) that returns the number of significant digits after the decimal place for the numeral described in N(x)? If so, can we multiply a and b by a power of ten determined by D(a) and D(b), round as necessary to produce the correct integer results (for situations where they are representable as
double
values), subtract them, and divide by the power of ten?
- Can we establish criteria for which
b-a
or some simple expression can be quickly rounded to something near a decimal numeral, bypassing the code that would be necessary for harder cases? E.g., could we prove that for numbers within a certain range, (round(10000*b)-round(10000*a))/10000
always produces the desired result?
You can convert to 'integers' by multiplying then dividing by a power of ten:
(10*.3 - 10*.2)/10 == 0.1000000000000000055511151231257827021181583404541015625
It may be possible to work out the appropriate power of ten from the string representation of the number. @PatriciaShanahan suggests looking for repeated 0's or 9's.
Consider using a BigDecimal library such as javascript-bignum instead.
You could also inquire in Smalltalk Pharo 2.0 where your request translates:
^(b asMinimalDecimalFraction - a asMinimalDecimalFraction) asFloat
Code could be found as attachment to issue 4957 at code.google.com/p/pharo/issues - alas, dead link, and the new bugtracker requires a login...
https://pharo.fogbugz.com/f/cases/5000/Let-asScaledDecimal-use-the-right-number-of-decimals
source code is also on github, currently:
https://github.com/pharo-project/pharo-core/blob/6.0/Kernel.package/Float.class/instance/printing/asMinimalDecimalFraction.st
The algorithm is based on:
Robert G. Burger and R. Kent Dybvig
Printing Floating Point Numbers Quickly and Accurately
ACM SIGPLAN 1996 Conference on Programming Language Design and Implementation
June 1996.
http://www.cs.indiana.edu/~dyb/pubs/FP-Printing-PLDI96.pdf