double trouble

xs0 xs0 at xs0.com
Wed Mar 1 05:09:57 PST 2006


Lionello Lunesu wrote:
> I would get rid of the while, or at least the /= and *= therein, these will 
> only add to the inaccuracy. Better keep track of integers (++n, --n) and use 
> pow when you need the 10^n or so. Remember that the 10.0 is already 
> inaccurate (imagine it's 9.999 or so), using it multiple times will 
> definately add to the inaccuracy.
> 
> L. 
> 
> 

Actually, 10.0 is accurately representable, but 0.1 isn't..
Perhaps the most accurate way to calculate decimal digits would be

digit/pow(10.0, -exp)

instead of digit*pow(0.1, exp), avoiding exponentiating an inaccurate 
number..


Well, I went ahead and tested it (in Java, but the results should match 
D's double). Using digit*constant (0.000...0001) and digit/pow(10, exp) 
produce about the same results, while repeatedly multiplying with 0.1 
and exponentiation of 0.1 also produce about the same results, except 
the latter two are slightly more off.

http://www.xs0.com/prec/


xs0



More information about the Digitalmars-d mailing list