Another take on decimal data types
rumbu
rumbu at rumbu.ro
Fri Jan 12 04:51:38 UTC 2018
On Thursday, 11 January 2018 at 23:57:29 UTC, kdevel wrote:
What about the failed comparison:
>
> gt.d
> ```
> import std.stdio;
> import decimal;
>
> void loopme(T) ()
> {
> "---".writeln;
> T e = 10;
> while (e > 1e-6) {
> e /= 10;
> writeln (e, ' ', e > 1e-6);
> }
> }
>
> void main ()
> {
> loopme!decimal32;
> loopme!decimal64;
> loopme!decimal128;
> }
> ```
>
> This gives here:
>
> ---
> 1 true
> 0.1 false
> ---
> 1 true
> 0.1 false
> ---
> 1 true
> 0.1 true
> 0.0100000 true
> 0.00100000 true
> 0.000100000 true
> 1.00000e-05 true
> 1.00000e-06 true
> 1.00000e-07 false
This is not failed comparison. 1e-6 cannnot be represented
exactly as binary floating point, it is in fact
0.00000099999999999999995481 as double which happens to be less
than 1e-6.
decimal32 and decimal64 are rounding up the value, decimal128 has
enough precision (34 digits) to render exactly the value above.
writefln(" 7 digits: %#.7f", 1e-6);
writefln("15 digits: %#.15f", 1e-6);
writefln("34 digits: %#.34f", 1e-6);
-----
7 digits: 0.0000010
15 digits: 0.000001000000000
34 digits: 0.0000009999999999999999548100000000
More information about the Digitalmars-d-announce
mailing list