Another take on decimal data types

rumbu rumbu at rumbu.ro
Fri Jan 12 04:51:38 UTC 2018


On Thursday, 11 January 2018 at 23:57:29 UTC, kdevel wrote:
What about the failed comparison:
>
> gt.d
> ```
> import std.stdio;
> import decimal;
>
> void loopme(T) ()
> {
>    "---".writeln;
>    T e = 10;
>    while (e > 1e-6) {
>       e /= 10;
>       writeln (e, ' ', e > 1e-6);
>    }
> }
>
> void main ()
> {
>    loopme!decimal32;
>    loopme!decimal64;
>    loopme!decimal128;
> }
> ```
>
> This gives here:
>
>    ---
>    1 true
>    0.1 false
>    ---
>    1 true
>    0.1 false
>    ---
>    1 true
>    0.1 true
>    0.0100000 true
>    0.00100000 true
>    0.000100000 true
>    1.00000e-05 true
>    1.00000e-06 true
>    1.00000e-07 false

This is not failed comparison. 1e-6 cannnot be represented 
exactly as binary floating point, it is in fact 
0.00000099999999999999995481 as double which happens to be less 
than 1e-6.

decimal32 and decimal64 are rounding up the value, decimal128 has 
enough precision (34 digits) to render exactly the value above.

writefln(" 7 digits: %#.7f", 1e-6);
writefln("15 digits: %#.15f", 1e-6);
writefln("34 digits: %#.34f", 1e-6);

-----

  7 digits: 0.0000010
15 digits: 0.000001000000000
34 digits: 0.0000009999999999999999548100000000







More information about the Digitalmars-d-announce mailing list