Casting double to ulong weirdness

Steven Schveighoffer via Digitalmars-d digitalmars-d at puremagic.com
Mon Aug 24 11:59:58 PDT 2015


On 8/24/15 2:38 PM, Steven Schveighoffer wrote:
> On 8/24/15 2:06 PM, rumbu wrote:
>> BTW, 1.2 and 12.0 are directly representable as double
>>
>> In C++:
>>
>> printf("%.20f\r\n", 1.2);
>> printf("%.20f\r\n", 12.0);
>>
>> will output:
>>
>> 1.20000000000000000000
>> 12.00000000000000000000
>>
>> Either upcasting to real is the wrong decision here, either the writeln
>> string conversion is wrong.
>
> I don't think they are directly representable as floating point, because
> they are have factors other than 2 in the decimal portion. From my
> understanding, anything that only has to do with powers of 2 are
> representable in floating point, just like you cannot represent 1/3 in
> decimal exactly.
>
> But there is definitely something weird going on with the casting.
>
> I wrote this program:
>
> testfp.d:
> extern(C) void foo(double x);
> void main() {
>      double x = 1.2;
>      foo(x);
> }
>
> testfp2.d:
> extern(C) void foo(double x)
> {
>      import std.stdio;
>      writeln(cast(ulong)(x * 10.0));
> }
>
> testfp2.c:
> #include <stdio.h>
>
> void foo(double x)
> {
>      printf("%lld\n", (unsigned long long)(x * 10));
> }
>
>
> If I link testfp.d against testfp2.c, then it outputs 12. If I link
> against testfp2.d, it outputs 11.
>

More data:

It definitely has something to do with the representation of 1.2 * 10.0 
in *real*.

I changed the code so that it writes the result of the multiplication to 
a shared double.

In this case it *works* and prints 12, just like C does.

This also works:

double x = 1.2;
double y = x * 10.0;
writeln(cast(ulong)y); // 12

However, change y to a real, and you get 11.

Note that if I first convert from real to double, then convert to ulong, 
it works.

This code:

     double x = 1.2;
     double x2 = x * 10.0;
     real y = x * 10.0;
     real y2 = x2;
     double y3 = y;
     writefln("%a, %a, %a", y, y2, cast(real)y3);


outputs:

0xb.ffffffffffffep+0, 0xcp+0, 0xcp+0

So some rounding happens in the conversion from real to double, that 
doesn't happen in the conversion from real to ulong.

All this gets down to: FP cannot accurately represent decimal. Should 
this be fixed? Can it be fixed? I don't know. But I would be very 
cautious about converting anything FP to integers without some epsilon.

-Steve


More information about the Digitalmars-d mailing list