Casting double to ulong weirdness

Matthias Bentrup via Digitalmars-d digitalmars-d at puremagic.com
Tue Aug 25 07:35:53 PDT 2015


On Monday, 24 August 2015 at 16:52:54 UTC, Márcio Martins wrote:
> I'm posting this here for visibility. This was silently 
> corrupting our data, and might be doing the same for others as 
> well.
>
> import std.stdio;
> void main() {
>   double x = 1.2;
>   writeln(cast(ulong)(x * 10.0));
>   double y = 1.2 * 10.0;
>   writeln(cast(ulong)y);
> }
>
> Output:
> 11
> 12
>

Internally the first case calculates x * 10.0 in real precision 
and casts it to ulong in truncating mode directly. As 1.2 is not 
representable, x is really 1.199999999999999956 and the result is 
trunc(11.99999999999999956) = 11.

In the second case x * 10.0 is calculated in real precision, but 
first converted to double in round-to-nearest mode and then the 
result is truncated.




More information about the Digitalmars-d mailing list