Casting double to ulong weirdness
Matthias Bentrup via Digitalmars-d
digitalmars-d at puremagic.com
Tue Aug 25 07:35:53 PDT 2015
On Monday, 24 August 2015 at 16:52:54 UTC, Márcio Martins wrote:
> I'm posting this here for visibility. This was silently
> corrupting our data, and might be doing the same for others as
> well.
>
> import std.stdio;
> void main() {
> double x = 1.2;
> writeln(cast(ulong)(x * 10.0));
> double y = 1.2 * 10.0;
> writeln(cast(ulong)y);
> }
>
> Output:
> 11
> 12
>
Internally the first case calculates x * 10.0 in real precision
and casts it to ulong in truncating mode directly. As 1.2 is not
representable, x is really 1.199999999999999956 and the result is
trunc(11.99999999999999956) = 11.
In the second case x * 10.0 is calculated in real precision, but
first converted to double in round-to-nearest mode and then the
result is truncated.
More information about the Digitalmars-d
mailing list