Casting double to ulong weirdness

[Steven Schveighoffer via Digitalmars-d]

On 8/24/15 4:34 PM, "=?UTF-8?B?Ik3DoXJjaW8=?= Martins\" 
<marcioapm at gmail.com>\"" wrote:
> On Monday, 24 August 2015 at 19:23:44 UTC, Steven Schveighoffer wrote:
>> On 8/24/15 1:43 PM, bachmeier wrote:
>>> On Monday, 24 August 2015 at 16:52:54 UTC, Márcio Martins wrote:
>>>> I'm posting this here for visibility. This was silently corrupting our
>>>> data, and might be doing the same for others as well.
>>>>
>>>> import std.stdio;
>>>> void main() {
>>>>   double x = 1.2;
>>>>   writeln(cast(ulong)(x * 10.0));
>>>>   double y = 1.2 * 10.0;
>>>>   writeln(cast(ulong)y);
>>>> }
>>>>
>>>> Output:
>>>> 11
>>>> 12
>>>>
>>>>
>>>> to!ulong instead of the cast does the right thing, and is a viable
>>>> work-around.
>>>>
>>>> Issue: https://issues.dlang.org/show_bug.cgi?id=14958)
>>>
>>> I would not describe to!ulong as a "work-around". You just discovered
>>> one of the reasons to! exists: it is the right way to do it and
>>> cast(ulong) is the wrong way. As the others have noted, floating point
>>> is tricky business, and you need to use the right tools for the job.
>>
>> real y = x * 10.0;
>> writeln(y.to!ulong); // 11
>>
>> to! does not do anything different than cast. What is happening here
>> is the implicit cast from real to double. D treats the result of x *
>> 10.0 as type double, but it's done at real precision. In that
>> conversion, the error is hidden by a rounding automatically done by
>> the processor I think.
>
> Whatever the issue is, it is not unavoidable, because as has been shown,
> other languages do it correctly.

Your other examples use doubles, not reals. It's not apples to apples.

>  From the data presented so far, it seems like the issue is that the mul
> is performed in 80-bit precision, storing it before the cast forces a
> truncation down to 64-bit.

Not just truncation, rounding too.

> Similarly, passing it to a function will also
> truncate to 64-bit, due to ABIs. This is why to! works as expected.
>
> Please do keep in mind that the issue is not one of precision, but one
> of inconsistency.

It is an issue of precision. In order to change from real to double, 
some bits must be lost. Since certain numbers cannot be represented, the 
CPU must round or truncate.

> They are not the same thing. The result being 11 or 12
> is irrelevant to this issue. It should just be the same for two
> instances of the same expression.

They are not the same expression. One goes from double through 
multiplication to real, then back to double, then to ulong. The other 
skips the real to double conversion and goes directly to ulong.

The real issue here is that you are not correctly converting from a 
floating point number to an integer.

> In an attempt to make things more obvious, consider this example, which
> also illustrates why to! works, despite apparently doing nothing extra
> at all.
>
> double noop(double z) {
>    return z;
> }
>
> void main() {
>    double x = 1.2;
>    writeln(cast(ulong)(x * 10.0));
>    writeln(cast(ulong)noop(x * 10.0));
> }
>
> Outputs:
> 11
> 12

I understand the inconsistency, and I agree it is an issue that should 
be examined. But the issue is entirely avoidable by not using incorrect 
methods to convert from floating point to integer after floating point 
operations introduce some small level of error.

Perhaps there is some way to make it properly round in this case, but I 
guarantee it will not fix all floating point errors.

-Steve