Always false float comparisons

Marco Leise via Digitalmars-d digitalmars-d at puremagic.com
Thu May 12 16:06:13 PDT 2016


Am Mon, 9 May 2016 04:26:55 -0700
schrieb Walter Bright <newshound2 at digitalmars.com>:

> > I wonder what's the difference between 1.30f and cast(float)1.30.  
> 
> There isn't one.
 
Oh yes, there is! Don't you love floating-point...

cast(float)1.30 rounds twice, first from a base-10
representation to a base-2 double value and then again to a
float. 1.30f directly converts to float. In some cases this
does not yield the same value as converting the base-10
literal directly to a float!

Imagine this example (with mantissa bit count reduced for
illustration):

Original base-10 rational number converted to base-2:
111110|10000000|011010111011...
       ↖        ↖
        float &  double mantissa precision limits

The 1st segment is the mantissa width of a float.
The 2nd segment is the mantissa width of a double.
The 3rd segment is the fraction used for rounding
 the base-10 literal to a double.

Conversion to double rounds down, since fraction <0.5:
111110|10000000 (double mantissa)

Proposed cast to float rounds down, too, because of
round-to-even rule for 0.5:
111110 (float mantissa)

But the conversion of base-10 directly to float, rounds UP
since the fraction is then >0.5
111110|10000000_011010111011...
111111

It happens when the 29 bits difference between double and
float mantissa are 100…000 and the bit in front and after are
0. The error would occur to ~0.000000047% of numbers.

-- 
Marco



More information about the Digitalmars-d mailing list