cast(int) getting an unexpected number

[Lars T. Kyllingstad]

rmcguire wrote:
> "Lars T. Kyllingstad" <public at kyllingen.NOSPAMnet> wrote:
>  
>> Charles Hixson wrote:
>>> Lars T. Kyllingstad wrote:
>>>> Michal Minich wrote:
>>>>> Hello rmcguire,
>>>>>
>>>>>> why is this not a compiler bug?
>>>>>> because:
>>>>>> import std.stdio;
>>>>>> void main() {
>>>>>> float f=0.01;
>>>>>> writefln("%0.2f->%d",f,cast(int)(f*100f));
>>>>>> writefln("%0.2f->%d",f,cast(int)(.01*100f));
>>>>>> writefln("%0.2f->%f",f,(f*100f));
>>>>>> }
>>>>>> results in:
>>>>>> 0.01->0
>>>>>> 0.01->1
>>>>>> 0.01->1.000000
>>>>>> I would say something is dodgy.
>>>>>>
>>>>>> -Rory
>>>>>>
>>>>> I think this may be case of:
>>>>> At comple time floating point computations may be done at a higher
>>>>> precision than run time.
>>>>
>>>> Yes, if you do this:
>>>>
>>>> float f = 0.01;
>>>> float g = f * 100f;
>>>> real r = f * 100f;
>>>> writeln("%s, %s, %s", f, cast(int) g, cast(int) r);
>>>>
>>>> you get:
>>>>
>>>> 0.01, 0, 1
>>>>
>>>> I believe just writing cast(int)(f*100f) is more or less the same as the
>>>> 'real' case above.
>>>>
>>>> -Lars
>>> Can that *really* be the explanation??  I know that float doesn't have 
>>> all that much precision, but I thought it was more than 5 or 6 
>>> places...and this is, essentially, two places.
>>
>> Yes it does, but that's not what matters.
>>
>> Say that for floats, the representable number closest to 0.01 is 
>> 0.01000000000001. (This is just an example, I don't know the true 
>> number.) Then you have a lot more precision than the two digits you 
>> mention, and multiplying with 100 gives 1.000000000001. Round this 
>> towards zero (which is what cast(int) does) and you get 1. This is the 
>> 'float g = f*100f;' case in my example.
>>
>> Now, say that for reals, which (I think) is what the compiler uses 
>> internally, the number closest to 0.01 is
>>      0.0099999999999999999999999999999999999999.
>> Again, just an example, the point is that the precision is higher than 
>> the above, but the closest number is now smaller than 0.01. Multiply 
>> this by 100, and you get
>>      0.99999999999999999999999999999999999999.
>> This number will be cast to the integer 0, which happens in the OP's case.
>>
>> I admit I'm no expert in these things, but I suspect this is how it 
>> goes. By the way, I recommend Don's excellent article on floating-point 
>> numbers. It has really cleared things up for me:
>>
>>    http://www.digitalmars.com/d/2.0/d-floating-point.html
>>
>> -Lars
>>
> 
> Hmm, thanks.
> 
> pity its not consistent. Would be nice if you didn't have to learn everything 
> about floating point just to make a calculator.


You don't, really.  You just stay away from casts. :)  IMHO, casts are 
only good for hammering a value of one type into a different type, no 
worries about the result.

I (almost) never use cast(int) to round floating-point numbers.  There 
are two reasons for this:

   1. It always rounds towards zero, which is usually not what I want.
   2. It doesn't perform any checks, and can easily overflow.

If the rounding mode is not important, or if for some reason I want the 
same result as cast(int), I use std.conv.to().  This function checks 
that the converted number fits in the target type and then performs an 
ordinary cast.  (Note: D2 only!)

   import std.conv;

   void main()
   {
       int i = to!int(1.23);     // OK
       assert (i == 1);

       int j = to!int(real.max); // ConvOverflowError
   }

If "natural" rounding, i.e. towards nearest integer, is what I want, 
then I use std.math.lround() or std.math.round().  There are a lot of 
rounding functions in std.math that do different things and that are 
worth checking out.

   import std.math;

   void main()
   {
       long i = lround(1.23);
       assert (i == 1);

       long j = lround(0.5);
       assert (j == 1);
   }

-Lars