cast(int) getting an unexpected number
rmcguire
rjmcguire at gmail.com
Wed Nov 4 23:16:01 PST 2009
"Lars T. Kyllingstad" <public at kyllingen.NOSPAMnet> wrote:
> Charles Hixson wrote:
>> Lars T. Kyllingstad wrote:
>>> Michal Minich wrote:
>>>> Hello rmcguire,
>>>>
>>>>> why is this not a compiler bug?
>>>>> because:
>>>>> import std.stdio;
>>>>> void main() {
>>>>> float f=0.01;
>>>>> writefln("%0.2f->%d",f,cast(int)(f*100f));
>>>>> writefln("%0.2f->%d",f,cast(int)(.01*100f));
>>>>> writefln("%0.2f->%f",f,(f*100f));
>>>>> }
>>>>> results in:
>>>>> 0.01->0
>>>>> 0.01->1
>>>>> 0.01->1.000000
>>>>> I would say something is dodgy.
>>>>>
>>>>> -Rory
>>>>>
>>>>
>>>> I think this may be case of:
>>>> At comple time floating point computations may be done at a higher
>>>> precision than run time.
>>>
>>>
>>> Yes, if you do this:
>>>
>>> float f = 0.01;
>>> float g = f * 100f;
>>> real r = f * 100f;
>>> writeln("%s, %s, %s", f, cast(int) g, cast(int) r);
>>>
>>> you get:
>>>
>>> 0.01, 0, 1
>>>
>>> I believe just writing cast(int)(f*100f) is more or less the same as the
>>> 'real' case above.
>>>
>>> -Lars
>> Can that *really* be the explanation?? I know that float doesn't have
>> all that much precision, but I thought it was more than 5 or 6
>> places...and this is, essentially, two places.
>
>
> Yes it does, but that's not what matters.
>
> Say that for floats, the representable number closest to 0.01 is
> 0.01000000000001. (This is just an example, I don't know the true
> number.) Then you have a lot more precision than the two digits you
> mention, and multiplying with 100 gives 1.000000000001. Round this
> towards zero (which is what cast(int) does) and you get 1. This is the
> 'float g = f*100f;' case in my example.
>
> Now, say that for reals, which (I think) is what the compiler uses
> internally, the number closest to 0.01 is
> 0.0099999999999999999999999999999999999999.
> Again, just an example, the point is that the precision is higher than
> the above, but the closest number is now smaller than 0.01. Multiply
> this by 100, and you get
> 0.99999999999999999999999999999999999999.
> This number will be cast to the integer 0, which happens in the OP's case.
>
> I admit I'm no expert in these things, but I suspect this is how it
> goes. By the way, I recommend Don's excellent article on floating-point
> numbers. It has really cleared things up for me:
>
> http://www.digitalmars.com/d/2.0/d-floating-point.html
>
> -Lars
>
Hmm, thanks.
pity its not consistent. Would be nice if you didn't have to learn everything
about floating point just to make a calculator.
-Rory
More information about the Digitalmars-d-learn
mailing list