cast(int) getting an unexpected number

rmcguire rjmcguire at gmail.com
Wed Nov 4 08:14:55 PST 2009


"Steven Schveighoffer" <schveiguy at yahoo.com> wrote:
 
> On Wed, 04 Nov 2009 06:14:36 -0500, Joel Christensen <joelcnz at gmail.com>  
> wrote:
> 
>> Input:
>> import std.stdio;
>> void main() {
>>  float f=0.01;
>>  writefln("%0.2f->%d",f,cast(int)(f*100f));
>> }
>>
>> Output:
>> 0.01->0
>>
>> Comment:
>> What!?
> 
> hehe.  .01 is not exactly represented by float, because it's stored as a  
> binary value, not a decimal value.  Think about how there is no way to  
> represent 1/3 in decimal.
> 
> If you imagined that the computer stored things in decimal, see how the  
> 1/3 example would work
> 
> a = 1.0/3; // truncated to 0.3333333
> a *= 3; // 0.9999999
> auto i = cast(int)a; // 0
> 
> This is analagous to what you are asking the compiler to do.
> 
> To be safe, whenever converting to int, always add a small epsilon.  I  
> think you can use float.epsilon, but I don't have any experience with  
> whether that is always reasonable.
> 
> -Steve
> 

why is this not a compiler bug?
because:
import std.stdio;

void main() {
        float f=0.01;
        writefln("%0.2f->%d",f,cast(int)(f*100f));
        writefln("%0.2f->%d",f,cast(int)(.01*100f));
        writefln("%0.2f->%f",f,(f*100f));
}

results in:
0.01->0
0.01->1
0.01->1.000000

I would say something is dodgy.

-Rory



More information about the Digitalmars-d-learn mailing list