cast(int) getting an unexpected number
rmcguire
rjmcguire at gmail.com
Wed Nov 4 08:14:55 PST 2009
"Steven Schveighoffer" <schveiguy at yahoo.com> wrote:
> On Wed, 04 Nov 2009 06:14:36 -0500, Joel Christensen <joelcnz at gmail.com>
> wrote:
>
>> Input:
>> import std.stdio;
>> void main() {
>> float f=0.01;
>> writefln("%0.2f->%d",f,cast(int)(f*100f));
>> }
>>
>> Output:
>> 0.01->0
>>
>> Comment:
>> What!?
>
> hehe. .01 is not exactly represented by float, because it's stored as a
> binary value, not a decimal value. Think about how there is no way to
> represent 1/3 in decimal.
>
> If you imagined that the computer stored things in decimal, see how the
> 1/3 example would work
>
> a = 1.0/3; // truncated to 0.3333333
> a *= 3; // 0.9999999
> auto i = cast(int)a; // 0
>
> This is analagous to what you are asking the compiler to do.
>
> To be safe, whenever converting to int, always add a small epsilon. I
> think you can use float.epsilon, but I don't have any experience with
> whether that is always reasonable.
>
> -Steve
>
why is this not a compiler bug?
because:
import std.stdio;
void main() {
float f=0.01;
writefln("%0.2f->%d",f,cast(int)(f*100f));
writefln("%0.2f->%d",f,cast(int)(.01*100f));
writefln("%0.2f->%f",f,(f*100f));
}
results in:
0.01->0
0.01->1
0.01->1.000000
I would say something is dodgy.
-Rory
More information about the Digitalmars-d-learn
mailing list