cast(int) getting an unexpected number
Steven Schveighoffer
schveiguy at yahoo.com
Wed Nov 4 03:30:47 PST 2009
On Wed, 04 Nov 2009 06:14:36 -0500, Joel Christensen <joelcnz at gmail.com>
wrote:
> Input:
> import std.stdio;
> void main() {
> float f=0.01;
> writefln("%0.2f->%d",f,cast(int)(f*100f));
> }
>
> Output:
> 0.01->0
>
> Comment:
> What!?
hehe. .01 is not exactly represented by float, because it's stored as a
binary value, not a decimal value. Think about how there is no way to
represent 1/3 in decimal.
If you imagined that the computer stored things in decimal, see how the
1/3 example would work
a = 1.0/3; // truncated to 0.3333333
a *= 3; // 0.9999999
auto i = cast(int)a; // 0
This is analagous to what you are asking the compiler to do.
To be safe, whenever converting to int, always add a small epsilon. I
think you can use float.epsilon, but I don't have any experience with
whether that is always reasonable.
-Steve
More information about the Digitalmars-d-learn
mailing list