float comparison gives wrong result in loop?

Steve Horne stephenwantshornenospam100 at aol.com
Wed Sep 20 22:05:39 PDT 2006


On Tue, 19 Sep 2006 01:42:51 -0700, Bradley Smith
<digitalmars-com at baysmith.com> wrote:

>With the following code, why does the assert fail?
>
>import std.stdio;
>
>void main() {
>	const float STEP_SIZE = 0.2f;
>	
>	float j = 0.0f;
>	while (j <= ( 1.0f / STEP_SIZE)) {
>		j += 1.0f;
>		writefln(j <= ( 1.0f / STEP_SIZE));
>	}
>	assert(!(j <= ( 1.0f / STEP_SIZE)));
>}
>
>The loop exits early, but it should not. The result of
>"j <= ( 1.0f / STEP_SIZE)" is somehow different in the while statement 
>versus in the call to writefln.
>
>Interestingly, if the code is optimized, the assert passes.

There's a good chance that this is just an approximate arithmetic
issue.

1.0f / 0.2f is approximately 5.0f. It's approximate because this is
floating point arithmetic, and because 0.2 in particular cannot be
represented exactly in binary floating point.

The adding of 1.0f should be exact, since all whole numbers (up to
some rather large limit) can be represented precisely as binary
floats. So the incrementing will reach precisely 5.0.

Therefore, the comparison for <= may fail when writefln showns 5.0
(because it tends to round off for printing). You are comparing
exactly 5.0 with approximately 5.0, and the result depends on whether
'approximately 5.0' is slightly higher or lower.

All floating point arithmetic must be considered approximate. That's a
feature of floating point, irrespective of what programming language
you use. All comparisons of floating point results need to take this
into account. How you deal with it depends on what you are doing, but
a common approach is simply to allow the value to go over slightly.

For instance, start with j a very little bit below zero...

	float j = -0.00001f;

When the comparison is made against approximately 5.0, j will be
slightly lower than zero so the <= should still return true even if
'approximately 5.0' actually means 4.99999999999999.

Oh, and D does all floating point work at 80 bit precision, but saving
a result to a float variable cuts it down to 32 bit precision.
Optimising will affect how often this is done, and can cause slightly
different values to be seen at different points in the code.

-- 
Remove 'wants' and 'nospam' from e-mail.



More information about the Digitalmars-d-learn mailing list