Always false float comparisons

Ola Fosheim Grøstad via Digitalmars-d digitalmars-d at puremagic.com
Wed May 18 02:21:30 PDT 2016


On Wednesday, 18 May 2016 at 07:21:30 UTC, Joakim wrote:
> On Wednesday, 18 May 2016 at 05:49:16 UTC, Ola Fosheim Grøstad 
> wrote:
>> On Wednesday, 18 May 2016 at 03:01:14 UTC, Joakim wrote:
>>> There is nothing "random" about increasing precision till the 
>>> end, it follows a well-defined rule.
>>
>> Can you please quote that well-defined rule?
>
> It appears to be "the compiler carries everything internally to 
> 80 bit precision, even if they are typed as some other 
> precision."
> http://forum.dlang.org/post/nh59nt$1097$1@digitalmars.com

"The compiler" means: implementation defined. That is the same as 
not being well-defined. :-)

>> It is indeed random, or arbitrary (which is the same thing):
>
> No, they're not the same thing: rules can be arbitrarily set 
> yet consistent over time, whereas random usually means both 
> arbitrary and inconsistent over time.

In this case it is the same thing then. I have no guarantee that 
my unit tests and production code will behave the same.

> I believe that means any calculation used to compute y at 
> compile-time will be done in 80-bit or larger reals, then 
> rounded to a const float for run-time, so your code comments 
> would be wrong.

No. The "const float y" will not be coerced to 32 bit, but the 
"float y" will be coerced to 32 bit. So you get two different y 
values. (On a specific compiler, i.e. DMD.)

> I don't understand why you're using const for one block and not 
> the other, seems like a contrived example.  If the precision of 
> such constants matters so much, I'd be careful to use the same 
> const float everywhere.

Now, that is a contrived defense for brittle language semantics! 
:-)

> If matching such small deltas matters so much, I wouldn't be 
> using floating-point in the first place.

Why not? The hardware gives the same delta. It only goes wrong if 
the compiler decides to "improve".

>> It depends on the unit tests running with the exact same 
>> precision as the production code.
>
> What makes you think they don't?

Because the language says that I cannot rely on it and the 
compiler implementation proves that to be correct.

>> Fast floating point code depends on the specifics of the 
>> hardware. A system level language should not introduce a 
>> different kind of bias that isn't present in the hardware!
>
> He's doing this to take advantage of the hardware, not the 
> opposite!

I don't understand what you mean.  He is not taking advantage of 
the hardware?


>> D is doing it wrong because it makes it is thereby forcing 
>> programmers to use algorithms that are 10-100x slower to get 
>> reliable results.
>>
>> That is _wrong_.
>
> If programmers want to run their code 10-100x slower to get 
> reliably inaccurate results, that is their problem.

Huh?

What I said is that D is doing it wrong because the 
"improvements" is forcing me to write code that is 10-100x slower 
to get the same level of reliability and required accuracy as I 
would get without the "improvements".


> If you're so convinced it's exact for a few cases, then check 
> exact equality there.  For most calculation, you should be 
> using approxEqual.

I am sorry, but this is not a normative rule at all. The rule is 
that you check for the bounds required. If it is exact, it just 
means the bounds are the same value (e.g. tight).

It does not help to say that people should use "approxEqual", 
because it does not improve on correctness. Saying such things 
just means that non-expert programmers assume that guessing the 
bounds will be sufficient. Well, it isn't sufficient.


> Since the real error bound is always larger than that, almost 
> any error bound you pick will tend to be closer to the real 
> error bound, or at least usually bigger and therefore more 
> realistic, than checking for exact equality.

I disagree. It is much better to get extremely wrong results 
frequently and therefore detect the error in testing.

What you are saying is that is better to get extremely wrong 
results infrequently which usually leads to error passing testing 
and enter production.

In order to test well you also need to understand for input makes 
the algorithm unstable/fragile.


> You can still test with approxEqual, so I don't understand why 
> you think that's not testing.

It is not testing anything if the compiler can change the 
semantics when you use a different context.


> The computer doesn't know that, so it will just plug that x in 
> and keep cranking, till you get nonsense data out the end, if 
> you don't tell it to check that x isn't too close to 2 and not 
> just 2.

Huh? I am not getting nonsense data. I am getting what I am 
asking for, I only want to avoid dividing by zero because it will 
make the given hardware 100x slower than the test.


> You have a wrong mental model that the math formulas are the 
> "real world," and that the computer is mucking it up.

Nothing wrong with my mental model. My mental model is the 
hardware specification + the specifics of the programming 
platform. That is the _only_ model that matters.

What D prevents me from getting is the specifics of the 
programming platform by making the specifics hidden.


> The truth is that the computer, with its finite maximums and 
> bounded precision, better models _the measurements we make to 
> estimate the real world_ than any math ever written.

I am not estimating anything. I am synthesising artificial 
worlds. My code is the model, the world is my code running at 
specific hardware.

It is self contained. I don't want the compiler to change my 
model because that will generate the wrong world. ;-)


>>> Oh, it's real world alright, you should be avoiding more than 
>>> just 2 in your example above.
>>
>> Which number would that be?
>
> I told you, any numbers too close to 2.

All numbers close to 2 in the same precision will work out ok.


> On the contrary, it is done because 80-bit is faster and more 
> precise, whereas your notion of reliable depends on an 
> incorrect notion that repeated bit-exact results are better.

80 bit is much slower. 80 bit mul takes 3 micro ops, 64 bit takes 
1. Without SIMD 64 bit is at least twice as fast. With SIMD 
multiply-add is maybe 10x faster in 64bit.


And it is neither more precise or more accurate when you don't 
get consistent precision.

In the real world you can get very good performance for the 
desired accuracy by using unstable algorithms by adding a stage 
that compensate for the instability. That does not mean that it 
is acceptable to have differences in the bias as that can lead to 
accumulating an offset that brings the result away from zero 
(thus a loss of precision).


> You noted that you don't care that the C++ spec says similar 
> things, so I don't see why you care so much about the D spec 
> now.
>  As for that scenario, nobody has suggested it.

I care about what the C++ spec. I care about how the platform 
interprets the spec. I never rely on _ONLY_ the C++ spec for 
production code.

You have said previously that you know the ARM platform. On Apple 
CPUs you have 3 different floating point units: 32 bit NEON, 64 
bit NEON and 64 bit IEEE.

It supports 1x64bit IEEE, 2x64bit NEON and 4x32 bit NEON.

You have to know the language, the compiler and the hardware to 
make this work out.


>> And so is "float" behaving differently than "const float".
>
> I don't believe it does.

I have proven that it does, and posted it in this thread.



More information about the Digitalmars-d mailing list