Always false float comparisons

jmh530 via Digitalmars-d digitalmars-d at puremagic.com
Fri May 13 22:52:17 PDT 2016


On Saturday, 14 May 2016 at 01:26:18 UTC, Walter Bright wrote:
>
> An anecdote: a colleague of mine was once doing a chained 
> calculation. At every step, he rounded to 2 digits of precision 
> after the decimal point, because 2 digits of precision was 
> enough for anybody. I carried out the same calculation to the 
> max precision of the calculator (10 digits). He simply could 
> not understand why his result was off by a factor of 2, which 
> was a couple hundred times his individual roundoff error.
>
>

I'm sympathetic to this. Some of my work deals with statistics 
and you see people try to use formula that are faster but less 
accurate and it can really get you in to trouble. Var(X) = E(X^2) 
- E(X)^2 is only true for real numbers, not floating point 
arithmetic. It can also lead to weird results when dealing with 
matrix inverses.

I like the idea of a float type that is effectively the largest 
precision on your machine (the D real type). However, I could be 
convinced by the argument that you should have to opt-in for this 
and that internal calculations should not implicitly use it. 
Mainly because I'm sympathetic to the people who would prefer 
speed to precision. Not everybody needs all the precision all the 
time.


More information about the Digitalmars-d mailing list