type comparisons

[Bill Baxter]

Denton Cockburn wrote:
> Jérôme M. Berger Wrote:
> 
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Denton Cockburn wrote:
>>> On Sat, 26 Jan 2008 18:36:41 +0000, BCS wrote:
>>>
>>>> Reply to Denton,
>>>>
>>>>> [quoted text muted]
>>>> rounding error?
>>>>
>>>> rounding 0.6584 (base 10) to real and then converting to double may round 
>>>> different than rounding it directly to double.
>>> I can't think of a reason that would be acceptable.
>>> This means I can't reliably do operations with converted reals/doubles :(
>>>
>>> This is also weird:
>>> double a = 0.65;
>>> real b = a + 0.1;
>> 	Here, you're computing "a + 0.1" using double, then converting it
>> to real
>>
>>> b -= 0.1;
>>> assert(b == a); // fails
>>>
>>> real c = a;
>>> c += 0.1;
>> 	Here, you converted a to real, then you add 0.1 using real.
>>
>>> c -= 0.1;
>>> assert(c == a); // passes
>> 	Which could explain that rounding errors wind up different and make
>> one assertion pass while the other fails.
>>
>> 		Jerome
>> - --
>> +------------------------- Jerome M. BERGER ---------------------+
>> |    mailto:jeberger at free.fr      | ICQ:    238062172            |
>> |    http://jeberger.free.fr/     | Jabber: jeberger at jabber.fr   |
>> +---------------------------------+------------------------------+
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.7 (GNU/Linux)
>>
>> iD8DBQFHm4y1d0kWM4JG3k8RAgfQAKC3LwsQfklEVT+cP2w3HrR/xtK6JACfdPr9
>> PQIwRTbf2zyhpst9zdeZn9s=
>> =RA4Q
>> -----END PGP SIGNATURE-----
> 
> The problem I'm having is that we are comparing the same numerical value for equality, and getting a (logically) false result.


One tenth is not exactly representable in floating point.  Therefore you 
get rounding errors the moment you try to store .1 in a 
float/double/real.  real can represent 1/10 a bit more accurately than 
double.  So .1 as a double, then cast to a real is not quite the same as 
.1 that started as a real.

Try .125 or 0.0625 and you'll get a good answer.  But .1 is bad news for 
floating point.

--bb