[Issue 3248] lossless floating point formatting

[d-bugmail at puremagic.com]

http://d.puremagic.com/issues/show_bug.cgi?id=3248





--- Comment #2 from assorted <moi667 at hotmail.com>  2009-08-12 15:40:10 PDT ---
(In reply to comment #1)
> It's not that easy, actually. When should it print 0.09999999999999999, and
> when should it print 0.1 ? The code to do it correctly is amazingly
> complicated.
> Just be aware that what you're asking for is much more difficult than you
> probably imagine.

It is less difficult than you imagine :)

Lets take floats:

A float has at most 24bits of precision

2^-24 = 0.000000059604644775390625
2^-23 = 0.00000011920928955078125

to distinguish between these two you only need a precision of 8.

Thus %.8e will always be lossless but isn't always the nicest way of
representation. %g fixes this by using %f if the exponent for an e format is
greater than -5 and less than the precision. The less than precision part is
correct, but the greater than 10^-5 is bad as the precision specifies the
number of digits generated after the decimal point; not excluding leading
zeros.
If %g would be changed to use %f only between 10^-1 and precision that would
solve that problem, if %f were to elide trailing zeros.

Back to the 0.1 question. 0.1 is actually saved as 0.1000000012...
Eliding trailing zeros from %f.8 would be sufficient to get 0.1

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------