Always false float comparisons
Marco Leise via Digitalmars-d
digitalmars-d at puremagic.com
Mon May 16 15:16:30 PDT 2016
I wondered how the precision of typical SSE vs FPU code would
compare. SSE uses floats and doubles as the main mean to
control precision, while the FPU is configured via x87 control
word.
The attached program calculates 5^-441 iteratively using these
methods, then converts to double and prints the mantissa:
Compiled with Digital Mars D in 64-bit
float SSE : 00000000000000000000000000000000000000000000000000000
float x87 : 00100000101010100111001110000000000000000000000000000
double SSE: 00100000101010100111001101111011011110011011110010001
double x87: 00100000101010100111001101111011011110011011110010000
real x87 : 00100000101010100111001101111011011110011011110001111
Take-aways:
- SSE results generally differ from x87 results at the same
precision
- x87 produces accurate single precision results, whereas
the error with SSE is fatal (and that's not a flush-to-zero;
the greatest power that still produces 1 bit of mantissa
is 5^-64(!))
--
Marco
-------------- next part --------------
A non-text attachment was scrubbed...
Name: app.d
Type: text/x-dsrc
Size: 1131 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20160517/6c78132a/attachment.d>
More information about the Digitalmars-d
mailing list