Always false float comparisons

Marco Leise via Digitalmars-d digitalmars-d at puremagic.com
Mon May 16 15:16:30 PDT 2016


I wondered how the precision of typical SSE vs FPU code would
compare. SSE uses floats and doubles as the main mean to
control precision, while the FPU is configured via x87 control
word.

The attached program calculates 5^-441 iteratively using these
methods, then converts to double and prints the mantissa:

  Compiled with Digital Mars D in 64-bit
  float SSE : 00000000000000000000000000000000000000000000000000000
  float x87 : 00100000101010100111001110000000000000000000000000000
  double SSE: 00100000101010100111001101111011011110011011110010001
  double x87: 00100000101010100111001101111011011110011011110010000
  real x87  : 00100000101010100111001101111011011110011011110001111

Take-aways:
- SSE results generally differ from x87 results at the same
  precision
- x87 produces accurate single precision results, whereas
  the error with SSE is fatal (and that's not a flush-to-zero;
  the greatest power that still produces 1 bit of mantissa
  is 5^-64(!))

-- 
Marco
-------------- next part --------------
A non-text attachment was scrubbed...
Name: app.d
Type: text/x-dsrc
Size: 1131 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20160517/6c78132a/attachment.d>


More information about the Digitalmars-d mailing list