Floating Point + Threads?

[dsimcha]

On 4/16/2011 10:11 AM, dsimcha wrote:
> My only concern is whether this non-determinism represents some deep
> underlying bug. For a given work unit allocation (work unit allocations
> are deterministic and only change when the number of threads changes or
> they're changed explicitly), I can't figure out how scheduling could
> change the results at all. If I could be sure that it wasn't a symptom
> of an underlying bug in std.parallelism, I wouldn't care about this
> small amount of numerical fuzz. Floating point math is always inexact
> and parallel summation by its nature can't be made to give the exact
> same results as serial summation.

Ok, it's definitely **NOT** a bug in std.parallelism.  Here's a reduced 
test case that only uses core.thread, not std.parallelism.  All it does 
is sum an array using std.algorithm.reduce from the main thread, then 
start a new thread to do the same thing and compare answers.  At the 
beginning of the summation function the rounding mode is printed to 
verify that it's the same for both threads.  The two threads get 
slightly different answers.

Just to thoroughly rule out a concurrency bug, I didn't even let the two 
threads execute concurrently.  The main thread produces its result, then 
starts and immediately joins the second thread.

import std.algorithm, core.thread, std.stdio, core.stdc.fenv;

real sumRange(const(real)[] range) {
     writeln("Rounding mode:  ", fegetround);  // 0 from both threads.
     return reduce!"a + b"(range);
}

void main() {
     immutable n = 1_000_000;
     immutable delta = 1.0 / n;

     auto terms = new real[1_000_000];
     foreach(i, ref term; terms) {
         immutable x = ( i - 0.5 ) * delta;
         term = delta / ( 1.0 + x * x ) * 1;
     }

     immutable res1 = sumRange(terms);
     writefln("%.19f", res1);

     real res2;
     auto t = new Thread( { res2 = sumRange(terms); } );
     t.start();
     t.join();
     writefln("%.19f", res2);
}


Output:
Rounding mode:  0
0.7853986633972191094
Rounding mode:  0
0.7853986633972437348