Very limited shared promotion

[Walter Bright]

On 6/19/2019 6:45 AM, Timon Gehr wrote:
> On 19.06.19 11:13, Walter Bright wrote:
>>
>> Using the resulting value of x:
>>
>>    int x;
>>    void fun(scope ref shared(int) x) { ... }
>>    fun(x); // implicit promotion to shared in this case
>>    int y = x; // add this line
>>
>> And now there's a problem. The compiler thinks x is thread local, so it 
>> doesn't add synchronization to the read of x.
> 
> Probably `fun` should be `immutable` or `shared` as well. Otherwise there is no 
> way to tell at the call site that it does not also capture `x` mutably, in which 
> case the implicit promotion couldn't be allowed.

`scope` prevents capture of a reference to `x`, mutable or not, that persists 
after the return from `fun`.


>> Not adding synchronization means that although fun() has terminated all the 
>> threads it spawned accessing x, it does not mean the memory caches of x are 
>> updated.
>> ...
> 
> The @trusted code that decided to send the `scope` reference to other threads 
> has to make sure that all reads/writes to the `scope`d reference context are 
> visible on the current thread before it returns. (Otherwise, from the 
> perspective of the current thread, there are still copies of the `scope` 
> reference around, even though the function call has terminated, which goes 
> against `scope` guarantees.)

The way atomic synchronization of variables works is when a write is performed, 
a write access fence happens after. When a read access is performed, a read 
access fence is performed.

The last write to shared x in fun will do a write access fence, but the read 
access fence won't happen until there's the read of shared x, the read access 
fence won't happen. But the x after the return of fun is not shared, no read 
access fence will happen, and the read may not see the results of the last write 
to shared x.


>> While you may have coded fun() to update the caches of x before returning, the 
>> compiler can't know that, and so the compiler cannot allow the implicit 
>> conversion.
>> ...
> 
> The compiler can allow the implicit conversion because in @safe code there is no 
> way to send `scope` data to other threads and @trusted code has to respect 
> `scope` guarantees.

The scope guarantees are NOT sequential consistency data-race-free guarantees 
for multithreaded access.


>> In other words, adding `scope` does not guarantee SC-DRF (Sequential 
>> Consistency - Data Race Free).
> I think it does. The challenge is to ensure that the implicit promotion to 
> `shared` doesn't result in `shared`/unshared aliasing. I don't think there is a 
> problem with implicitly casting back to unshared, as the called function 
> promises not to leak the references.

It does not. A write to an atomic is:

     write variable
     read fence

A read from an atomic is:

     read fence
     read variable

SC-DRF of atomics absolutely depends on this. The example code does:

   thread 1:
     write x
     read fence

   thread 2:
     read x

This is wrong wrong wrong from what I know about atomics.

The only way a local reference can be implicitly converted to a shared reference 
is if the compiler can prove there are no other local references to that memory 
location, which is essentially what Rust does. The read of `x` after `fun` 
returns violates that principle, and the result is a data race.

---

There's another way to look at this:

1. When a politician promises a Free Lunch, you're gonna pay for it.
2. When a scientist discovers perpetual motion, he's made a mistake.
3. When a salesman sells you an effortless exercise machine, you won't get stronger.
4. When you respond to an ad for a "Get Rich Quick Through Real Estate" seminar, 
you're a sucker.

   -- and --

5. When a data-race-free solution is found that doesn't follow fencing 
protocols, it doesn't work.