Very limited shared promotion

[Timon Gehr]

On 20.06.19 03:30, Walter Bright wrote:
> On 6/19/2019 3:04 PM, Timon Gehr wrote:
>> I was talking about this:
>>
>> int x0;
>> void fun(scope ref shared(int) x1){
>>      assert(&x0 is &x1); //uh, oh
>>      // ...
>> }
>> fun(x);
> 
> I see.
> 
> 
>> Right now, in `@safe` code it guarantees that there is _no_ 
>> multithreaded access, no? How do you leak a `scope` reference to 
>> another thread? If there is _no_ multithreaded access, you don't get 
>> any data races, so `scope` guarantees no data races on the `scope`d 
>> memory location given that the reference wasn't accessible from 
>> multiple threads before.
> 
> `scope` only guarantees no references to the scoped variable exist 
> following the exit of the function. What happens inside is anything goes 
> as long as that invariant is maintained.

So in purely @safe code without any calls to @trusted functions you 
can't send that reference anywhere.

> That would include starting up 
> a new thread that can access the scope variable, as long as the thread 
> terminates its use of that variable before the function exits.
> ...


Yes, but terms such as "before" are tricky when multiple threads are 
involved. Semi-formally, what you are saying is that there has to be a 
path from each write access in each spawned thread to the return of the 
function in the happens- before graph. I don't think it is possible to 
establish this without making it valid to access an unique reference 
without further memory barriers. I don't think you can allow a race to 
exist between the function return and any write accesses to `scope`'d 
parameters.


> 
>> Yes, what's above certainly does not work. My point was that that is 
>> not what the code would do, at least not legally. If the above 
>> happens, there would be some `@trusted` code somewhere that you can 
>> blame for it.
>>
>> Sending a `scope` variable to another thread is a `@system` operation. 
>> In order to ensure that `scope` guarantees are met, the `@trusted` 
>> code that performs the send needs to add memory barriers that ensure 
>> that all writes to the `scope` variable are visible on the current 
>> thread before it returns. So on thread 2 you would actually also have 
>> barriers before `x` is accessed again.
> 
> `scope` does not offer such guarantees. It only guarantees no leaks of 
> the reference that survive the end of the function. It does not 
> guarantee synchronization of memory caches.
> ...

My complaint is that the last two sentences appear mutually contradictory.

>>
>> Let's consider a case _without_ implicit promotion:
>>
>> shared(int) x=0; // x is shared all the way
>> static void foo(scope ref shared(int) x)@trusted{
>>      // possibly send &x to other threads that mutate it and
>>      // then forget about &x
>>      // ...
>> }
>> foo(x);
>> int y=x; // read x
>> enforce(x==y); // read x again
>>
>> My claim is that if the enforcement fails, this indicates a bug in 
>> `foo`, because it violates guarantees you can derive from the fact 
>> that `x` is a local variable that is only passed to some other code 
>> via a `scope`d parameter: when `x` is read for the first time, we know 
>> that actually no other thread can have a reference to it which it can 
>> use for a write, so its value has to remain constant.
> 
> That is not what `scope` does,

We are using the same definition of `scope`. You haven't however 
provided a formal definition of what "before" means.

> but the example is still correct because 
> the language does not say that x can be converted to a local, and so it 
> is not converted, and the memory synchronization of x is maintained and 
> it works.
> ...

(Obviously that example works. That's not the point.)

> 
>> It should therefore be possible to read `x` without any further 
>> barriers after `foo` returns. This is because `foo` already needs to 
>> ensure all writes from other threads are visible.
> 
> `scope` does not add such barriers. The implementer of foo may add a 
> fence to do that, but the compiler doesn't know that and cannot rely on it.
> 
> 
>> It is possible that I am missing something (even though what you wrote 
>> is not it). Is there a possible implementation of `foo` that would be 
>> correct (in a language that does not have any implicit `shared` 
>> promotion), but for which a data race on `x` would result if we didn't 
>> add another fence before reading `x` after `foo` has returned?
> 
> AFAIK, another fence is required upon return of foo and before reading 
> x. It's not just about whether an extra reference exists, it's about 
> when the memory cache coherency happens.
> ...


I don't think it makes any sense to claim "no other reference exists" at 
a specific program point if at that point you didn't properly 
synchronize with parties potentially still holding such a reference.