Very limited shared promotion

[Manu]

On Fri, Jun 21, 2019 at 7:05 PM Walter Bright via Digitalmars-d
<digitalmars-d at puremagic.com> wrote:
>
> On 6/21/2019 1:17 AM, Manu wrote:
> > I agree it's 'theoretically' valid; that's why I suggested it and the
> > construct makes sense.
> > How would you write code that escapes the scope reference to another
> > thread? This must require some @trusted code to achieve sharing
> > between threads; at that point, the @trusted function is responsible
> > for re-instating the thread-locality guarantees (which are trivial to
> > implement). If you don't do @trusted mischief, there's no way to
> > migrate the value across threads, so your synchronisation issue
> > shouldn't exist if no @trusted code exists.
>
> I attempted to explain this before.

And we heard you.

> The trouble does NOT come from the shared
> function escaping a reference. It comes from the lack of memory coherency
> between threads.

There is no "between threads" unless you have already entered @trusted land.
The reason for my proposal in the OP is precisely to prevent passing
data to other threads, that's the key that makes this promotion safe.

Can you show how you managed to get a ref to another thread without
entering @trusted?

> Scope says NOTHING about synchronizing access to shared memory.

Yes, obviously. But the point is to prevent that situation occurring
in the first place. There's no possibility of the synchronisation
issue arising unless there's some way to escape a scope ref to another
thread, which scope prevents.

Can you show how to escape a scope ref to another thread?

> NOTHING.

No really, we get it. This isn't a revelation to anyone.

> Just
> because there's no longer a reference to the shared memory location from another
> thread does not mean all the memory caches are synchronized. For that,
> synchronization is needed, and your load of thread local `x` does NOT include
> the necessary synchronization.

So, as I attempted to explain before; there ARE NO OTHER THREADS
unless you engage in @trusted activity.
There are no synchronisation issues unless there are other threads,
and there are no other threads because you couldn't possibly get a
reference to another thread.
Can you show how you escaped a reference to another thread without
entering @trusted territory?

Now, assuming we intend to write a @trusted machine to implement a
parallel for, it's trivial to insert the 4 memory barriers to maintain
data consistency. This is fine, we're in unsafe land, and there is no
way to get to this situation without going unsafe.

> If `x` was types as shared, the compiler WOULD include the necessary
> synchronization when reading it.

It had better not. I don't believe it can know how to do that correctly.
I don't think a type qualifier is even remotely enough information for
the compiler to implement data fencing correctly and efficiently.

> That's what atomic reads are for. It's the
> entire point of atomic reads.

We're not talking about atomic reads here, we're talking about memory barriers.

`shared` just needs to separate thread-local from shared, and make
transfer between those two states unsafe. Then we can write the
libraries we need.
Remove all access from shared, then the only way we can do anything
with shared is to cast it away, which is a @system operation and as
such, places responsibility on us as library authors to do the right
stuff.

Novices don't write threading libraries. They tend to be very small in
surface area, and written once. It'll be fine, and it gives us the
control we need.
Anything more from the language at this very early stage is an over-reach.

If you want to explore @safe expansions to shared, then propose them
as follow up, because they will be very controversial, whereas the
foundation stuff is uncomplicated and mostly non-controversial.
`shared` the language primitive is just not safe, and let us write the
libraries.