Shared

[Jonathan M Davis]

On Tuesday, May 14, 2019 6:22:20 AM MDT Dominikus Dittes Scherkl via 
Digitalmars-d wrote:
> On Monday, 13 May 2019 at 16:20:30 UTC, Jonathan M Davis wrote:
> > On Monday, May 13, 2019 9:52:02 AM MDT Dominikus Dittes Scherkl
> >
> > via Digitalmars-d wrote:
> >> On Saturday, 11 May 2019 at 15:24:19 UTC, Jonathan M Davis
> >>
> >> wrote:
> >> > All that really would do is add a bit of extra syntax around
> >> > locking a mutex and casting away shared.
> >>
> >> No, it makes it possible to use mutex in safe functions
> >> without needing to cast and which the compiler CAN guarantee
> >> to really be safe.
> >
> > Actually, it doesn't. All you've done is lock a mutex and cast
> > away shared. There is no guarantee that that mutex is always
> > used when that object is accessed. There could easily be
> > another piece of code somewhere that casts away shared without
> > locking anything at the same time.
>
> But that piece of code is system, which explicitly allows you to
> shoot into your foot. If you want to stay safe, don't do that.

The point is that if that code can legally exist, then the compiler simply
cannot guarantee that removing shared from the object is thread-safe even
with the locking mechanism you're proposing.

> > And even if this were the only mechanism for removing shared,
> > you could easily use it with the same object and a completely
> > different mutex in another piece of code
>
> Yes, the lock block need a list of vars that it allows to be
> modified
>
> lock(var1, var2, ...)
> {
> }
>
> two mutexes can only be executed at parallel if their parameter
> set is disjunct.

Sure, but another thread could be using a completely different mutex with
one or more of those variables. So, even if just your locking mechanism is
used, there's no guarantee that the same mutex is always used with the same
set of variables, meaning that the compiler can't guarantee that the data is
actually protected.

> > And even if the compiler could check that all uses of a
> > particular variable were locked with a mutex, that still
> > wouldn't be enough, because other references to the same data
> > could exist.
>
> ok, so we need in addition that a reference to a shared var need
> not be lived beyond the end of the locked block or be immutable.
> Bad, but seems necessary.

A reference could already exist before your proposed locking mechanism was
reached in the code. If the type is a class or pointer, then there could be
other class references or pointers to the same data in @safe code. And in
@system/@trusted code, the address of the object could have been taken to
create a pointer to the object (and that could have been done in code for
removed from the code that's using the lock with all of the code around the
lock being @safe). Heck, there could even be references to data within the
object rather than to the object itself which are available elsewhere,
meaning that part of the object is protected by the lock and part isn't. If
any reference to any part of the data exists anywhere in the program, then
it's possible for another thread to access the data at the same time that
it's locked by the mechanism that you've proposed.

In order for the compiler to be able to actually guarantee that it's safe to
remove shared from an object, it has to be able to guarantee that there are
no other references to any part of that object which exist in the program.
That's why TDPL synchronized classes are so locked down. Without that, other
references to the data could exist. And even with all of the restrictions
that they have, the compiler would still only be able to remove the outer
layer of shared - the layer directly in the class - not any more than that.
With something as free-form as you're proposing, the compiler can't
guarantee anything, let alone that it's thread-safe to completely remove
shared from an object within that block.

If D had ownership semantics baked into its type system, then we could
probably do more, but as it is, the compiler is _very_ limited in its
ability to know that no other references to any portion of an object exist.
All it's dealing with are the types, not what owns them or what else could
refer to them. Even scope is only able to do its job by restricting the
operations that are allowed on a scope object, not by actually tracking an
object's lifetime. So, it's incredibly difficult for the compiler to have
enough information to know that an object is actually fully protected by a
mutex when it's locked. And if the compiler can't guarantee that accessing
the shared object is thread-safe within a particular piece of code, then it
can't safely remove shared.

For any proposal you might have with regards to how we might be able to have
the compiler safely remove shared from an object for us, you're going to
have to be able to prove that no other references to any piece of that
object could possibly exist or that there's no way that any reference to any
portion of that object could be accessed without the same mutex being locked
at every point that it's accessed. And it wouldn't surprise me if someone
else were able to point out why even that wasn't enough because of some
detail I'm not thinking of at the moment. Having the compiler be able to
prove that a piece of code is thread-safe such that shared can be safely
removed automatically from anything is incredibly difficult.

- Jonathan M Davis