Shared

[Jonathan M Davis]

On Wednesday, May 15, 2019 1:56:12 AM MDT Radu via Digitalmars-d wrote:
> On Tuesday, 14 May 2019 at 21:02:10 UTC, Jonathan M Davis wrote:
> > On Tuesday, May 14, 2019 8:32:45 AM MDT Radu via Digitalmars-d
> >
> > wrote:
> >> On Monday, 13 May 2019 at 16:20:30 UTC, Jonathan M Davis wrote:
> >> > On Monday, May 13, 2019 9:52:02 AM MDT Dominikus Dittes
> >> > Scherkl
> >> >
> >> > via Digitalmars-d wrote:
> >> >> [...]
> >> >
> >> > Actually, it doesn't. All you've done is lock a mutex and
> >> > cast away shared. There is no guarantee that that mutex is
> >> > always used when that object is accessed. There could easily
> >> > be another piece of code somewhere that casts away shared
> >> > without locking anything at the same time. And even if this
> >> > were the only mechanism for removing shared, you could
> >> > easily use it with the same object and a completely
> >> > different mutex in another piece of code, thereby making the
> >> > mutex useless. The type system has no concept of ownership
> >> > and no concept of a mutex being associated with any
> >> > particular object aside from synchronized classes (and those
> >> > don't even currently require that the mutex always be used).
> >> > And even if the compiler could check that all uses of a
> >> > particular variable were locked with a mutex, that still
> >> > wouldn't be enough, because other references to the same
> >> > data could exist. So, with the construct you've proposed,
> >> > there's no way for the compiler to guarantee that no other
> >> > thread is accessing the data at the same time. All it
> >> > guarantees is that a mutex has been locked, not that it's
> >> > actually protecting the data.
> >> >
> >> > [...]
> >>
> >> I had this idea for some time, not sure I can best articulated
> >> now, but I think a workable solution for shared is closely
> >> linked with the dip1000 - scoped pointers.
> >>
> >> What I mean is that something like:
> >>
> >> void doSharedStuff(scope shared(Foo) foo)
> >> {
> >> }
> >>
> >> Will be able to safely lock/unlock foo and cast away
> >> shared'ness
> >> in the function's scope.
> >> The compiler can provide guarantees here that foo will not
> >> escape.
> >
> > Sure, it can guarantee that no reference will escape that
> > function, but all that's required is that another reference to
> > the same data exist elsewhere, and another thread could muck
> > with the object while the mutex was locked. There's no question
> > that helpers could be created which would help users avoid
> > mistakes when casting when casting away shared, but the
> > compiler can't actually make the guarantee that casting away
> > shared is thread-safe.
> >
> > - Jonathan M Davis
>
> My view is that the compiler could automatically insert locking
> logic (ala synchronized) when the shared parameters gets
> references inside the function, and also automatically cast away
> shared so for the function internals it would be like working
> with local non-shared data.
>
> Given that compiler inferes lifetime, it could safely elide
> locking if the parameter is passed to other functions that have
> the same signature (scope is explicit or inferred).
>
> The idea is to provide the tools that simplify concurrent
> programming, the compiler will need to insert all the checks
> automatically using the lifetime tracking.

The compiler does not have enough information to know which mutexes to use
when even if we wanted it to insert locks. TDPL synchronized classes are a
special case in that they would provide a way in the language to associate a
specific lock with a specific set of variables in a way that the compiler
could then guarantee that it's safe to remove the outer layer of shared
within a synchronized function. Without a similar mechanism to associate a
mutex with one or more variables and guarantee that that mutex is always
locked when they're accessed, the compiler won't be able to even know what
to lock, let alone that it's safe to remove shared within a particular
section of code.

And the issue with what you're proposing with scope is that unless the
compiler can actually guarantee that no other references to the shared
object exist which could be used to access the object at the same time, then
the compiler cannot safely remove shared even temporarily. scope is just
enough to guarantee that that particular function can't escape any
references, not enough to guarantee that they don't exist. So, while
features like scope can be used to make it easier to reason about the code
and cast away shared in a way that your code is thread-safe, I fully expect
that it's going to have to be up to the programmer to know when it's safe to
remove shared, thus requiring a cast or some other @trusted mechanism to
temporarily remove shared. TDPL synchronized classes are the only proposal
I've seen that would be able to guarantee thread-safety, and it can only do
it for what's directly in the class, making TDPL synchronized classes
arguably pretty useless (on top of the fact that it means requiring classes
when most D code wouldn't normally use classes for something like this).

- Jonathan M Davis