synchronized - shared but actually useful

[Jonathan M Davis]

On Friday, October 26, 2018 7:21:50 AM MDT FeepingCreature via Digitalmars-d 
wrote:
> ping
>
> On Tuesday, 23 October 2018 at 08:56:39 UTC, FeepingCreature
>
> wrote:
> > Disadvantages:
> > * inability to generate giant forum debates (fingers crossed)
>
> I consider the point proven, but the downside of being unable to
> generate giant flamewars is that you eventually drop off the
> frontpage.
>
> I really think the shared viewpoint on thread-owned data and how
> to move data from thread to thread is very alien to how thread
> safety is handled in practice. In practice, a lot of data is
> shared; I would argue a large fraction of *all* mutable data is
> shared, and access is protected using synchronization in the
> owning class. That is, objects, not threads, are the owners and
> overseers of mutable data.
>
> Whatever solution for thread safety we arrive at should account
> for this.

Really, all shared is about is preventing common bugs related to
multithreading by segregating data shared across threads and making
operations which aren't guaranteed to be thread-safe illegal (the main
benefit then being that you know that everything else then isn't shared
across threads). The language doesn't do that entirely correctly at the
moment (e.g. copying and assigning shared data is currently legal in spite
of not being thread-safe) but ultimately, when you use shared, what you're
doing is pretty much exactly what you'd do in C/C++ except that the compiler
prevents many bugs with regards to data shared across threads, and because
of the extra type information in place to enable the restrictions to give
you that protection, you're often forced to cast away shared in @trusted
code to then do what you would have done in C++. But all of the atomics and
mutexes and whatnot are the same as what you'd be doing in C++. The basic
design patterns are the same. It's the fact that you you're getting compiler
errors when you do something that's not thread-safe that's different, and
it's the fact that you have to cast away shared after locking a mutex that's
different (since C++ doesn't have shared).

But aside from that, what you're doing is the same. The fact that the
compiler prevents you from doing basic stuff while the variable is shared
and requires casting (unless you use atomics) frequently causes folks to get
mad at shared and misunderstand it, but that's really the compiler
preventing you from shooting yourself in the foot. Once you understand
what's going on, it's really just the same as C++ but with some extra casts.

So, shared is very low level ultimately. It's about how the data is stored
and accessed, not about how higher level constructs are written. We can then
build other, higher level mechanisms on top of that - such as
std.concurrency - but at the language level, whether you're passing data
across threads or having multiple threads access data on the same thread
isn't really part of the design. The language is just concerned about the
fact that the data is shared.

The only higher level threading mechanism that the language has is
synchronized, and all that really is is a built-in mutex, which in
conjunction whith synchronized classes (if they're ever fully implemented)
would allow for the implicit casting away of the top level of shared inside
of member functions, but that's really as far as it goes, and without some
kind of ownership model in the language, it's _very_ hard to go much farther
than that - either with implicitly removing shared in code or with having
passing ownership across threads work without explicit casts.

Certainly, I think that any attempt to move shared away from being a low
level construct indicating simply that the data is shared across threads is
mis-guided. It's what other constructs for sharing data across threads are
built upon. So, if someone wants to push for some new mechanism that
improves sharing data across threads (rather than passing it across threads)
that's built on top of shared, that's fine. If someone wants to push for
some new mechanism that improves sharing data by passing it across threads,
that's fine. But shared is a low level construct and should stay that way.
It's the base for everything else.

And to be honest, I don't really expect it to be reasonable to add much in
the way of higher level mechanisms for sharing data to D at the language
level - not with it being a systems language. Most stuff would require
restricting it further and/or introducing a serious ownership model to it,
which we really don't want to do. Even synchronized classes are pretty
questionable, because they can only safely remove the outer layer of shared,
which is arguably better than nothing, but it doesn't get you very far.
Without a full-blown ownership model, we're pretty much reduced to manual
casting when mutexes or synchronized are used, and the language really isn't
going to be much help. If someone can come up with some bright ideas, great.
All the more power to them. Maybe we can improve the situation. But again,
they should be built on top of shared as a low level mechanism, not try to
change what shared is.

- Jonathan M Davis