Cleaner way of working with a shared resource?

[Jonathan M Davis]

On Thursday, March 29, 2018 18:50:39 Chris M. via Digitalmars-d-learn wrote:
> On Thursday, 29 March 2018 at 18:07:50 UTC, Jonathan M Davis
>
> wrote:
> > On Thursday, March 29, 2018 17:41:15 Chris M. via
> >
> > Digitalmars-d-learn wrote:
> >> [...]
> >
> > In general, the correct way to deal with a shared object is to
> > protect access to it with a mutex and then within that
> > protected section, you cast away shared so that it's treated as
> > thread-local so that it can be operated on. Then when you
> > release the mutex, you make sure that no thread-local
> > references to the shared object persist. Most operations really
> > aren't supposed to work while the object is treated as shared,
> > because then you have a threading problem if the operation is
> > non-atomic and doesn't protect itself with a mutex (though a
> > member function on an object certainly could use a mutex
> > internally, allowing it to be shared and work on a shared
> > object).
> >
> > [...]
>
> Awesome, thanks for the explanation. Didn't realize synchronized
> had that extra syntax and what it did, that'll save me a few
> headaches. Now what would really be nice is if that syntax also
> cast away the shared implicitly while inside the critical
> section, but I can deal with it for now.

Well, there's no guarantee that you've actually properly protected access to
the object everywhere, so it wouldn't be safe to automatically cast away
shared for you. The language feature that is supposed to act similarly to
that is synchronized classes, but they've never been fully implemented.
Right now, marking a class as synchronized just makes all of its functions
synchronized and disallows public member variables. It doesn't actually
provide any benefits with regards to removing shared like it's supposed to
(and even the fact that it disallows public member variables is fairly
recent).

Fully implemented synchronized classes would remove the outer layer of
shared on the member variables, because the compiler could guarantee that
the outer layer had not escaped, because it would be impossible to access
the member variables except through the member functions which would all be
synchronized. So, you could then have a shared object protected by a
synchronized class and sort of use it as thread-local within the member
functions of the synchronized class. The problem is that it can only safely
remove the outer layer of shared, so anything referred to by the shared
object would still be shared - e.g. shared(T*) would become shared(T)*. So,
the pointer would then be manipulated as thread-local but not what it refers
to. Value types would then be fully thread-local, so they'd be fine, but
anything involving any kind of pointer reference would be in trouble. I
don't think that it would even be possible to strip shared from a member
variable that's a class reference, because the only part that could safely
have shared stripped away would be the reference itself, and the type system
doesn't treat that as separate from the class. Regardless, the class itself
couldn't be treated as thread-local any more than the data pointed to by a
pointered could be.

Basically, for value types, you'd get to treat them as thread-local, but
anything referred to by the member (be it via pointer or reference or a
dynamic array) would then still be shared. So, while _some_ operations could
then be done as if the member were thread-local, many operations still
couldn't be. It would work great for value types, but for reference types,
you're pretty much just as screwed as you are now.

So, synchronized classes would help, but they wouldn't really fix the
problem. Unfortunately, they're also the best solution that has been
proposed. We'd very much like to have a way to safely remove shared
automatically while the mutex is locked, but we simply don't have a way to
guarantee that everything has been properly protected by the same mutex
everywhere, so we can't automatically strip away shared while the mutex is
locked. synchronized classes have to restrict things a fair bit in order to
be able to remove as much of shared as they do, and even then, they can't
remove much. Anything which wanted to fully strip away shared would probably
have to somehow guarantee that _nothing_ that it referred to was accessible
elsewhere, and the type system doesn't support anything like that. The
programmer can often see that that's the case, but the compiler can't prove
it. So, ultimately, it's a bit like @trusted. The programmer has to examine
the code in question and verify that it's doing the right thing, which
includes managing the casting away of shared. It's annoying to be sure, but
no one has come up with a better solution yet, and honestly, I doubt that
they will without adding something pretty major to the type system.

Ultimately, the way that shared is used is pretty much the same as what you
get in C++, except in D, you're prevented from manipulating the object in a
manner that isn't thread-safe (assuming shared has no bugs anyway), and
you're forced to cast away shared while the mutex protecting the object is
locked, whereas in C++, you get no proctections against using the object in
a manner which is not thread-safe, and you don't have to cast. But the C++
code would be pretty much the same except that it wouldn't include a cast,
and the way that the mutex would be locked wouldn't involve synchronized.
However, all of the compiler errors that folks get when trying to use shared
seems to confuse and anger most folks. It's frequently assumed that you
should be able to use the object while it's shared, but unfortunately, we
really don't have a way to do that, much as we'd like to - at least, not
while having shared do its job of preventing you from using it in a way that
isn't thread-safe.

- Jonathan M Davis