I wish all qualifiers were revisited with an eye for simplification

[Manu]

On Tue, Aug 4, 2020 at 11:55 PM Sebastiaan Koppe via Digitalmars-d <
digitalmars-d at puremagic.com> wrote:

> On Tuesday, 4 August 2020 at 12:52:01 UTC, Manu wrote:
> > On Tue, Aug 4, 2020 at 5:40 PM Sebastiaan Koppe via
> > Digitalmars-d < digitalmars-d at puremagic.com> wrote:
> >> There is just one thing about shared I don't understand. If I
> >> design my object such that the non-shared methods are to be
> >> used thread-local and the shared methods from any thread, it
> >> follows that I should be able to call said shared methods from
> >> both a shared and non-shared instance of that object.
> >
> > Yes, this is a thing I talked at great length 1-2 years ago.
> > If you take shared to mean "is thread-safe", then my idea was
> > that
> > not-shared -> shared implicit conversion should be possible.
> > What I often do is this:
> >
> > struct Thing
> > {
> >   ref shared(Thing) implSharedCast() { return
> > *cast(shared)&this; }
> >   alias implSharedCast this;
> > }
> >
> > If that were an implicit conversion, that implies a slight
> > change of
> > meaning of shared (to one that I consider immensely more
> > useful), but it's
> > more challenging for the compiler to prove with confidence, and
> > there's a
> > lot of resistance to this change.
>
> What exactly does the compiler need to prove? The restrictions
> are all in place, you can only call shared methods on a shared
> object, and you can only access shared members in a shared method.
>

Marking the function shared (and therefore the data accessible) doesn't
magically make the function body threadsafe; what it does it makes the
function potentially racy, and subject to VERY careful implementation.
Sadly, I'm not aware of any CS research that can help you write
atomic/threadsafe functions with any degree of proof in an environment like
this (ie, without isolation and/or things like copy-on-write, etc).

Calling shared methods from shared methods isn't safe either. Each call may
be threadsafe atomically, but you can't author a leaf function without just
as much care (or more) than the lower-level ones. If the function is more
than 1-line, and carries some state across a few statements that call
lower-level shared methods, then you need to be confident about the atomic
state guarantees (or not! which is more likely) of the dependency API's,
such that you don't create a state race between calls.

Needless to say, it's still tricky, and there's really nothing the language
can help you do... other than provide a strong mechanism to lock it off
from normal code.
As long as shared methods are few, and very well defined, then it is
possible to implement very interesting and useful machines with this
scheme, and also MASSIVELY help with validation of your ecosystem, and I
have done so... but it's not magic, it's just a lot better than C++.

The thing `shared` protects most against, is failure to correct
multithreaded code when refactoring. I maintain a majorly-superscalar
engine, and 90% of the race bugs we have had to spend heaps of time chasing
down are the result of refactors or changes that had very peripheral
contact with the multithreaded core; contact was narrow enough that we
didn't notice then making changes, and there was nothing in the type system
to alert us. For this reason alone, `shared` is worth its weight in gold.

Hmm, well I guess if members themselves were also implicitly
> promoted to shared that could cause havoc.
>

The catch though, is that if you allow implicit conversion from
unshared->shared, then the rule changes from "shared methods must be
threadsafe", to " shared methods must be threadsafe, AND unshared methods
must also be threadsafe in the event they are called in conjunction with
any of the shared methods (just not with eachother)".
While I have plenty of small tools where the second rule is easy to
implement, I can also think of many situations where even the first rule is
hard to implement, and the second rule is virtually impossible.

Unless we learn better ways to handle this, I don't think implicit
conversion that way can scale. For the time being, you can implement the
implicit conversion for the tools that may support that using `alias this`
like I showed above, and I think that's appropriate for the time being.

In general, I'd like to find ways we can enhance shared to be more than a
marker; consider TSAN (ThreadSanitizer) in Clang; it instruments atomics
with runtime tracking to detect races at runtime. I think it would be
possible to instrument shared data/methods in a similar way in debug
builds, and then we'd have meaningful compiler assistance.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20200805/09119952/attachment-0001.htm>