Generality creep

[Jonathan M Davis]

On Wednesday, April 3, 2019 8:05:15 PM MDT Andrei Alexandrescu via 
Digitalmars-d wrote:
> On 3/31/19 6:25 PM, Manu wrote:
> > On Sat, Mar 30, 2019 at 5:45 PM Andrei Alexandrescu via Digitalmars-d
> >
> > <digitalmars-d at puremagic.com> wrote:
> >> On 3/30/19 2:49 PM, Jonathan M Davis wrote:
> >>> This would also be a great opportunity to fix some of the issues with
> >>> shared in druntime
> >>
> >> The problem here is, of course, that nobody knows exactly what shared
> >> is
> >> supposed to do. Not even me. Not even Walter.
> >
> > As an immediate stop-gap, shared must have *no read or write access*
> > to data members. Simple. Make that change. Right now.
> > Then shared will at least be useful and not embarrassing, and you can
> > continue to argue about what shared means while I get to work.
> >
> >> One monumental piece of work would be DIP 1021 "Define the semantics of
> >> shared". Then people who build with -dip1021 know what to do and what
> >> to
> >> expect. Now that would patch one of the bigger holes mentioned in the
> >> other post.
> >
> > Sure. But in the meantime, fix the objective bug with the current
> > semantics where you can read/write un-protected data freely. Right
> > now. Please for the love of god.
>
> Incidentally Walter and I discussed in the early days the idea that
> "shared" would offer no access whatsoever aside from a few special
> functions in druntime such as "atomicRead" and "atomicWrite". Of course,
> that raised the question how those functions could be implemented - the
> possible solution being some casts for typing and asm for the needed
> memory barriers.
>
> In the end we got scared that there was no precedent in other languages,
> and we could not predict whether that would have been good or bad. The
> result is the current semantics, which should be a felony in the 48
> contiguous US states.
>
> This does not work as a two stages process, though the "stop the
> bleeding first then come with the new solution" metaphor seems
> attractive. The main issues being when we break code that people got to
> work, we need to offer the alternative as well. Another being that the
> exact kind of things we disable/enable may be dependent on the ultimate
> solution.
>
> This would be a large effort requiring a strong team. Walter, yourself,
> and I would be helpful participants but I think between the three of us
> we don't have the theoretical chops to pull this off. At least I know I
> don't. We need the likes of Timon Gehr, Johan Engelen, and David
> Nadlinger (whom I cc'd just in case).

shared already tries to prevent various read/write operations that aren't
guaranteed to be thread-safe. So, it's already gone partially in that
direction, and if it's supposed to guarantee any kind of thread-safety,
that's the only way that it can go unless it's going to try to insert code
that ensures thread-safety - which would mean doing stuff like having
mutexes built in to what it's doing so that it knows what to lock and when,
and that's not something that's going to work as a low level construct. Even
the concept of synchronized classes as proposed in TDPL couldn't go far
enough with it to be very useful, because it could only safely cast away the
outer layer of shared. I thought that it was pretty clear that the way we
were going - and have to go - is to make it so that you simply can't read or
write to shared data without either using operations that are guaranteed to
be thread-safe or by casting away shared (at which point, the code is
@system, and it's up to the programmer to get it right, and that code is
properly segregated thanks to how @safe and @trusted work).

I grant you that we really do need the best people we have to look at this
at the low level and make sure that it works properly, but I don't see where
else we can go with this if we want any kind of guarantees from shared. If
we decide that we don't want shared to provide guarantees and that we want
to allow shared data to be read and written to regardless of whether the
compiler can guarantee thread-safety, then we can do that, but then things
that are currently illegal should be made legal (e.g. IIRC, incrementing
shared integers is illegal), and at that point, shared is just indicating
which variables are shared across threads without actually protecting you at
all. That would put it more in line with languages like C++, but it also
would mean that it would be a lot less useful for catching bugs.

Either way, what we have right now is halfway in between. It makes some
stuff that isn't thread-safe illegal but doesn't make it all illegal, making
it a pain to use if you don't cast away shared like you would need to if all
operations which weren't guaranteed to be thread-safe were illegal but
simultaneously failing to actually protect you and guarantee that the code
with operations that are potentially not thread-safe is @system. It looks to
me like it has to go one way or the other. Either make it all illegal, thus
requiring code that deals directly with synchronization mechanisms to use
casts and be @trusted - or make all of the read/write operations on shared
objects legal and just restrict it in that it can't be converted to or from
thread-local without a cast.

- Jonathan M Davis