DIP 1024--Shared Atomics--Community Review Round 1

[Jonathan M Davis]

On Saturday, October 12, 2019 5:26:42 AM MDT Ola Fosheim Grøstad via 
Digitalmars-d wrote:
> I hope people try to sketch out what such libraries will look
> like before the semantics are decided on. Maybe an experimental
> branch would be appropriate.

The fact that we have thread-local in the language requires that the type
system then have the concept of shared. So, there should be no question
about that. As part of that, converting between thread-local and shared has
to require casts (otherwise, the compiler can't guarantee that the
thread-local stuff is actually thread-local). Those casts then have to be
@system, because they potentially violate the compiler's guarantees about
thread-local and shared data.

The question that then comes is what kinds of operations should be allowed
on shared, and there are basically two approaches that can be taken:

1. Not allow read-write operations that aren't guaranteed to be atomic.

2. Ignore thread-safety and just let people do whatever they want to with
shared other than implicitly convert between shared and thread-local (since
that would violate the compiler guarantees about which data is shared and
which is thread-local).

#1 allows us to segregate all of the code that has to deal with threading
stuff in a manner that requires the programmer to get involved, because
operating on shared in a way that isn't guarantee to be atomic then becomes
@system. So, it becomes far easier to locate the pieces of code that need to
be verified as thread-safe by the programmer, and you get actual compiler
guarantees that any @safe code operating on shared variables doesn't have
any threading problems. Only the @trusted code has to be examined.

Unlike #1, #2 would mean that you couldn't know that code operating on
shared was thread-safe. You wouldn't have to do as much casting, because
basic operations would work on shared data, but you'd then have to examine
all code involving shared to determine whether what it was doing was
thread-safe. And because many types are designed to be thread-local rather
than shared, you'd still have to cast to thread-local in many cases, and
that code would still have to be @system, because the programmer is
potentially violating the compiler guarantees about thread-local when
casting to or from shared.

So, while #2 would reduce the amount of casting required, it would do a far
worse job segregating code that needed to be examined when dealing with
threading issues, and it doesn't really remove the casting and the like that
folks like to complain about with regards to shared. We were locked into
that once we had shared as part of the type system. Languages like C++ can
only get away with not having it because they ignore the issue. That means
that in such languages, you can just operate on shared stuff as if it were
thread-local, but the programmer then also has to worry about threading
issues throughout the entire program, because the type system isn't helping
them at all. Good programming practices help make that problem more
manageable, but by having shared as part of the type system in D, the
language and the compiler make the problem much more manageable by
segregating the threading code.

Right now, we basically have #2. What this DIP is trying to do is move us to
#1. How libraries should work is exactly the same in either case. It's just
that with #1, the places where you operate on shared data in a manner which
isn't guaranteed to be atomic, the compiler prevents you from doing it
unless you  use core.atomic or have @system code with casts. Even if we have
#2 and thus no such compiler errors, the code should still have been doing
what #1 would have required, since if it doesn't, then it isn't thread-safe.

It may be that if we can come up with ways for the compiler to know that
it's thread-safe to implicitly remove shared in a piece of code, libraries
won't have to have as much @trusted code, but the low level semantics would
still need to be the same, and any code that wasn't using whatever idiom or
feature allowed the compiler to implicitly remove shared in a piece of code
would still need to do what it would need to do right now, which is what it
would need to do if this DIP were implemented.

All this DIP really does is force code that the compiler can't guarantee is
atomic to use either core.atomic or be @system/@trusted. The idioms and
design patterns involved with thread-safe code are the same either way.

- Jonathan M Davis