DIP 1024--Shared Atomics--Community Review Round 1

[Jonathan M Davis]

On Saturday, October 12, 2019 4:22:38 AM MDT Ola Fosheim Grøstad via 
Digitalmars-d wrote:
> On Saturday, 12 October 2019 at 09:45:41 UTC, Jonathan M Davis
>
> wrote:
> > It's the same as with any @trusted code. It's up to the
> > programmer to ensure that what the code is doing is actually
> > @safe, and if the code isn't able to provide an API that is
> > @safe, then it shouldn't be @trusted.
>
> But didn't Walter in another thread some time ago say that he
> would add Rust-like linear typing for memory management?
>
> If this is the goal, shouldn't you also use linear typing for
> allowing more flexible @safe/@trusted APIs?

I don't know anything about what you mean by linear types (I really don't
know much about Rust). So, I can't say how that would come into play, but in
general, with @trusted, the programmer must verify that what the code is
doing is @safe and must provide an API that is @safe. If the programmer
can't guarantee that what the code is doing is @safe, then they shouldn't be
marking it is @trusted. Any operations that the compiler can guarantee as
@safe are already considered @safe. By definition, once @system is
involved, it's because the compiler can't guarantee that the code is @safe,
and @trusted is the programmer's tool for telling the compiler that a
particular piece of code is actually @safe even though the compiler couldn't
verify that.

In the case of shared, in general, it's not thread-safe to read or write to
such a variable without either using atomics or some other form of thread
synchronization that is currently beyond the ability of the compiler to make
guarantees about and will likely always be beyond the ability of the
compiler to make guarantees about except maybe in fairly restricted
circumstances. The type system separates thread-local from shared, but it
doesn't have any concept of ownership and simply doesn't have the kind of
information required to figure out whether something is thread-safe. The
closest that anyone has proposed is TDPL's synchronized classes, and even
those could only safely remove the outer layer of shared - and even that
required locking down synchronized classes pretty thoroughly.

That's why it doesn't make sense to allow reading from or writing to shared
variables, and it has to be up to the programmer to deal with the
synchronization primitives and temporarily casting to thread-local to
operate on the shared data while it's protected by those synchronization
primitives. The casts automatically make such code @system as it should be,
and then the programmer is going to have to guarantee the thread-safety just
like they'd do in a language like Java or C++. The difference is that D's
type system helps the programmer, whereas languages like Java or C++ leave
it entirely up to the programmer to get it right. The type system segregates
the data that's shared, and the code that does all of the tricky threading
stuff is @system, requiring the programmer to verify that piece of code and
mark it as @trusted to then be used by the @safe portions of the program.
Yes, the casts make the code more verbose than it would be in C++, but other
than that, you're basically doing exactly the same thing that you'd be doing
in C++.

By no means does that mean that any of the threading stuff is easy, but as
long as the type system cannot make guarantees about thread-safety,
segregating the code that deals with the threading and requiring that the
programmer verify its correctness in order to have it work with @safe code
is the best that we can do. Regardless, even if we're able to later come up
with improvements that allow shared to be implicitly removed under some set
of circumstances (and thus have such code be @safe), shared in general still
needs to be locked down in a manner similar to what this DIP is proposing.
Any improvements that we later come up with would then just be laid on top
of that.

- Jonathan M Davis