shared - i need it to be useful

[Manu]

On Tue, Oct 16, 2018 at 10:45 PM Walter Bright via Digitalmars-d
<digitalmars-d at puremagic.com> wrote:
>
> On 10/15/2018 11:46 AM, Manu wrote:
> > [...]
>
> Shared has one incredibly valuable feature - it allows you, the programmer, to
> identify data that can be accessed by multiple threads. There are so many ways
> that data can be shared, the only way to comprehend what is going on is to build
> a wall around shared data.
>
> (The exception to this is immutable data. Immutable data does not need
> synchronization, so there is no need to distinguish between shared and unshared
> immutable data.)
>
> This falls completely apart if shared and unshared data can be aliased.

What does it mean 'aliased' precisely? I'm trying to prevent that in
practical terms.
If you can't read/write to a shared object... is it actually aliased?
Or do you just have a fancy int that happens to look like a pointer?

> When Andrei and I came up with the rules for:
>
>     mutable
>     const
>     shared
>     const shared
>     immutable
>
> and which can be implicitly converted to what, so far nobody has found a fault
> in those rules.

Oh bollocks... everyone has been complaining about this for at least
the 10 years I've been here!
Shared is uninteresting and mostly useless as spec-ed, everyone knows this.
Interaction with shared via barely-controlled blunt casting in
@trusted blocks is feeble and boring. It doesn't really give us
anything in practice that we don't have in C++. It's an uninspired
design.

I am working against a design where *everything* may be shared, and
it's a very interesting design space. I have no doubt it represents
the future of my field.
We are limited by what the type system can express on this front. I'm
trying to explore opportunities to make shared interesting and
useful... and I'm mostly being met with prejudice here.
Eject what you think you know about shared from your brain, and try
and take a fresh look from the perspective I'm proposing.
I think you'll find that the current wisdom and interactions with
shared are actually preserved in practice, but new opportunities
become available. If not, I want to understand how the design is
broken, so I can continue to iterate.

If D offered a real advantage here over C++, this would really
represent a strong attracting force.

> Timon Gehr has done a good job showing that they still stand
> unbreached.

His last comment was applied to a different proposal.
His only comment on this thread wasn't in response to the proposal in
this thread.
If you nominate Timon as your proxy, then he needs to destroy my
proposal, or at least comment on it, rather than make some prejudiced
comment generally.

> So, how can mutable data become shared, and vice versa? I've never found a way
> to do that that is proveably safe. Therefore, it's up to the programmer.

I'm proposing a way, and that is:
1. the rule must be applied that shared object can not be read or written
2. attributing a method shared is a statement and a promise that the
method is threadsafe

The rest just follows naturally.

> If a programmer does the following:
>
>      T* p;
>      shared(T)* sp;
>
>      p = cast(T*)sp;           (1)
>      sp = cast(shared(T)*) p;  (2)
>
> those casts will be rejected in @safe code. They'll have to be in @trusted or
> @system code. It will be up to the programmer to ensure (via mutexes, locks,
> uniqueness, etc.) that:
>
>      (1) sp is a unique pointer and that ownership of the data is thus
> transferred for the lifetime of p
>
>      (2) p is a unique pointer and that ownership of the data is thus
> transferred for the lifetime of sp

'Transfer' is a nice idea, but it's not what's happening here. That's
not what these operations do.
Transferring ownership is a job for move semantics, and that's nowhere in sight.

Also, there's no need to 'transfer' a this pointer to shared to call a
threadsafe method. You can always just call it safely.

> A sensible program should try to minimize the places where there are these
> "gates" through the wall, as those gates will be where you'll be looking when
> threading bugs appear.

I agree, that's my goal here. Shared right now is an unsafe mess; I
want to apply aggressive restriction so that you can't do unsafe
operations without explicit casts.

> Allowing implicit aliasing between shared and unshared data means the entire
> program needs to reviewed for threading bugs, rather than just the shared parts.

That's backwards, I'm suggesting conversion TO shared, such that you
are able to call threadsafe methods on thread-local things. There's no
possible harm in that.
I'm trying to eliminate aliasing by removing read/write access.
The result is, the only thing you are able to do with a shared
reference are explicitly threadsafe things. Any other operation is
inaccessible without deliberate blunt casting, and the rules
surrounding that are the same as always; those that you listed above.

> One might as well just not have shared as a language feature at all. D being a
> systems programming language, one can certainly write code that way, just like
> one does with C, and with all of the threading bugs one gets doing that.

And this is a silly claim that's been made before in this thread. It
doesn't make sense, and I kinda have to presume here you have either
not read or do not understand my proposal...
Are you suggesting that I hate shared, and I want to make it
worthless, therefore I'm trying to change the rules to eliminate it
from practical existence?
I promise you, I'd *really* like it if shared were a _useful_ language
feature. The suggestion that I intend to undermine it such that it not
be a language feature at all is obviously ridiculous.


> ----------
>
> Currently, the compiler allows you to read/write shared data without any locks
> or atomics.

Right, this is the core of the problem, and it must change for shared
to model anything useful.

> I.e. the compiler doesn't attempt to insert any synchronization on
> your behalf. Given all the ways synchronization can be done, it just seems
> presumptive and restrictive to impose a particular scheme. It's pretty much left
> up to the user.

But it's a useless model; it's just a boring sentinel tag with no
functional application. The shared rules don't model any useful
behaviour.
I'm trying to express how shared could model a useful behaviour, and
coincidentally be *more* conceptually safe, and intuitively model and
communicate appropriate interactions with threadsafe objects.

> Which is why I recommend minimizing the amount of code that actually has to deal
> with shared data.

Well, the future is SMP. 100% of our code potentially deals with
shared data, and it would be idea to do so in a typesafe manner.
We can't model this with C++. We can't model it in D either, because
D's `shared` doesn't model anything... but with just the one change
that shared can't read or write, we would have the foundation of
something useful.
Being able to read/write to shared objects is weird; it's completely
unsafe in every event. It's a useless access right, and changing it
such that shared can not read/write would improve the model in such a
way that it starts to become useful.