shared - i need it to be useful

[Manu]

On Sun., 21 Oct. 2018, 2:55 am Walter Bright via Digitalmars-d,
<digitalmars-d at puremagic.com> wrote:
>
> On 10/20/2018 11:24 AM, Manu wrote:
> > This is an unfair dismissal.
>
> It has nothing at all to do with fairness. It is about what the type system
> guarantees in @safe code. To repeat, the current type system guarantees in @safe
> code that T* and shared(T)* do not point to the same memory location.
>
> Does your proposal maintain that or not? It's a binary question.

By the definition Nick pulled from Wikipedia and posted for you a few
posts back, yes, my proposal satisfies Wikipedia's definition of no
aliasing. I understand that property is critical, and I have carefully
designed for it.

> > I'm not sure you've understood the proposal.
> > This is the reason for the implicit conversion. It provides safe
> > transition.
>
> I don't see any way to make an implicit T* to shared(T)* safe, or vice versa.
> The T* code can create more aliases that the conversion doesn't know about, and
> the shared(T)* code can hand out aliases to other threads. So it all falls to
> pieces.

T* can't make additional T* aliases on other threads; there can only
be one thread with T*.
shared(T)* can not make a T*.
shared(T)* has no read or write access, so it's not an alias of T* by
Wikipedia's definition.

Only threadsafe functions can do anything to T.
The leap of faith is; some @trusted utility functions at the bottom of
the shared stack makes a promise that it is threadsafe, and must
deliver that promise.
I don't think this is unreasonable; this is the nature of @trusted
functions, they make a promise, and they must keep it.
If the trusted function does not lie, then the chain of trust holds
upwards through the stack.

The are very few such trusted functions in practise. Like, similar to
the number of digits you have.

> Using a 'scope' qualifier won't work, because 'scope' isn't transitive,
> while shared is, i.e. U** and shared(U*)*.

I don't think I depend on scope in any way.
That was an earlier revision of thinking in an older thread.

>  > I'm not sure how to clarify it, what can I give you?
>
> Write a piece of code that does such an implicit conversion that you argue is
> @safe. Make the code as small as possible. Your example:
>
>  > int* a;
>  > shared(int)* b = a;
>
> This is not safe.
>
> ---- Manu's Proposal ---
> @safe:
> int i;
> int* a = &i;
> StartNewThread(a); // Compiles! Coder has no idea!
>
> ... in the new thread ...
> void StartOfNewThread(shared(int)* b) {
>
>      ... we have two threads accessing 'i',
>      one thinks it is shared, the other unshared,
>      and StartOfNewThread() has no idea and anyone
>      writing code for StartOfNewThread() has no way
>      to know anything is wrong ...
>
>      lockedIncrement(b);  // Data Race!
> }

This program doesn't compile. You receive an error because it is not safe.
The function is `lockedIncrement(int*)`. It can't receive a shared
argument; the function is not threadsafe by my definition.
You have written a program that produces the expected error that
alerts you that you have tried to do un- at safe and make a race.

Stanislav produced this same program, and I responded with the correct
program a few posts back.
I'll repeat it here; the @safe program to model this interaction is:

@safe:

// function is NOT threadsafe by my definition, can not be called on
shared arguments
void atomicIncrement(int*);

struct Atomic(T)
{
  // encapsulare the unsafe data so it's inaccessible by any unsafe means
  private T val;

  // perform the unsafe cast in a trusted function
  // we are able to assure a valid calling context by encapsulating
the data above
  void opUnary(string op : "++")() shared @trusted {
atomicIncrement(cast(T*)&val); }
}

Atomic!int i;
Atomic!int* a = &i;
StartNewThread(a); // Compiles, of course!
++i; // no race

... in the new thread ...
void StartOfNewThread(shared(Atomic!int)* b) {
  ... we have two threads accessing 'i', one has thread-local access,
this one has a restricted shared access.
  here, we have a shared instance, so we can only access `b` via
threadsafe functions.
  as such, we can manipulate `b` without fear.
  ++i; // no race!
}


> Your proposal means that the person writing the lockedIncrement(), which is a
> perfectly reasonable thing to do, simply cannot write it in a way that has a
> @safe interface

Correct, the rules of my proposal apply to lockedIncrement(). They
apply to `shared` generally.
lockedIncrement() is not a threadsafe function. You can't call it on a
shared instance, because `int`s API (ie, all intrinsic operations) are
not threadsafe.
lockedIncrement() can't promise threadsafe access to `shared(int)*`,
so the argument is not shared.

Your program made the correct compile error about doing unsafety, but
the location of the compile error is different under my proposal;
complexity is worn by the shared library author, rather than every
calling user ever.
I think my proposal places the complexity in the right location.
`shared` is intrinsically dangerous; it's not reasonable to ask every
user that ever calls a shared API to write unsafe code when when
calling. That's just plain bad design.

> because the person writing the lockedIncrement() library
> function has no way to know that the data it receives is actually unshared data.

The author of `shared` tooling must assure a valid context such that
its threadsafety promises are true. Atomic(T) does that with a private
member.
The @safe way to interact with atomics is to use the Atomic utility
type I showed above.
That is one such @trusted tool that I talk about as being "at the
bottom of the stack".
It is probably joined by a mutex/semaphore, and some
containers/queues. That is probably all the things, and other things
would be @safe compositions of those tools.

> I.e. @trusted code is obliged to proved a safe interface. Your proposal makes
> that impossible because the compiler would allow unshared data to be implicitly
> typed as shared.

What? No.
Please, try and understand my proposal...