shared - i need it to be useful

[Manu]

On Mon, Oct 22, 2018 at 3:30 AM Timon Gehr via Digitalmars-d
<digitalmars-d at puremagic.com> wrote:
>
> On 22.10.18 02:54, Manu wrote:
> > On Sun, Oct 21, 2018 at 5:40 PM Timon Gehr via Digitalmars-d
> > <digitalmars-d at puremagic.com> wrote:
> >>
> >> On 21.10.18 21:04, Manu wrote:
> >>> On Sun, Oct 21, 2018 at 12:00 PM Timon Gehr via Digitalmars-d
> >>> <digitalmars-d at puremagic.com> wrote:
> >>>>
> >>>> On 21.10.18 17:54, Nicholas Wilson wrote:
> >>>>>
> >>>>>> As soon as that is done, you've got a data race with the other
> >>>>>> existing unshared aliases.
> >>>>>
> >>>>> You're in @trusted code, that is the whole point. The onus is on the
> >>>>> programmer to make that correct, same with regular @safe/@trusted at system
> >>>>> code.
> >>>>
> >>>> Not all of the parties that participate in the data race are in @trusted
> >>>> code. The point of @trusted is modularity: you manually check @trusted
> >>>> code according to some set of restrictions and then you are sure that
> >>>> there is no memory corruption.
> >>>>
> >>>> Note that you are not allowed to look at any of the @safe code while
> >>>> checking your @trusted code. You will only see an opaque interface to
> >>>> the @safe code that you call and all you know is that all the @safe code
> >>>> type checks according to @safe rules. Note that there might be an
> >>>> arbitrary number of @safe functions and methods that you do not see.
> >>>>
> >>>> Think about it this way: you first write all the @trusted and @system
> >>>> code, and some evil guy who does not like you comes in after you looks
> >>>> at your code and writes all the @safe code. If there is any memory
> >>>> corruption, it will be your fault and you will face harsh consequences.
> >>>> Now, design the @safe type checking rules. It won't be MP!
> >>>>
> >>>> Note that there may well be a good way to get the good properties of MP
> >>>> without breaking the type system, but MP itself is not good because it
> >>>> breaks @safe.
> >>>
> >>> Show me. Nobody has been able to show that yet. I'd really like to know this.
> >>>
> >>
> >> I just did,
> >
> > There's no code there... just a presumption that the person who wrote
> > the @trusted code did not deliver the promise they made.
> > ...
>
> Yes, because there is no way to write @trusted code that holds its
> promise while actually doing something interesting in multiple threads
> if @safe code can implicitly convert from unshared to shared.

How do my examples prior fail to hold their promise?
struct S
{
  private int x;
  void method() shared @trusted { /* safely manipulate x */ }
}

How can you not trust that function? How can a 3rd party invalidate
that functions promise? `x` is inaccessible.
S can be thread-local or shared as much as you like, and it's safe,
and I don't know how a 3rd party could @safely undermine that?

> >> but if you really need to, give me a non-trivial piece of> correct multithreaded code that accesses some declared-unshared field
> >> from a shared method and I will show you how the evil guy would modify
> >> some @safe code in it and introduce race conditions. It needs to be your
> >> code, as otherwise you will just claim again that it is me who wrote bad
> >> @trusted code.
> >
> > You can pick on any of my prior code fragments. They've all been ignored so far.
> >
>
> I don't want "code fragments". Show me the real code.
>
> I manually browsed through posts now (thanks a lot) and found this
> implementation:
>
> struct Atomic(T){
>    void opUnary(string op : "++")() shared { atomicIncrement(&val); }
>    private T val;
> }
>
> This is @system code. There is no @safe or @trusted here, so I am
> ignoring it.
>
>
> Then I browsed some more, because I had nothing better to do, and I
> found this. I completed it so that it is actually compilable, except for
> the unsafe implicit conversion.
>
> Please read this code, and then carefully read the comments below it
> before you respond. I will totally ignore any of your answers that
> arrive in the next two hours.
>
> ---
> module borked;
>
> void atomicIncrement(int* p)@system{
>      import core.atomic;
>      atomicOp!("+=",int,int)(*cast(shared(int)*)p,1);
> }
>
> struct Atomic(T){
>      private T val;
>      void opUnary(string op : "++")() shared @trusted {
>          atomicIncrement(cast(T*)&val);
>      }
> }
> void main()@safe{
>      Atomic!int i;
>      auto a=&[i][0];// was: Atomic!int* a = &i;
>      import std.concurrency;
>      spawn((shared(Atomic!int)* a){ ++*a; }, a);
>      ++i.val; // race
> }
> ---
>
>
> Oh no! The author of the @trusted function (i.e. you) did not deliver on
> the promise they made!
>
> Now, before you go and tell me that I am stupid because I wrote bad
> code, consider the following:
>
> - It is perfectly @safe to access private members from the same module.
>
> - You may not blame the my @safe main function for the problem. It is
> @safe, so it cannot be blamed for UB. Any UB is the result of a bad
> @trusted function, a compiler bug, or hardware failure.
>
> - The only @trusted function in this module was written by you.
>
> You said that there is a third implementation somewhere. If that one
> actually works, I apologize and ask you to please paste it again in this
> subthread.

Last time I checked, main does not go in core.atomic. I've never added
any code to core.atomic.
These things wouldn't live in peoples code.

I understand that my proposal relies on trusting a very small number
of low-level implementations at the bottom of the stack... but they're
strongly encapsulated, live in libraries, and if there existed a world
where you COULD describe threadsafe interaction with shared; you
would.
@trusted user code, especially where it related to `shared` would be
terrifying, and you wouldn't do it.
I understand that the situation you present is technically possible,
but why would it happen in reality? I find the proposition of a @safe
threading stack to far far outweight that risk.
It's also possible that tech may be improved to assist with this
particular problem, I haven't tried to address it; I'm interested in
if the rules are sound.

One suggestion was to make `private int val` shared, then external
functions have no access... that helps mitigate this particular
mistake.

It's back to this 1:many thing. There's one place you can possibly
make this mistake (and it's probably maintained by an expert author),
and it should be well encapsulated in a core lib; whereas the current
`shared` requires that _end-users_ do unsafe casts all the time among
user code, and they're almost certainly not experts.
That's rigged totally backwards. The values are all wrong.

If my scheme is sound above this issue, then it's reasonable to focus
on ideas to reduce the odds of mistake for the one implementer of the
low-level tool.