shared - i need it to be useful

[Simen Kjærås]

On Thursday, 18 October 2018 at 16:31:02 UTC, Stanislav Blinov 
wrote:
> You contradict yourself and don't even notice it. Per your 
> rules, the way to open that locked box is have shared methods 
> that access data via casting. Also per your rules, there is 
> absolutely no way for the programmer to control whether they're 
> actually sharing the data. Therefore, some API can steal a 
> shared reference without your approval, use that with your 
> "safe" shared methods, while you're continuing to threat your 
> data as not shared.

Yes, and that's fine. Because it's thread-safe, remember?


> You and Manu both seem to think that methods allow you to 
> "define a thread-safe interface".
>
> struct S {
>     void foo() shared;
> }
>
> Per your rules, S.foo is thread-safe. It is here that I remind 
> you, *again*, what S.foo actually looks like, given made-up 
> easy-to-read mangling:
>
> void struct_S_foo(ref shared S);
>
> And yet, for some reason, you think that these are not 
> thread-safe:
>
> void foo(shared int*);
> void bar(ref shared int);

Again, no. No. No, no, no, no, no. We have not said that, we are 
not saying that, and we will not say that. Because it's not true, 
and I pointed out exactly this in a previous post. I have no idea 
where you got this idea, and I hope we can excise it. There is 
absolutely nothing wrong with void foo(shared int*). (apart from 
the fact it can't safely do anything)

For clarity: the interface of a type is any method, function, 
delegate or otherwise that may affect its internals. That means 
any free function in the same module, and any non-private members.


> Your implementation of 'twaddle' is *unsafe*, because the 
> compiler doesn't know that 'payload' is shared. For example, 
> when inlining, it may reorder the calls in it and cause races 
> or other UB. At least one of the reasons behind `shared` *was* 
> to serve as compiler barrier.

Ah, now this is a good point - thanks! That does seem like it's a 
harder problem than has come up thus far.


>> Alright, so I have this shared object that I can't read from, 
>> and can't write to. It has no public shared members. What can 
>> I do with it? I can pass it to other guys, who also can't do 
>> anything with it. Are there other options?
>
> It can have any number of public shared "members" per UFCS. The 
> fact that you forget is that there's no difference between a 
> method and a free function, other than syntax sugar. Well, OK, 
> there's guaranteed private access for methods, but same is true 
> for module members.

This again? See point 2, above. I hope we can stop this silliness 
soon.


>>>>> The rest just follows naturally.
>>>
>>> Nothing follows naturally. The proposal doesn't talk at all 
>>> about the fact that you can't have "methods" on primitives,
>>
>> You can't have thread-safe methods operating directly on 
>> primitives, because they already present a non-thread-safe 
>> interface. This is true. This follows naturally from the rules.
>
> Everything in D already presents a non-threadsafe interface. 
> Things that you advocate included.
>
> struct S {
>     void foo() shared;
> }
>
> That is not threadsafe. This *sort of* is:
>
> struct S {
>     @disable this(this);
>     @disable void opAssign(S);
>
>     void foo() shared;
> }
>
> ...except not quite still. Now, if the compiler generated above 
> in the presence of any `shared` members or methods, then we 
> could begin talking about it being threadsafe. But that part is 
> mysteriously missing from Manu's proposal, even though I keep 
> reminding of this in what feels like every third post or so 
> (I'm probably grossly exaggerating).

Again, this is good stuff. This is an actual example of what can 
go wrong. Thanks!


>>> that you can't distinguish between shared and unshared data 
>>> if that proposal is realized,
>
>> And you can't do that currently either. Just like today, 
>> shared(T) means the T may or may not be shared with other 
>> thread. Nothing more, nothing less.
>
> I don't think it means what you think it means. "May or may not 
> be shared with other thread" means "you MUST treat it as if 
> it's shared with other thread". That's it.

Yup, hence 'shared' on a method meaning 'thread-safe'. So it's 
fine. I think this has been mentioned before.


> That's why automatic conversion doesn't make *any* sense, and 
> that's why compiler error on attempting to pass over mutable as 
> shared makes *perfect* sense.

No. Because shared access is thread-safe.


>>> that you absolutely destroy D's TLS-by-default treatment...
>> I'm unsure what you mean by this.
>
> You lose the ability to distinguish thread-local and shared 
> data.

And when is this a problem? Again, anything that has shared 
access to something is incapable of doing anything 
non-thread-safe to it.


>>> Functions that you must not be allowed to write per this same 
>>> proposal. How quaint.
>>
>> What? Which functions can't I write?
>
> Uh-huh, only due to some weird convention that "methods" are 
> somehow safer than free functions. Which they're not.

No. Again, point 2. Nobody says this.


>> Yup, this is correct. But wrap it in a struct, like e.g. 
>> Atomic!int, and everything's hunky-dory.
>
> So again,
>
> void atomicInc(shared int*); // is "not safe", but
> void struct_Atomic_int_opUnary_plus_plus(ref shared Atomic); // 
> is "safe"

No, void atomicInc(shared int*) is perfectly safe, as long as it 
doesn't cast away shared. Again, the problem is int already has a 
non-thread-safe interface, which Atomic!int doesn't. And once 
more, for clarity, this interface includes any function that has 
access to its private members, free function, method, delegate 
return value from a function/method, what have you. Since D's 
unit of encapsulation is the module, this has to be the case. For 
int, the members of that interface include all operators. For 
pointers, it includes deref and pointer arithmetic. For arrays 
indexing, slicing, access to .ptr, etc. None of these lists are 
necessarily complete.


>> I have no idea where I or Manu have said you can't make 
>> functions that take shared(T)*.
>
> Because you keep saying they're unsafe and that you should wrap 
> them up in a struct for no other reason than just "because 
> methods are kosher".

Again, point 2. I think we have been remiss in the explanation of 
what we consider the interface.


>> Let's say it together: for a type to be thread-safe, all of 
>> its public members must be written in a thread-safe way.
>
> It's shared private parts also must be written in a thread-safe 
> way. Yes, they're private, but they still may be shared. 
> Welcome to the communism of multithreading.

For the public members to be thread-safe, yes, the private parts 
must be thread-safe. That's always the case. If I'm going to 
build a skyscraper, I will not make the foundation out of 
cardboard.


Now, Two very good points came up in this post, and I think it's 
worth stating them again, because they do present possible issues 
with MP:

1) How does MP deal with reorderings in non-shared methods?

I don't know. I'd hide behind 'that's for the type implementor to 
handle', but it's a subtle enough problem that I'm not happy with 
that answer.


2) What about default members like opAssign and postblit?

The obvious solution is for the compiler to not generate these 
when a type has a shared method or is taken as shared by a free 
function in the same module. I don't like the latter part of 
that, but it should work.

--
   Simen