shared - i need it to be useful

[Manu]

On Tue, Oct 16, 2018 at 2:20 PM Steven Schveighoffer via Digitalmars-d
<digitalmars-d at puremagic.com> wrote:
>
> On 10/16/18 4:26 PM, Manu wrote:
> > On Tue, Oct 16, 2018 at 11:30 AM Steven Schveighoffer via
> > Digitalmars-d <digitalmars-d at puremagic.com> wrote:
> >>
> >> On 10/16/18 2:10 PM, Manu wrote:
> >>> On Tue, Oct 16, 2018 at 6:35 AM Steven Schveighoffer via Digitalmars-d
> >>> <digitalmars-d at puremagic.com> wrote:
> >>>>
> >>>> On 10/16/18 9:25 AM, Steven Schveighoffer wrote:
> >>>>> On 10/15/18 2:46 PM, Manu wrote:
> >>>>
> >>>>>>>   From there, it opens up another critical opportunity; T* -> shared(T)*
> >>>>>> promotion.
> >>>>>> Const would be useless without T* -> const(T)* promotion. Shared
> >>>>>> suffers a similar problem.
> >>>>>> If you write a lock-free queue for instance, and all the methods are
> >>>>>> `shared` (ie, threadsafe), then under the current rules, you can't
> >>>>>> interact with the object when it's not shared, and that's fairly
> >>>>>> useless.
> >>>>>>
> >>>>
> >>>> Oh, I didn't see this part. Completely agree with Timon on this, no
> >>>> implicit conversions should be allowed.
> >>>
> >>> Why?
> >>
> >> int x;
> >>
> >> shared int *p = &x; // allow implicit conversion, currently error
> >>
> >> passToOtherThread(p);
> >>
> >> useHeavily(&x);
> >
> > What does this mean? It can't do anything... that's the whole point here.
> > I think I'm struggling here with people bringing presumptions to the
> > thread. You need to assume the rules I define in the OP for the
> > experiment to work.
>
> OK, I wrote a whole big response to this, and I went and re-quoted the
> above, and now I think I understand what the point of your statement is.
>
> I'll first say that if you don't want to allow implicit casting of
> shared to mutable,

It's critical that this is not allowed. It's totally unreasonable to
cast from shared to thread-local without synchronisation.
It's as bad as casting away const.

> then you can't allow implicit casting from mutable to
> shared. Because it's mutable, races can happen.

I don't follow...

> There is in fact, no difference between:
>
> int *p;
> shared int *p2 = p;
> int *p3 = cast(int*)p2;

Totally illegal!! You casted away shared. That's as bad as casting away const.

> and this:
>
> int *p;
> shared int *p2 = p;
> int *p3 = p;

There's nothing wrong with this... I don't understand the point?

> So really, the effort to prevent the reverse cast is defeated by
> allowing the implicit cast.

Only the caller has the thread-local instance. You can take a
thread-local pointer to a thread-local within the context of a single
thread.
So, it's perfectly valid for `p` and `p3` to exist in a single scope.
`p2` is fine here too... and if that shared pointer were to escape to
another thread, it wouldn't be a threat, because it's not readable or
writable, and you can't make it back into a thread-local pointer
without carefully/deliberately deployed machinery.

> There is a reason we disallow assigning from mutable to immutable
> without a cast. Yet, it is done in many cases, because you are sometimes
> building an immutable object with mutable pieces, and want to cast the
> final result.

I don't think analogy to immutable has a place in this discussion, or
at least, I don't understand the relevance...
I think the reasonable analogy is const.

> In this case, it's ON YOU to make sure it's correct, and the traditional
> mechanism for the compiler giving you the responsibility is to require a
> cast.

I think what you're talking about are behaviours relating to casting
shared *away*, and that's some next-level shit. Handling in that case
is no different to the way it exists today. You must guarantee that
the pointer you possess becomes thread-local before casting it to a
thread-local pointer.
In my application framework, I will never cast shared away under my
proposed design. We don't have any such global locks.

> -----
>
> OK, so here is where I think I misunderstood your point. When you said a
> lock-free queue would be unusable if it wasn't shared, I thought you
> meant it would be unusable if we didn't allow the implicit cast. But I
> realize now, you meant you should be able to use a lock-free queue
> without it being actually shared anywhere.

Right, a lock-free queue is a threadsafe object, and it's methods work
whether the queue is shared or not.
The methods are attributed shared because they can be called on shared
instances... but they can ALSO be called from a thread-local instance,
and under my suggested promotion rules, it's fine for the this-pointer
to promote to shared to make the call.

> What I say to this is that it doesn't need to be usable. I don't care to
> use a lock-free queue in a thread-local capacity. I'll just use a normal
> queue, which is easy to implement, and doesn't have to worry about race
> conditions or using atomics. A lock free queue is a special thing, very
> difficult to get right, and only really necessary if you are going to
> share it. And used for performance reasons!

I'm more interested in the object that has that lock-free queue as a
member... it is probably a mostly thread-local object, but may have a
couple of shared methods.
I have a whole lot of objects which have 3 tiers of API access; the
thread-local part, the threadsafe part, and the const part. Just as a
mutable instance can call a const method, there's no reason a
thread-local instance can't call a threadsafe method.

You can ask 'why', and I can't give any more satisfactory answer than
"you can call a const method from a mutable object", this is
commonsense, and should be possible.
We want to call shared methods from threadlocal objects all the time.
It's possible, and it's safe... there's no reason we shouldn't be able
to.

> Why would I want to incur performance penalties when using a lock-free
> queue in an unshared mode? I would actually expect 2 separate
> implementations of the primitives, one for shared one for unshared.

Overloading for shared and unshared is possible, and may be desirable
in many cases.
There are also many cases where the code duplication and tech-debt
does not carry its weight. It should not be required, because it's not
technically required.

> What about primitives that would be implemented the same? In that case,
> the shared method becomes:
>
> auto method() { return (cast(Queue*)&this).method; }
>
> Is this "unusable"? Without a way to say, you can call this on shared or
> unshared instances, then we need to do it this way.

I don't understand here...?
I'm saying, we *don't* need a way to say "you can call on shared or
unshared instances*, because that's always safe to do. If it wasn't
safe to call a const method with a mutable instance, something is
terrible wrong; same applies here, it is always safe to call a
threadsafe method with a thread-local instance. If that's not true,
the method is objectively not threadsafe.

> But I would trust the queue to handle this properly depending on whether
> it was typed shared or not.

The queue is less interesting than the object that aggregates the queue.