shared - i need it to be useful

Steven Schveighoffer schveiguy at gmail.com
Wed Oct 17 13:25:28 UTC 2018


On 10/16/18 8:26 PM, Manu wrote:
> On Tue, Oct 16, 2018 at 2:20 PM Steven Schveighoffer via Digitalmars-d
> <digitalmars-d at puremagic.com> wrote:
>>
>> On 10/16/18 4:26 PM, Manu wrote:
>>> On Tue, Oct 16, 2018 at 11:30 AM Steven Schveighoffer via
>>> Digitalmars-d <digitalmars-d at puremagic.com> wrote:
>>>> int x;
>>>>
>>>> shared int *p = &x; // allow implicit conversion, currently error
>>>>
>>>> passToOtherThread(p);
>>>>
>>>> useHeavily(&x);
>>>
>>> What does this mean? It can't do anything... that's the whole point here.
>>> I think I'm struggling here with people bringing presumptions to the
>>> thread. You need to assume the rules I define in the OP for the
>>> experiment to work.
>>
>> OK, I wrote a whole big response to this, and I went and re-quoted the
>> above, and now I think I understand what the point of your statement is.
>>
>> I'll first say that if you don't want to allow implicit casting of
>> shared to mutable,
> 
> It's critical that this is not allowed. It's totally unreasonable to
> cast from shared to thread-local without synchronisation.

OK, so even with synchronization in the second thread when you cast, you 
still have a thread-local pointer in the originating thread WITHOUT 
synchronization.

> It's as bad as casting away const.

Of course! But shared has a different problem from const. Const allows 
the data to change through another reference, shared cannot allow 
changes without synchronization.

Changes without synchronization are *easy* with an unshared reference. 
Data can't be shared and unshared at the same time.

>> then you can't allow implicit casting from mutable to
>> shared. Because it's mutable, races can happen.
> 
> I don't follow...

You seem to be saying that shared data is unusable. But why the hell 
have it then? At some point it has to be usable. And the agreed-upon use 
is totally defeated if you also have some stray non-shared reference to it.

> 
>> There is in fact, no difference between:
>>
>> int *p;
>> shared int *p2 = p;
>> int *p3 = cast(int*)p2;
> 
> Totally illegal!! You casted away shared. That's as bad as casting away const.

But if you can't do anything with shared data, how do you use it?

> 
>> and this:
>>
>> int *p;
>> shared int *p2 = p;
>> int *p3 = p;
> 
> There's nothing wrong with this... I don't understand the point?

It's identical to the top one. You now have a new unshared reference to 
shared data. This is done WITHOUT any agreed-upon synchronization.

>> So really, the effort to prevent the reverse cast is defeated by
>> allowing the implicit cast.
> 
> Only the caller has the thread-local instance. You can take a
> thread-local pointer to a thread-local within the context of a single
> thread.
> So, it's perfectly valid for `p` and `p3` to exist in a single scope.
> `p2` is fine here too... and if that shared pointer were to escape to
> another thread, it wouldn't be a threat, because it's not readable or
> writable, and you can't make it back into a thread-local pointer
> without carefully/deliberately deployed machinery.

Huh? If shared data can never be used, why have it?

Pretend that p is not a pointer to an int, but a pointer to an UNSHARED 
type that has shared methods on it and unshared methods (for when you 
don't need any sync).

Now the shared methods will obey the sync, but the unshared ones won't. 
The result is races. I can't understand how you don't see that.

>> There is a reason we disallow assigning from mutable to immutable
>> without a cast. Yet, it is done in many cases, because you are sometimes
>> building an immutable object with mutable pieces, and want to cast the
>> final result.
> 
> I don't think analogy to immutable has a place in this discussion, or
> at least, I don't understand the relevance...
> I think the reasonable analogy is const.

No, immutable is more akin to shared because immutable and mutable are 
completely different. const can point at mutable or immutable data. 
shared can't be both shared and unshared. There's no comparison. Data is 
either shared or not shared, there is no middle ground. There is no 
equivalent of const to say "this data could be shared, or could be 
unshared".

>> In this case, it's ON YOU to make sure it's correct, and the traditional
>> mechanism for the compiler giving you the responsibility is to require a
>> cast.
> 
> I think what you're talking about are behaviours relating to casting
> shared *away*, and that's some next-level shit. Handling in that case
> is no different to the way it exists today. You must guarantee that
> the pointer you possess becomes thread-local before casting it to a
> thread-local pointer.
> In my application framework, I will never cast shared away under my
> proposed design. We don't have any such global locks.

OK, so how does shared data actually operate? Somewhere, the magic has 
to turn into real code. If not casting away shared, what do you suggest?

>> -----
>>
>> OK, so here is where I think I misunderstood your point. When you said a
>> lock-free queue would be unusable if it wasn't shared, I thought you
>> meant it would be unusable if we didn't allow the implicit cast. But I
>> realize now, you meant you should be able to use a lock-free queue
>> without it being actually shared anywhere.
> 
> Right, a lock-free queue is a threadsafe object, and it's methods work
> whether the queue is shared or not.
> The methods are attributed shared because they can be called on shared
> instances... but they can ALSO be called from a thread-local instance,
> and under my suggested promotion rules, it's fine for the this-pointer
> to promote to shared to make the call.

It's fine in terms of a specific object we are talking about, with 
pre-determined agreements as to whether the data will be passed to other 
threads. But the compiler has no idea about these agreements. It can't 
logically determine that this is safe without you telling it, hence the 
requirement for casting.

>> What I say to this is that it doesn't need to be usable. I don't care to
>> use a lock-free queue in a thread-local capacity. I'll just use a normal
>> queue, which is easy to implement, and doesn't have to worry about race
>> conditions or using atomics. A lock free queue is a special thing, very
>> difficult to get right, and only really necessary if you are going to
>> share it. And used for performance reasons!
> 
> I'm more interested in the object that has that lock-free queue as a
> member... it is probably a mostly thread-local object, but may have a
> couple of shared methods.

I get that, I think the shared methods just need to have unshared 
versions for the case when the queue is fully thread-local.

> I have a whole lot of objects which have 3 tiers of API access; the
> thread-local part, the threadsafe part, and the const part. Just as a
> mutable instance can call a const method, there's no reason a
> thread-local instance can't call a threadsafe method.

If you call a const method, it can squirrel away a const pointer to the 
otherwise mutable data, which is then usable later (and safe to do so).

The same cannot be said for a shared pointer. That can be then easily 
moved to another thread, WITHOUT the expectation that it was. And in 
that case, the now thread-local queue is actually shared between 
threads, and calling the thread-local API will cause races.

I didn't respond to the rest of the comments, because they were simply 
another form of "shared is similar to const", which is not true.

threadsafe use of shared data depends on ALL threads using those same 
mechanisms. If one uses simple thread-local use, then it all falls apart.

-Steve


More information about the Digitalmars-d mailing list