Possible to pass a member function to spawn?

[Artur Skawina]

On 02/09/12 02:46, Timon Gehr wrote:
> On 02/09/2012 12:50 AM, Artur Skawina wrote:
>> On 02/08/12 22:47, Timon Gehr wrote:
>>> On 02/08/2012 10:26 PM, Artur Skawina wrote:
> ...
>>>
>>>> If we effectively passed ownership of our unique instance to another context, 'x' can no longer
>>>> be "unique". If it were to mutate to the target type, then leaving it
>>>> accessible from the current context should be reasonably safe.
>>>
>>> The idea was that spawn could take unique class references and pass ownership to a different thread -- eliminating the need to cast to and from shared.
>>
>> I'll rephrase what i said in that d.learn post; *all* I'm suggesting is this:
>>
>> a) Any result of an expression that the compiler can determine is unique is
>>     internally flagged as such. This means eg array concatenation or new-expressions.
>>     Just a simple bitflag set, in addition to the the stored "real" type.
>> b) Any access to the data clears this flag (with just a few exceptions, below).
>> c) If the expression needs to be implicitly converted to another type *and*
>>     no implicit cast is possible *and* the "unique" flag is set - then additional
>>     safe conversions are tried, and if one succeeds, the "unique" flag gets cleared
>>     and the type gets modified to the that of the target.
>>
>> This allows for things which are 100% safe, but currently prohibited by the
>> compiler and require explicit casts.
> 
> Your 'simple bit flag' already necessitates a flow analysis, and it does not solve the problem Manu describes. Why not make it powerful enough to be useful?

No, it does not really require *extra* flow analysis - you only need to clear the
flag on access/lookup. This makes it relatively cheap and while this approach has
its limits, it solves ~80% of the problem; it *does* let you write code without
having to use explicit casts where there should be none.

Manu's problem is the need for casts for sharing/unsharing, right? This *is* a
problem, and my approach allows for an implicit unique=>shared conversion.
Could you show an example message-passing function that uses your explicit
unique class together with sender code and receiver signature? Maybe I'm
missing something. Just a one-line spawn() that calls the receiver with an
argument provided by the sender. What i'm interested in is: would your new
"unique" allow implementing it *without* explicit unsharing?

>> If I understood you right, you'd like (b) to be much less restrictive, which i
>> think complicates things too much. Some (b)-restrictions for cases that always
>> are cheap to discover /can/ be removed, but this needs to be determined on a case-
>> -by-case basis.
> 
> Everything that can modularly be shown to work should work. Of course the analysis will still be conservative.

It has to be exact. ie it has to always predictably work for every case and every
compiler.

>> Eg. I think any leaked refs to the data don't qualify (IOW any
>> assignment, even if only indirectly via this expression, needs to clear the flag).
> 
> If the leaked ref can be shown to be dead, there is no problem. (this is simple!)

It's not the trivial dead refs examples that are the problem.

>> One thing the (b) probably /has/ to allow is storing the result in an "auto"
>> variable. But making another copy should clear the flag.
>>
>> While i originally needed this for immutable/const/mutable, it would also work
>> for shared. If spawn() takes a "shared" argument, passing it a "unique" one
>> will work too. And i'm not even convinced the ref needs to disappear from the
>> current context (obviously accessing the now shared data has to treat it as such
>> - but this is not different from what we had before, when using explicit casts;
>> in fact now it's marked as "shared" so it should be safer.)
> 
> Then you still have to cast away shared in the receiver thread. As I said before, the idea is that you can send unique objects, not that they implicitly convert to shared and are then sent.

The receiver should *not* be passed unique "shared" objects. This is why I'm
wondering about your spawn() implementation above.

>> So the question is: does having an explicit "unique" storage class improve
>> things further?
> 
> Yes. Then the concept persists function boundaries. You cannot pass an unshared object to another thread if there is no explicit unique storage class.

Well, I know what you meant, but sometimes you can - eg immutable or shared.
The interesting case is *unsharing* the object.
And, yes, some kind of "unique" could allow for this, i'm just not yet
convinced your proposal isn't prohibitively expensive, both for the compiler
and user (by making things too complicated to use).

>> Other than using it [1] to mark things as unique that the compiler can't figure
>> out by itself.
>>
>> [1] I'm using "unique", but if it were to become a keyword it should be
>>      "uniq" or "@uniq", for the same reasons as "int", "auto" or "ref".
> 
> "immutable"

"synchronized". The fact that the language has flaws does not mean we should
add more. :)

>>>> And if it can work for "auto" - is an explicit "unique" for 'x' needed?
>>>> This example *could* work, but the interesting cases are the ones when the
>>>> assignments/mutations are non-trivial and done because the data is actually
>>>> used - i'm worried that doing the analysis *then* would be too expensive.
>>>>
>>>
>>> I don't think so.
>>
>> Consider a container, that you need to verify and/or extract some data from,
>> before passing it on to somewhere else. All the accesses before this handover
>> happens may indeed be safe and no reference be live anymore. But the compiler
>> needs to *prove* this before allowing the conversion to happen.
>> This could be very expensive and because it is not an optimization, but a
>> correctness issue, the check can *not* be disabled. Every D compiler now has
>> to do it, every time.
> 
> I think you are overstating the issue. Everything I proposed is quite cheap to carry out.

It's something you need to implement in order to have a working D compiler;
CTFE is bad enough, but that can be easily worked around by an extra internal
compile-execute pass.
Also, imagine generated D code that uses this feature; i'd rather not have to
wait for minutes every time that module is compiled, because the compiler has
no choice but to run the checks.


In general, I agree a "uniq" storage class would be useful; remember in the other
thread I said:
> There should have been another class, in addition to immutable/const, say "uniq".
It was actually you that then pointed out that some things could be made to work
even without it.
"uniq" is D3 material, and should wait until it's clear if "shared" can be made
to work at all. I'm certainly not convinced it can; started implementing a sane
thread module yesterday, we'll see in a coup[e weeks, i guess. :)

artur