RFC: scope and borrowing

[via Digitalmars-d]

On Monday, 22 September 2014 at 12:37:47 UTC, Manu via 
Digitalmars-d wrote:
> On 22 September 2014 22:14, via Digitalmars-d 
> <digitalmars-d at puremagic.com>
> wrote:
>
>> On Monday, 22 September 2014 at 11:45:39 UTC, Manu via 
>> Digitalmars-d wrote:
>>
>>> Application to scope will be identical to ref. A function 
>>> that returns or
>>> receives scope that is inserted into generic code must have 
>>> that property
>>> cascaded outwards appropriately. If typeof() or alias loses 
>>> 'scope', then
>>> it will all go tits-up.
>>>
>>
>> For receiving it's not necessary, because whether or not the 
>> argument is
>> scoped, the function can always borrow it. The lifetime of its 
>> parameter is
>> narrower than what it gets passed.
>>
>
> It's particularly common in D to produce templates that wrap 
> functions.
> If the wrapper doesn't propagate scope outwards, then it can no 
> longer be
> called by a caller who borrowed arguments which are to be 
> forwarded to the
> function being called. Likewise for return values.

You have a point there.

>
> For return values, the situation is a bit different: They can 
> of course not
>> be assigned to non-scoped variables. But the solution for this 
>> simple: the
>> generic code needs to use scope, too.
>
>
> This is precisely the problem with ref...
> Are you saying that ALL generic code needs to be 'scope' 
> always? That's not
> semantically correct.
>

To be clear, I am referring to the implementation, the actual 
code of the generic functions, not to its signature. The 
signature of course needs to match the semantics of the generic 
function.

I also over-generalized when I said that the return value cannot 
be assigned to non-scope. It can theoretically depend on the 
input, though I'm not sure whether it's a good idea to allow this:

     scope!a T scopeFunc(scope T a, scope T b);

     T* genericFunc(T)(T* input1, T* input2) {
         ...
         // this is fine in theory: input1 points to GC or global 
data
         // (because it's not designated as scope)
         string temp = scopeFunc(input1, input2);
         ...
         return temp;
     }

Evidently, this generic function cannot accept scoped pointers, 
thus it can't take advantage of the fact that scopeFunc() does. 
It's therefore a good idea, to make any generic (and non-generic, 
too) function take its parameters by scope if at all possible:

     scope!input1 T* genericFunc(T)(scope T* input1, scope T* 
input2) {
         ...
         scope temp = scopeFunc(input1, input2);
         ...
         return temp;
     }

This second version of the function will work with scope and 
non-scope inputs alike. More importantly, it doesn't depend on 
whether it's allowed to assign a scope return value to non-scope 
if its owners aren't scoped (which I'd like to avoid).

Now, `genericFunc()` in turn returns a scoped reference, so any 
other generic code that calls it must again be treated in the 
same way. Everything else would be unsafe, after all. But note 
that this only goes as far as an actual scoped value is returned 
up the call-chain. Once you stop doing so (because you only need 
to call the scope-returning functions internally for intermediate 
results, for example), returning scope would no longer be 
necessary. It still makes sense for these higher-up functions to 
_accept_ scope, of course, if it's possible.

Of course, this is only true as long as the generic function 
knows about the semantics of `scopeFunc()`. Once you're trying to 
wrap functions (as alias predicates, opDispatch), there needs to 
be another solution. I'm not sure what this could be though. I 
see now why you mentioned ref. But the problem is not restricted 
to ref and scope, it would also apply to UDAs. Maybe, because it 
is a more general problem independent of scope, the solution 
needs to be a more general one, too.

As far as I can see, there's always a variadic template parameter 
involved (which is actually a list of aliases in most cases, 
right?). Would it work if aliases would forward their storage 
classes, too? Thinking about it, this seems natural, because 
aliases mean "pass by name".

> > A function that returns scope does so for a reason after all.
>
>
> And the generic code can't know what it is. That knowledge must 
> be encoded
> in the type system.
>
> This will work even if the return value of the called function 
> turns out
>> not to be scoped for this particular instantiation. And all 
>> this is an
>> implementation of the generic code, it won't bleed outside, 
>> unless the
>> generic code wants to return the scoped value. In this case, 
>> simply apply
>> the same technique, just one lever higher.
>>
>
> I can't see the solution you're trying to ilustrate, can you 
> demonstrate?

I hope that the examples above illustrate what I mean. Of course, 
this doesn't solve the "perfect forwarding" problem, which should 
maybe be treated separately.

Maybe you can give counter examples too, if you think it doesn't 
work.