RFC: scope and borrowing

[via Digitalmars-d]

On Friday, 3 October 2014 at 19:08:10 UTC, Ivan Timokhin wrote:
> 29.09.2014 18:17, "Marc =?UTF-8?B?U2Now7x0eiI=?= 
> <schuetzm at gmx.net>" пишет:
>> * What is ElementType!(ByLineImpl!(char, "\n")) in the example 
>> from the
>> wiki page [1]?
>
> OK, I think I have an idea, but it's not overly elegant.
>
> First of all, I would like to note that the same issue exists 
> with, for example, findSubstring function (from "scope with 
> owners" section), which does not have a definite return type 
> (so it's unclear what ReturnType!findSubstring should be).
>

Indeed. It's everywhere that an owner depends on another function 
argument (including `this`), but AFAICS nowhere else.

> Now, an idea that I have is that bare scope should be just a 
> syntactic sugar for self-owned scope; i.e., `scope(int*) x;` 
> would be just a short way of writing `scope!x(int*) x;`. This 
> would imply that every scoped variable has its own unique type, 
> which is probably not that terrible, considering that they 
> can't be assigned freely to each other either way. This is also 
> somewhat more natural, because it means that there is no such 
> thing as just "scoped" variable, it is always connected to a 
> particular lifetime.

I've already suggested this as an implementation detail.

>
> Then, the following:
> ---
> 	string findSubstring(scope(string) haystack,
> 		             scope(string) needle)
> ---
> would be equivalent to
> ---
> 	string findSubstring(scope!haystack(string) haystack,
> 		             scope!needle(string) needle)
> ---
> so each argument is self-owned and all ownership information is 
> lost (but function is still callable with scoped arguments, 
> because scope narrowing is allowed).
>
> To preserve ownership information, which is encoded in an 
> arguments' types, template is needed:
> ---
> 	scope!haystackOwner(string)
> 	findSubstring(alias haystackOwner)
> 		            (scope!haystackOwner(string) haystack,
> 		             scope(string) needle)
> ---
> So, findSubstring *still* doesn't have a definite return type, 
> but now it's because it is a function template, not a function.

This of course has the unfortunate side effect of incredible 
template bloat: For any distinct passed argument (which in 
practice means for almost every call), we'd get a new template 
instance. IMO this is not acceptable.

>
> As for passing unscoped strings to findSubstring, I see two 
> alternatives:
> 1) Declare that all unscoped references are implicitly 
> convertible to scope!GC or something like that (and this 
> conversion is used in such cases). This one is probably better.

OTOH it would preclude automatic demoting of GC allocations to 
stack allocations. And it would marry us to the GC is "default" 
allocation strategy, which we might want to move away from 
(probably in favor of generic allocators changeable at any point 
in time).

We also need to allow borrowing of temporaries if we want to have 
rvalue references. And this runs into problems if we are dealing 
with scoped non-references, for which I'm seeing more use cases 
now than just file descriptors (in particular reference 
counting). In short, I'd like to avoid it.

> 2) Require a separate overload for an unscoped string.

Ugly, of course, and I believe it's not necessary.

>
> Now, to address ElementType problem, I would like to reconsider 
> `this` lifetime. Since
> ---
> 	struct S {
> 		void f() {}
> 	}
> ---
> is more or less equivalent to
> ---
> 	// not a valid D code
> 	struct S {}
> 	void f(ref S this) {}
> ---
> (not strictly equivalent, of course, but the general idea is 
> like that),
> `this` inside a method body would behave like a ref parameter 
> and have approximately the same lifetime as normal parameters.
>
> Then this code:
> ---
> 	@property scope!(const this)(Char[]) front() const
> ---
> would become illegal, since the return value is declared as 
> having function-local scope. However, this code:
> ---
> 	@property scope!(const this)(Char[])
> 		front(alias thisOwner)() const scope!thisOwner
> ---
> would work just fine, because it explicitly propagates current 
> object's scope (the return type seems the same, but `this` 
> itself now has a different type). The downside is that `front` 
> is now callable only for scoped variables (it seems 
> inappropriate to convert structs to scope!GC).

And of course, again, the template bloat :-( This time, a new 
instance of `front` for every instance of the range.

>
> Now, ByLineImpl!(char, "\n") wouldn't be an input range, 
> because front would not be defined in an unscoped case, and 
> `scope ByLineImpl!(char, "\n")` is not a type (because bare 
> scope would be purely a syntactic sugar in declarations), so 
> not a range.
>
> However, with this declaration:
> ---
> 	scope(ByLineImpl!(char, "\n")) x = ...;
> ---
> typeof(x) would be scope!x(ByLineImpl!(char, "\n")), and 
> ElementType!(typeof(x)) would be scope!(const x)(char[]).
>
> As I have said in the beginning, not too elegant, but I think 
> it may work. As a bonus, this is a little bit more 
> straightforward and requires less special-casing from the 
> compiler side: has only one scope type instead of two and no 
> magic tricks with scoped return values. The only problems that 
> I can see right away are that the code now is a little verbose, 
> and that 'front' is restricted to scoped variables in a new 
> version.
>
> Any thoughts?

I think the key is in separating the scope attribute and the 
owner. The former needs to be part of the type, the latter 
doesn't. In this vein, it's probably a good idea to restrict the 
`scope!owner` syntax to function signatures, where it may only 
refer to other parameters and `this`. The use cases for it 
elsewhere are very marginal (if they exist at all).

This naturally makes the return type of `findSubstring()` just 
`scope(string)`; declaring a variable of it simply works:

     typeof(findSubstring("", "")) s = findSubstring("Hello, 
world", "world");

is equivalent to:

     scope(string) s = findSubstring("Hello, world", "world");

This is a valid assignment, and owner propagation would even take 
care of preserving the owners (though only on declaration, but 
that's natural because the owners are only known at the call 
site):

     string haystack, needle;
     scope(string) s = findSubstring(haystack, needle);
     // type of `s` is scope(string), owner is `haystack`

In other words, the type part of `scope!a(T)` is just `scope(T)`, 
the owner is not part of the type and tracked separately.

Let's look at isInputRange:

     template isInputRange(R)
     {
         enum bool isInputRange = is(typeof(
             (inout int = 0)
             {
                 R r = R.init;     // can define a range object
                 if (r.empty) {}   // can test for empty
                 r.popFront();     // can invoke popFront()
                 auto h = r.front; // can get the front of the 
range
             }));
     }

     auto byline = stdin.byLine();

`isInputRange!(typeof(byline))` expands to:

     scope(ByLineImpl...) r = scope(ByLineImpl...).init;
     if (r.empty) {}
     r.popFront();
     auto h = r.front;

The first line is valid, the last line too, because `scope` is 
now part of the type and will of course be deduced by `auto`.

So this solves the template bloat problem, along with making 
ElementType usable. That still leaves us with the problem of 
forwarding and wrappers.

     auto trace(alias func, Args...)(Args args) {
         writeln("Calling " ~ func.stringof ~ "(" ~ args.stringof 
~ ")");
         return func(args);
     }

     auto s = trace!findSubstring(haystack, needle);

Here, `Args` becomes `(scope(string), scope(string))`. This 
cannot work, of course, because it drops the owners. How can we 
a) preserve the owners on the parameters, and b) propagate the 
result's owner back outward?

@Manu: Under the condition that we'd find a solution for "perfect 
forwarding", or imperfect if you like ;-), preferrably one also 
applicable to `ref`, would you be okay with the above?