Rvalue references - The resolution

[Walter Bright]

Thanks to the many recent threads on this, and the dips on it, everyone was 
pretty much up to speed and ready to find a resolution. This resolution only 
deals with the memory safety issue.

The first point is that rvalues are turned into references by the simple 
expedient of creating a temporary, copying the rvalue into the temporary, and 
taking the address of that temporary. Therefore, the issue is really about 
returning references to stack variables that have gone out of scope. From a 
memory safety issue, this is unacceptable as D strives to be a memory safe 
language. The solution in other languages of "just don't do that" is invalid for D.

Cases where this can occur:

Case A:
     ref T fooa(ref T t) { return t; }
     ref T bar() { T t; return fooa(t); }

Case B:
     ref T foob(ref U u) { return u.t; }   // note that T is derivable from U
     ref U bar() { T t; return foob(t); }

Case C:
     struct S { T t; ref T fooc() { return t; } }
     ref T bar() { S s; return s.fooc(); }

Case D:
     Returning ref to uplevel local:

     ref T food() {
         T t;
         ref T bar() { return t; }
         return bar();
     }

case E:
     Transitively calling other functions:

     ref T fooe(T t) { return fooa(t); }



Observations:

1. Always involves a return statement.
2. The return type must always be the type of the stack variable or a type type 
derived from a stack variable's type via safe casting or subtyping.
3. Returning rvalues is the same issue, as rvalues are always turned into local 
stack temporaries.
4. Whether a function returns a ref derived from a parameter or not is not 
reflected in the function signature.
5. Always involves passing a local by ref to a function that returns by ref, and 
that function gets called in a return statement.

Scope Ref

http://wiki.dlang.org/DIP35 is one solution, but Andrei and I argued strongly 
against it due to the perceived complexity the user would face with it. I also 
argued against it due to Case C (where would the scope annotation go) and the 
possibility that functions returning ref would have to appear in pairs - one 
with scope ref parameters, the other without - and the copy/pasta duplication of 
the function bodies (which appears in C++ const& functions).

Andrei & I argued that we needed to make it work with just ref annotations.


Static Compiler Detection (in @safe mode):

1. Do not allow taking the address of a local variable, unless doing a safe type 
'paint' operation.

2. In some cases, such as nested, private, and template functions, the source is 
always available so the compiler can error on those. Because of the .di file 
problem, doing this with auto return functions is problematic.

3. Issue error on return statements where the expression may contain a ref to a 
local that is going out of scope, taking into account the observations.

Runtime Detection

There are still a few cases that the compiler cannot statically detect. For 
these a runtime check is inserted, which compares the returned ref pointer to 
see if it lies within the stack frame of the exiting function, and if it does, 
halts the program. The cost will be a couple of CMP instructions and an LEA. 
These checks would be omitted if the -noboundscheck compiler switch was provided.

The runtime check would not be on all ref returning functions. It'll only be on 
those where the compiler cannot prove a ref to a local is not being returned.

The good thing about the runtime detection is that ref's use is restricted 
enough that merely executing all the code paths will check all the possibilities.