An important potential change to the language: transitory ref

[Andrei Alexandrescu]

On 03/20/2010 12:56 AM, Steven Schveighoffer wrote:
> On Sat, 20 Mar 2010 00:07:55 -0400, Andrei Alexandrescu
> <SeeWebsiteForEmail at erdani.org> wrote:
>> I remember you brought up a similar point in a related discussion a
>> couple of years ago. It's a good point, and my current understanding
>> of the matter is that functions that take and return ref could and
>> should be handled conservatively.
>
> I don't like the sound of that... What I fear is that the compiler will
> force people to start using pointers because refs don't cut it. I'm
> guessing you mean you cannot return ref returns from other functions?
> That breaks abstraction principles, I should be able to delegate a task
> to a sub-function.

Perhaps it means you can't return ref returns from other functions if 
you pass them references to local state.

(I've read a paper at some point about a program analysis that stored 
for each function the "return pattern" - a mini-graph describing the 
relationship between parameters and result. If it rings a bell to 
anyone... please chime in.)

>>> For instance, try to find a rule that prevents the above from compiling,
>>> but allows the following to compile.
>>>
>>> struct S
>>> {
>>> private int x;
>>> ref int getX() { return x;}
>>> }
>>>
>>> struct T
>>> {
>>> S s;
>>> ref int getSX() { return s.x; }
>>> }
>>
>> In the approach discussed with Walter, S is illegal. A struct can't
>> define a method to return a reference to a direct member. This is
>> exactly the advice given in Scott's book for C++. (A class can because
>> classes sit on the heap.)
>
> A struct may sit on the heap too.

Yes. For those cases you can always use pointers, which are not subject 
to the restrictions I envision for ref.

It's a very small inconvenience. For example, if you have a linked list 
struct, you may feel constrained that you can't do:

struct List {
     List * next;
     List * prepend(List * lst) {
         lst.next = &this;
         return lst;
     }
}

In my approach, &this is illegal. And actually for a good reason. This 
code bombs:

List iForgotThePointer() {
     List lst;
     lst.prepend(new List);
     return lst;
}

My response to the above issue is two-pronged:

(a) For List a class would be an alternative
(b) To work with pointers to structs use static member functions and 
pointers instead of methods and references

> I don't know how else to describe it,
> but it feels like you are applying the solution in the wrong place. I
> understand what you are trying to solve, but your solution may be too
> blunt an instrument.

The goal is worth pursuing, so let's keep on thinking of how to make it 
work. If D manages to define demonstrably safe encapsulated containers, 
that would be an absolutely huge win.

> Here's another case:
>
> struct S
> {
> int*x;
> ref int getX() {return *x;}
> }
>
> Is x on the heap or not? How do you know? Arrays are just a wrapped
> pointer, so they too could be stack allocated.

struct S
{
     int*x;
     static ref int getX(S * p) {return *p.x;}
}

In an ideal world, if you have your hands on a pointer to a struct, you 
should be reasonably certain that that lives on the heap. It would be 
just great if D could guarantee that.

> Consider this:
>
> void foo(ref int x)
> {
> x++;
> }
>
> struct S
> {
> int x;
> int y;
> bool xisy;
> ref int getX() {if(xisy) return y; return x;}
> }
>
> foo(S.x);
> foo(S.getX());

Hm, I assume the two lines refer to an object of type S. The example 
above would again have to be rewritten in terms of static functions with 
pointers.

> Another case:
>
> struct S
> {
> int x;
> ref S opUnary(string op)() if (op == "++") {++x; return this;}
> }
>
> I feel this should all be possible.

I think opUnary should return void and the compiler should worry about 
that result being used.

> ------
> counter proposal:
>
> What about having a new kind of ref that can only be passed up the
> stack, or down only one level if you are the one who initiated it.
>
> Call it scope ref:
>
> ref int baz(ref y)
> {
> return y;
> }
> scope ref int foo(scope ref int x, ref int y)
> {
> //return x; // illegal, we did not make x scope ref
> //return baz(x); // illegal, cannot convert scope ref into ref
> return y; // legal, you can convert a ref parameter into scope ref.
> }
>
> scope ref int bar()
> {
> int y;
> //return foo(y, y); //illegal, you cannot pass scope refs down the stack
> more than one level
> }
>
> At least this leaves ref alone to be used without restrictions that the
> compiler can't prove are necessary. If we find scope ref is the only
> kind of ref we ever use, then maybe we can get rid of scope ref and just
> make ref be the restricted form. Or you could keep scope ref and reserve
> ref for only provable heap-variables.
>
> Man, it would be nice to have escape analysis...

It sure would, but it quickly gets into the interprocedural tarpit.

Your idea is good, except I don't see why not make ref scoped ref. After 
all ref is currently not an enabler - it could be missing from the 
language; pointers are fine. So why not make ref do something actually 
interesting?


Andrei