DMD Symbol Reference Analysis Pass

[via Digitalmars-d]

On Wednesday, 27 May 2015 at 08:38:48 UTC, Per Nordlöw wrote:
> On Wednesday, 27 May 2015 at 08:30:33 UTC, Marc Schütz wrote:
>> See above. Conceptually, you can of course treat it as if it 
>> were marked with `scope`, but an actual annotation should not 
>> be necessary.
>
> But now you're talking about an upcoming feature in DMD, right?
>

Well, obviously, nothing of what we're talking about works with 
current DMD. Even scope doesn't do anything (except for 
delegates).

> AFAIK, in current DMD, I can't get any help in avoiding 
> patterns such as
>
>     char[] saved_line;
>     foreach (line; File("foo.txt").byLine)
>     {
>         saved_line = line; // should give error
>     }
>
> Right?

Yes.

>
> Are you saying that adding DMD support for qualifying `line` as 
> `scope` is not the right way to solve this problem?

Yes. First of all, `File.byLine.front` is the function that needs 
to get annotated, like this:

     char[] front() return {
         // ...
         return buffer;
     }

The `return` keyword here means the same thing as in DIP25, 
namely that the returned value is owned by `this`. In your 
example, the owner is the temporary returned by 
`File("foo.txt").byLine`, which means that the returned buffer 
must no longer be used when that temporary gets destroyed, i.e. 
the read strings must not escape the foreach (without being 
copied), which allows the buffer to be safely released then. This 
is the original problem that `scope` was meant to address.

Conceptually, you're right that now `line` needs to be annotated 
with `scope`, because otherwise you wouldn't be allowed to store 
the scoped slices there. But there really isn't any point in 
actually adding that annotation, because it can always be 
inferred by the compiler (it can add the annotation for you when 
it sees that you assign a scoped value to it).

This alone is then already enough to prevent the "volatility" 
problem in your example, because it is longer possible for the 
individual lines to outlive the byLine() temporary that gets 
iterated over.

However, it is not enough in the general case:

     auto lines = stdin.ByLine;
     auto line1 = lines.front;
     lines.popFront();
     // line1 now changes
     auto line2 = lines.front;

In this case, the rule that `line1` must not outlive `lines` is 
fulfilled, but still it gets invalidated. With byLine(), this 
just leads to unexpected behaviour, but with e.g. reference 
counting, it can cause memory corruption (use after free). 
Therefore, any complete scope proposal needs to address this 
problem, too.

What I propose is the following: The compiler keeps track of 
outstanding "loans" to owned objects. As long as any such "loan" 
exists (in the above example, `line1` and `line2`), the owner 
(i.e. `lines`) will either become read-only (const), or 
alternatively, it will stay mutable, but mutating it will become 
@system. This effectively addresses both the safety problems as 
well as volatile ranges, because they are actually the same 
problem.

I hope it is now clear what I want to say. It is unfortunately a 
complicated topic...