DIP77 - Fix unsafe RC pass by 'ref'

[via Digitalmars-d]

On Friday, 10 April 2015 at 17:43:00 UTC, Walter Bright wrote:
> On 4/10/2015 7:23 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= 
> <schuetzm at gmx.net>" wrote:
>> On Wednesday, 8 April 2015 at 23:11:08 UTC, Walter Bright 
>> wrote:
>>> http://wiki.dlang.org/DIP77
>>
>> In general, this is way too much focused on reference 
>> counting. The DIP does
>> define "RCO" in more general terms, but still, it feels like 
>> the wrong approach.
>>
>> You also seem to look at Rust's type system from the same POV 
>> ("only one mutable
>> reference at a time, for efficiency"), but that's not the 
>> point at all: it's
>> about safety in general, not efficiency for RC. It encompasses 
>> so much more:
>> memory safety, race-free sharing of data, avoiding iterator 
>> invalidation...
>> DIP77 cannot help with general iterator invalidation at all.
>
> D uses ranges, not iterators.

I used "iterator invalidation" because it is an established term. 
You get the same problems with ranges, once the contents aren't 
GC managed. And even if they are, your code is merely memory 
safe, but not necessarily correct. stdin.byLine() without idup is 
a prime example.

>
>
>> Another thing that just feels wrong is the automagic copying. 
>> We're passing
>> something by `ref`, and the compiler inserts a copy behind our 
>> back?! And this
>> isn't even visible to the user...
>>
>> Because the topic is correctness, @safe/@system seems the 
>> correct way to
>> approach the problem. The compiler should just treat unsafe 
>> pass-by-ref as
>> @system, as Martin suggests. Basically, instead of inserting a 
>> copy, pass-by-ref
>> will be @system, when no copy would need to be inserted, it 
>> will be @safe.
>
> A difficulty with that is the semantics will change as the 
> compiler improves and gets better at realizing no copy would be 
> needed. Having the @safety be "implementation defined" would be 
> unhappy.

Agreed, it needs to be specified. I'm aware that this is 
difficult. Though, why can't we use the same approach we use in 
other cases? Be restrictive at first, later accept more, as we 
refine the specification... At some point we'll find the sweet 
spot between false negatives and complexity of specification, and 
can stabilize on it, so that other compilers have a non-moving 
target.

>
>
>> I also think that the unsafety detection heuristic is too 
>> simplistic. When
>> scope/return is extended to pointers and other kinds of 
>> references, there would
>> probably be way too many false positives. A more sophisticated 
>> analysis is
>> necessary that can more realistically determine when mutable 
>> aliasing can occur.
>> E.g., it can also take uniqueness into account.
>>
>> Besides, the address and slice operators also need to be taken 
>> into account, and
>> these can appear outside of function calls.
>
> I expect any use of RCO's necessarily means the implementation 
> of the RCO is unsafe (after all, it calls free()), but a safe 
> interface is provided. The safe interface to an RCO is tightly 
> controlled with 'return ref'.

Uhm... how is this related to what I wrote?

Example:

void foo() @safe {
     RCArray!int arr = [0,1,2];
     {
         int* p = &arr[0];  // legal under new scope rules
         RCArray!int other;
         arr = other;
         writeln(*p);       // OOPS
     }
}

This shows that with the extended scope rules (i.e. scope not 
restricted to `ref`), the borrowing can happen everywhere, not 
just on function calls. By your rules, every address and slice 
operator applied to a mutable local would need to trigger the 
copying, because the resulting (borrowed) reference exists in the 
same scope as the original variable (owner).

I'd prefer instead to make writing to the owner unsafe, as long 
as borrowed references to it exist (this also includes potential 
writes, like passing a mutable ref to another function). This 
doesn't require copying behind the user's back, and allows the 
user to choose the solution best suited for the situation: manual 
copying, deferring the write to the owner, @trusted...