Prototype of Ownership/Borrowing System for D

[Walter Bright]

On 11/26/2019 5:07 AM, Timon Gehr wrote:
> You can't say that just because @trusted exists, @safe doesn't need to do any 
> checking either. Calling @live from non- at live or the other way around amounts to 
> an unsafe type cast.

Of course when @live functions are called, they must supply arguments that fit 
@live rules. When @live functions call other functions, the interface to those 
functions must agree with @live rules.

This is not in dispute. You don't have to keep hitting me over the head with it :-)


> I want O/B, but only where it makes sense.

I don't disagree with that, either.


>> I suspect it is possible to segregate such operations into separate, non- at live 
>> functions,
> 
> It is not possible. What if you store a std.container.Array as a field of a 
> class object and want to borrow the internal data? If you null out the class 
> pointer after borrowing the internal array data, the GC may deallocate the 
> internal array data. @live can't protect against such interactions.

@live never inserts null. It does not change the code at all. It only adds 
compile time checks. Marking pointers as "Invalid" after they are borrowed is 
not a code generation or runtime thing, it is purely inside the compiler.

BTW, under O/B rules, if you borrow a pointer to the internal data, the owner of 
the data structure (i.e. the cless reference) gets marked "Invalid" (not the 
field in the class) until the pointer is no longer live.


> Inconvenient and unsound. I would have to write @system code, and the compiler 
> wouldn't even alert me that there is a problem if I annotate it @safe.

This is true. The prototype has set up @live to be an attribute independent of 
@safe/@trusted/@system annotations. The idea is to try it out without disruption 
of the logic of the latter 3 attributes. If it works well, I anticipate working 
it in.


>> In your scheme, this issue is resolved by distinguishing the two by annotating 
>> the non-GC pointers with `scope`.
>> ...
> Yes, but `scope` does not track the allocator. `scope` restricts lifetime, and 
> possibly aliasing. As I also said, alternatively, and perhaps preferably, we 
> could annotate aliasing restrictions separately, but the accepted DIP1021 
> already introduces some checking against aliasing outside @live code.

Ok.

>> I understand that the salient difference between your scheme and mine is you 
>> attach it to the pointer type while mine attaches it to the function.
> You attach it to the function, but the meaning pertains to pointers. This should 
> be a red flag, but apparently you don't think so.

That's right.


> This makes no sense to me.  It seems rather weird to be debating the merits of 
> proposals without actually taking into account the consequences of each proposal.

The problem is I agree with your technical criticisms of my proposal, you do not 
have to keep reiterating them.

> I want a method that makes 
> sense, does not break @safe or GC and improves @safe. This can just as well be 
> some new combination.

We're both on the same team here.


>> 1. Doesn't seem to be a way to prevent un-scope pointers from existing and 
>> being unsafe.
>> ...
> It doesn't statically prevent memory corruption in @system code. I don't think 
> this is a "problem".

What happens with raw pointers? Would use of them produce a compile time error?


>> 2. The splitting pointers down the middle into two categories: scope and 
>> un-scope.
> It appears this split exists after 
> https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1021.md
> 
>> This is a very intrusive and drastic change.
> 
> I think it's less so than splitting functions into two mutually incompatible 
> categories.
> 
>> The only saving grace of the existing `scope` and `return` annotations is the 
>> compiler can infer the vast bulk of them.
> 
> But now there's DIP1021, which checks (unsoundly) that `scope` pointers don't 
> alias. This is not my invention and I would be on board with reverting the 
> decision on DIP1021.
> 
>> Will people accept manually adding `scope` to a big chunk of their types? I 
>> don't know.
> 
> If those types want manage memory manually and expose the internal memory 
> directly to a @safe caller, yes.
> If the @safe caller wants to pass around manually-managed memory it will have to 
> annotate stuff as `scope`. This is also true with @live.

The difference is that `scope` is inferred for parameters for templates, auto 
return functions, and lambdas. `scope` for local pointers is inferred for all 
functions. This works out rather nicely in practice such that most of the time, 
the user doesn't need to add any annotations. I am not seeing this happening 
with your proposal.

I also find concerning the idea that one can have a pointer to a scope pointer - 
how can that work with O/B?


> If you want to avoid intrusive and drastic language changes, what about 
> reverting DIP1021,

It's currently enabled only with a switch. It's well worth seeing how it works 
in practice before deciding.

> moving aliasing checks to run time?

Selling runtime checks is a tough battle.

> We could add opBorrow and 
> opUnborrow primitives. opBorrow would return a scoped value borrowed from the 
> receiver and opUnborrow would be called once the last borrow ends. This would 
> even be quite a bit more precise than doing everything in the type system, 
> because you would only prevent invalidation, not all mutation.

I've resisted similar things before on performance grounds. Rust has done a 
pretty good job selling safety as a compile time phenomenon.