Prototype of Ownership/Borrowing System for D

Timon Gehr timon.gehr at gmx.ch
Tue Nov 26 13:07:07 UTC 2019


On 26.11.19 10:34, Walter Bright wrote:
> On 11/23/2019 3:40 PM, Timon Gehr wrote:
>> I don't understand why this is not a concern for _your_ designs, which 
>> freely admit to being uncheckable and unsound. You can't say "@safe 
>> means memory safe and this is checked by the compiler" and at the same 
>> time "@live @safe relies on unchecked conventions to ensure memory 
>> safety across the @live boundary".
> 
> All languages that have an "unsafe" annotation rely on unchecked 
> conventions when the boundary is crossed.
> ...

I was very careful _not_ to cross that boundary. For your argument to 
make sense, crossing the boundary between @live and non- at live needs to 
be @system.

You can't say that just because @trusted exists, @safe doesn't need to 
do any checking either. Calling @live from non- at live or the other way 
around amounts to an unsafe type cast.

> 
>> I don't think this is the case. The GC-allocated raw pointer allows 
>> aliasing while @live does not allow aliasing. They have incompatible 
>> aliasing restrictions. It's like having a mutable and an immutable 
>> reference to the same memory location.
> 
> If I may sum this up, it is you wish to not follow O/B rules in @live 
> functions when you've got a GC pointer.

I don't really want @live functions. I want O/B, but only where it makes 
sense.

> I suspect it is possible to 
> segregate such operations into separate, non- at live functions,

It is not possible. What if you store a std.container.Array as a field 
of a class object and want to borrow the internal data? If you null out 
the class pointer after borrowing the internal array data, the GC may 
deallocate the internal array data. @live can't protect against such 
interactions.

> and I concede that you'll find this inconvenient.
> ...

Inconvenient and unsound. I would have to write @system code, and the 
compiler wouldn't even alert me that there is a problem if I annotate it 
@safe.

> In your scheme, this issue is resolved by distinguishing the two by 
> annotating the non-GC pointers with `scope`.
> ...

Yes, but `scope` does not track the allocator. `scope` restricts 
lifetime, and possibly aliasing. As I also said, alternatively, and 
perhaps preferably, we could annotate aliasing restrictions separately, 
but the accepted DIP1021 already introduces some checking against 
aliasing outside @live code.

> 
>> What about the fact that it is _optional_ for a /caller/ to respect 
>> the smart pointer's desired ownership restrictions? That's very 
>> restrictive for the smart pointer! It won't be able to provide @safe 
>> borrowing functionality.
> 
> I understand that the salient difference between your scheme and mine is 
> you attach it to the pointer type while mine attaches it to the 
> function.

You attach it to the function, but the meaning pertains to pointers. 
This should be a red flag, but apparently you don't think so. Then, you 
allow @live and non- at live code to interact in a @safe context. This is 
highly unsound because those calls change the interpretation of pointer 
types. Those are unsafe pointer casts.

> Pretty much all of this discussion is about consequences of 
> this difference,

Not true. Even if I accept that a function attribute is a reasonable way 
to go, @live comes short.

> so I don't think it is necessary to go through it 
> point-by-point agreeing with you on those consequences.
> ...

This makes no sense to me.  It seems rather weird to be debating the 
merits of proposals without actually taking into account the 
consequences of each proposal.

> But yours has problems, too, so the question is which method is better?
> ...

I don't care about "my" method vs "your" method. I want a method that 
makes sense, does not break @safe or GC and improves @safe. This can 
just as well be some new combination.

The issue was that you weren't responding to any of my points until I 
made some concrete proposal. I am not stuck to that. We can improve it.

> Some problems:
> 
> 1. Doesn't seem to be a way to prevent un-scope pointers from existing 
> and being unsafe.
> ...

It doesn't statically prevent memory corruption in @system code. I don't 
think this is a "problem".

> 2. The splitting pointers down the middle into two categories: scope and 
> un-scope.

It appears this split exists after 
https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1021.md

> This is a very intrusive and drastic change.

I think it's less so than splitting functions into two mutually 
incompatible categories.

> The only saving 
> grace of the existing `scope` and `return` annotations is the compiler 
> can infer the vast bulk of them.

But now there's DIP1021, which checks (unsoundly) that `scope` pointers 
don't alias. This is not my invention and I would be on board with 
reverting the decision on DIP1021.

> Will people accept manually adding 
> `scope` to a big chunk of their types? I don't know.

If those types want manage memory manually and expose the internal 
memory directly to a @safe caller, yes.
If the @safe caller wants to pass around manually-managed memory it will 
have to annotate stuff as `scope`. This is also true with @live.


If you want to avoid intrusive and drastic language changes, what about 
reverting DIP1021, moving aliasing checks to run time? We could add 
opBorrow and opUnborrow primitives. opBorrow would return a scoped value 
borrowed from the receiver and opUnborrow would be called once the last 
borrow ends. This would even be quite a bit more precise than doing 
everything in the type system, because you would only prevent 
invalidation, not all mutation.


More information about the Digitalmars-d mailing list