Prototype of Ownership/Borrowing System for D

[Timon Gehr]

On 20.11.19 23:45, Walter Bright wrote:
> On 11/20/2019 4:16 AM, Timon Gehr wrote:
>> - What do you want to achieve with borrowing/ownership in D?
> 
> I want to prevent the following common issues with pointer code:
> 
> 1. use after free
> 2. neglecting to free
> 3. double free
> ...

GC prevents those, and those problems cannot appear in @safe code.
@live doesn't prevent them at the interface between @live and non- at live 
code.

> 4. safe casting to immutable
> 5. safe conversion to/from a shared pointer
> ...

What about user-defined types? What about allowing internal pointers 
into manually-managed memory to be exposed in @safe code?

> 
>> - What can already be done with @live? (Ideally with runnable code 
>> examples.)
> 
> The test cases included with the PR should give an idea.
> ...

Well, they are compiler tests, not use cases.

> 
>> - How will I write a compiler-checked memory safe program that uses 
>> varied allocation strategies, including plain malloc,
> 
> I'm not sure what clarification you want about plain malloc/free, 
> although there are limitations outlined in ob.md.
> ...

I.e., it is not planned that we will be able to write such programs?

>> tracing GC
> 
> I don't see any value OB adds to tracing GC. A GC is an entirely 
> different solution.
> ...

The worry is that @live _removes_ value from tracing GC. If every 
pointer is owns its data, how do I express a pointer to GC-owned memory? 
Do I need to write a "smart" pointer data type that's just a shallow 
wrapper for a GC pointer? Also, if I do that, how do I make sure 
different GC-backed pointers don't lend out the same owning pointer at 
the same time?

>> and reference counting?
> 
> The main difficulty (as you pointed out) with RC is holding on to an 
> interior pointer via one reference while another reference free's what 
> it's pointing to.

Yup.

> There's been a lot of progress with this with the addition of DIP25, 
> DIP1000, and DIP1012. This further improves it by making the protections 
> transitive.
> ...

As far as I can tell, @live doesn't bring us closer to @safe RC, because 
it applies to built-in pointers instead of library-defined smart 
pointers. I think this is completely backwards. Every owning pointer 
also needs to know the allocation strategy. Therefore, allowing built-in 
pointers to own their memory is vastly less useful than allowing 
library-defined smart pointers to do so.

> 
>> Right now, the only use I can see for @live is as an incomplete and 
>> unsound linting tool in @system code.
> 
> The unsoundness is in dealing with thrown exceptions, which I have some 
> ideas on how to deal with,

That's not the only problem. @live code can call non- at live code and 
obtain pointers from such code. Therefore, @safe @live code is useless, 
as all accessible pointers mustn't be owning anyway.

> and conflating different allocators, which I 
> don't have a good idea on.
> ...

Do the checks for library-defined smart pointers instead of built-in 
pointers. Built-in pointers shouldn't care about lifetime nor allocator.

>> It doesn't make @safe code any more expressive. To me, added 
>> expressiveness in @safe code is the whole point of a borrowing scheme.
> 
> Technically, @live works by adding restrictions, not expressiveness.

The point of adding restrictions is to gain expressiveness. It's why 
type systems are a good idea. In this case, the point of borrowing 
restrictions should be to enable @safe code to manipulate interior 
pointers into manually-managed data structures.