Prototype of Ownership/Borrowing System for D

[Walter Bright]

On 11/20/2019 4:59 PM, Timon Gehr wrote:
> On 20.11.19 23:45, Walter Bright wrote:
>> On 11/20/2019 4:16 AM, Timon Gehr wrote:
>>> - What do you want to achieve with borrowing/ownership in D?
>>
>> I want to prevent the following common issues with pointer code:
>>
>> 1. use after free
>> 2. neglecting to free
>> 3. double free
>> ...
> 
> GC prevents those,

That's right. The GC is memory safe.

> and those problems cannot appear in @safe code.

@safe code has to call free() some time when manually managing memory.

> @live doesn't prevent them at the interface between @live and non- at live code.

@live relies on any function it calls obeying @live conventions for its 
interface. This allows incremental adoption of @live code.


> What about user-defined types? What about allowing internal pointers into 
> manually-managed memory to be exposed in @safe code?

Exposing an internal pointer in @live code is considered "borrowing" from the 
root of its container.


>>> - How will I write a compiler-checked memory safe program that uses varied 
>>> allocation strategies, including plain malloc,
>>
>> I'm not sure what clarification you want about plain malloc/free, although 
>> there are limitations outlined in ob.md.
>> ...
> 
> I.e., it is not planned that we will be able to write such programs?

I believe I covered that in ob.md. What am I missing?


> The worry is that @live _removes_ value from tracing GC. If every pointer is 
> owns its data, how do I express a pointer to GC-owned memory? Do I need to write 
> a "smart" pointer data type that's just a shallow wrapper for a GC pointer? 
> Also, if I do that, how do I make sure different GC-backed pointers don't lend 
> out the same owning pointer at the same time?

@live does not distinguish a GC-allocated raw pointer from a malloc-allocated 
raw pointer. This means you'll be able to write generic @live code that can 
handle both equally. Of course, if all you're using is the GC, you won't need to 
bother with @live at all.


>> There's been a lot of progress with this with the addition of DIP25, DIP1000, 
>> and DIP1012. This further improves it by making the protections transitive.
> As far as I can tell, @live doesn't bring us closer to @safe RC, because it 
> applies to built-in pointers instead of library-defined smart pointers. I think 
> this is completely backwards. Every owning pointer also needs to know the 
> allocation strategy. Therefore, allowing built-in pointers to own their memory 
> is vastly less useful than allowing library-defined smart pointers to do so.

Nothing about @live stops programmers from using library-defined smart pointers. 
The smart pointer would be the owning pointer, and if it exposed an internal 
pointer that internal pointer would be treated as "borrowing" from the owner and 
further access to the smart pointer would be denied until the borrower's last use.


>> and conflating different allocators, which I don't have a good idea on.
> Do the checks for library-defined smart pointers instead of built-in pointers. 
> Built-in pointers shouldn't care about lifetime nor allocator.

People use raw pointers, and that isn't going away (because performance). 
Telling people "just use smart pointers" is like telling C++ people to do that. 
It doesn't work reliably.

The checks on smart pointers can be done with RAII and reference counting, and 
the dips already implemented.


> The point of adding restrictions is to gain expressiveness. It's why type 
> systems are a good idea. In this case, the point of borrowing restrictions 
> should be to enable @safe code to manipulate interior pointers into 
> manually-managed data structures.

They can do that now as long as the container only exposes interior pointers as 
'ref'.