RFC: moving forward with @nogc Phobos

[Andrei Alexandrescu via Digitalmars-d]

Back when I've first introduced RCString I hinted that we have a larger 
strategy in mind. Here it is.

The basic tenet of the approach is to reckon and act on the fact that 
memory allocation (the subject of allocators) is an entirely distinct 
topic from memory management, and more generally resource management. 
This clarifies that it would be wrong to approach alternatives to GC in 
Phobos by means of allocators. GC is not only an approach to memory 
allocation, but also an approach to memory management. Reducing it to 
either one is a mistake. In hindsight this looks rather obvious but it 
has caused me and many people better than myself a lot of headache.

That said allocators are nice to have and use, and I will definitely 
follow up with std.allocator. However, std.allocator is not the key to a 
@nogc Phobos.

Nor are ranges. There is an attitude that either output ranges, or input 
ranges in conjunction with lazy computation, would solve the issue of 
creating garbage. 
https://github.com/D-Programming-Language/phobos/pull/2423 is a good 
illustration of the latter approach: a range would be lazily created by 
chaining stuff together. A range-based approach would take us further 
than the allocators, but I see the following issues with it:

(a) the whole approach doesn't stand scrutiny for non-linear outputs, 
e.g. outputting some sort of associative array or really any composite 
type quickly becomes tenuous either with an output range (eager) or with 
exposing an input range (lazy);

(b) makes the style of programming without GC radically different, and 
much more cumbersome, than programming with GC; as a consequence, 
programmers who consider changing one approach to another, or 
implementing an algorithm neutral to it, are looking at a major rewrite;

(c) would make D/@nogc a poor cousin of C++. This is quite out of 
character; technically, I have long gotten used to seeing most elaborate 
C++ code like poor emulation of simple D idioms. But C++ has spent years 
and decades taking to perfection an approach without a tracing garbage 
collector. A departure from that would need to be superior, and that 
doesn't seem to be the case with range-based approaches.

===========

Now that we clarified that these existing attempts are not going to work 
well, the question remains what does. For Phobos I'm thinking of 
defining and using three policies:

enum MemoryManagementPolicy { gc, rc, mrc }
immutable
     gc = ResourceManagementPolicy.gc,
     rc = ResourceManagementPolicy.rc,
     mrc = ResourceManagementPolicy.mrc;

The three policies are:

(a) gc is the classic garbage-collected style of management;

(b) rc is a reference-counted style still backed by the GC, i.e. the GC 
will still be able to pick up cycles and other kinds of leaks.

(c) mrc is a reference-counted style backed by malloc.

(It should be possible to collapse rc and mrc together and make the 
distinction dynamically, at runtime. I'm distinguishing them statically 
here for expository purposes.)

The policy is a template parameter to functions in Phobos (and 
elsewhere), and informs the functions e.g. what types to return. Consider:

auto setExtension(MemoryManagementPolicy mmp = gc, R1, R2)(R1 path, R2 ext)
if (...)
{
     static if (mmp == gc) alias S = string;
     else alias S = RCString;
     S result;
     ...
     return result;
}

On the caller side:

auto p1 = setExtension("hello", ".txt"); // fine, use gc
auto p2 = setExtension!gc("hello", ".txt"); // same
auto p3 = setExtension!rc("hello", ".txt"); // fine, use rc

So by default it's going to continue being business as usual, but 
certain functions will allow passing in a (defaulted) policy for memory 
management.

Destroy!


Andrei