draft proposal for ref counting in D

[Walter Bright]

This is based on n.g. discussions and ideas from you guys. I'll redo it as a DIP 
if it passes the smoke test from y'all.

----------------------------------------------------------------------
     Adding Reference Counting to D

D currently supports manual memory management and generalized GC. Unfortunately, 
the pausing
and memory consumption inherent in GC is not acceptable for many programs, and 
manual
memory management is error-prone, tedious, and unsafe. A third style, reference 
counting (RC),
addresses this. Common implementations are COM's AddRef/Release, Objective-C's ARC,
and C++'s shared_ptr<>.

None of these three schemes are guaranteed memory-safe, they all require the 
programmer
to conform to a protocol (even O-C's ARC). Stepping outside of the protocol 
results in
memory corruption. A D implementation must make it possible to use ref counted 
objects
in code marked as @safe, although it will be necessary for the implementation of 
those
objects to be unsafe.

Some aspects of a D implementation are inevitable:

1. Avoid any requirement for more pointer types. This would cause drastic 
increases in
complexity for both the compiler and the user. It may make generic code much 
more difficult
to write.

2. Decay of a ref-counted pointer to a non-ref-counted pointer is unsafe, and 
can only
be allowed (in @safe code) in circumstances where it can be statically proven to 
be safe.

3. A ref counted object is inherently a reference type, not a value type.

4. The compiler needs to know about ref counted types.


==Proposal==

If a class contains the following methods, in either itself or a base class, it is
an RC class:


     T AddRef();
     T Release();

An RC class is like a regular D class with these additional semantics:

1. In @safe code, casting (implicit or explicit) to a base class that does not
have both AddRef() and Release() is an error.

2. Initialization of a class reference causes a call to AddRef().

3. Assignment to a class reference causes a call to Release() on its original value
and AddRef() on the new value.

4. Null checks are done before calling any AddRef() or Release().

5. Upon scope exit of all RC variables or temporaries, a call to Release() is 
performed,
analogously to the destruction of struct variables and temporaries.

6. If a class or struct contains RC fields, calls to Release() for those fields will
be added to the destructor, and a destructor will be created if one doesn't 
exist already.

7. If a closure is created that contains RC fields, either a compile time error 
will be
generated or a destructor will be created for it.

8. Explicit calls to AddRef/Release will not be allowed in @safe code.

9. A call to AddRef() will be added to any argument passed as a parameter.

10. Function returns have an AddRef() already done to the return value.

11. The compiler can elide any AddRef()/Release() calls it can prove are redundant.

12. AddRef() is not called when passed as the implicit 'this' reference.

13. Taking the address of, or passing by reference, any fields of an RC object
is not allowed in @safe code. Passing by reference an RC field is allowed.

14. RC objects will still be allocated on the GC heap - this means that a normal
GC run will reap RC objects that are in a cycle, and RC objects will get 
automatically
scanned for heap references with no additional action required by the user.


==Existing Code==

D COM objects already have AddRef() and Release(). This proposal should not break
that code, it'll just mean that there will be extra AddRef()/Release calls made.
Calling AddRef()/Release() should never have been allowed in @safe code anyway.

Any other existing uses of AddRef()/Release() will break.

==Arrays==

Built-in arrays have no place to put a reference count. Ref counted arrays would 
hence
become a library type, based on a ref counted class with overloaded operators for
the array operations.

==Results==

This is a very flexible approach, allowing for support of general RC objects, as 
well
as specific support for COM objects and Objective-C ARC. AddRef()/Release()'s 
implementation
is entirely up to the user or library writer.

@safe code can be guaranteed to be memory safe, as long as AddRef()/Release() 
are correctly
implemented.