draft proposal for ref counting in D - ReferenceType proposal

[Sönke Ludwig]

To support most of the requirements we need to offer some control over 
the reference type. Forcing the reference to be a pointer to the class 
instance precludes proper weak references and makes thread-safety difficult.

Rainer Schütze's proposal [1] looked promising, but didn't quite work 
out. However, by going a bit further, I think this approach can be fixed 
and will provide all the flexibility needed to implement solutions that 
can satisfy any of those requirements.

The basic idea is the same: Any reference to a class that is recognized 
as being reference counted is replaced by a struct that performs the 
reference counting using RAII (e.g. std.typecons.RefCounted). This 
allows any reference counting scheme to be implemented 
(internal/external, support weak refs or not, global counter table, GC 
memory or malloc, etc.).

TL;DR Let some code speak instead of a full blown spec first:

struct RefCounted(T) { /* ... */ }

// @referenceType!RefCounted
class C {
	// the presence of a C.ReferenceType template makes the class a
	// reference counted class
	// Note: A @referenceType UDA as above defined in object.d
         //       could be a cleaner/more explicit alternative
	alias ReferenceType = RefCounted;

	// C is now internally renamed to __ref_C to avoid ambiguities
	// and "C" itself is an alias for ReferenceType!__ref_C
	pragma(msg, C.stringof); // "RefCounted!__ref_C"

	void method()
	{
		// The "this" pointer, too, is of the reference counted
		// type. caveats: how to handle private fields? COM call
  		// convention?
		pragma(msg, typeof(this)); // "RefCounted!__ref_C"

		// Alternative: allow only pure functions to avoid
		// escaping references

		// Another alternative is to make 'scope' powerful
		// enough and use that:
		pragma(msg, typeof(this)); // "scope __ref_C"
	}
}

// The reference type itself is never const/immutable
// to enable reference counting for qualified types
C c; // -> RefCounted!__ref_C
const(C) d; // -> RefCounted!(const(__ref_C))

// To support the usual implicit conversions, some substitution is
// needed, since we have no implicit cast support for UDTs in the
// language
d = c; // d = typeof(D)(c)
        // or
        // d = typeof(D).implicitCastFrom(c)
        // or
        // d = typeof(C).implicitCastTo!D

// shared, however, is applied to the reference count itself (and
// transitively to the object) to force a thread-safety - or rather to
// avoid accidental use of unsafe implementations for shared references)
shared(const(C)) e; // -> shared(RefCounted!(const(__ref_C)))

---

Caveat: Changing the "this" pointer from '__ref_C' to 
'RefCounted!__ref_C' has implications on the calling convention, which 
needs to be taken into account when COM objects are involved. A simple 
COMPtr-like struct that only contains the target pointer may be enough 
here, though.

Also, to guarantee memory safety, some additional measures need to be 
taken to avoid escaping plain references to refcounted memory. One 
solution is the use of isolated/owned types, another is to make 'scope' 
not only check for shallow reference escaping, but also check for 
escaping of references to fields (similar thing but behaves 
differently). Both combined would of course be ideal. I think this is an 
issue that is mostly orthogonal to the refcount topic. See also the 
corresponding thread [2].

[1]: 
http://forum.dlang.org/thread/l34lei$255v$1@digitalmars.com?page=5#post-l352nk:242g3b:246:40digitalmars.com
[2]: http://forum.dlang.org/thread/kluaojijixhwigoujeip@forum.dlang.org