draft proposal for ref counting in D

[Walter Bright]

Rainer Schuetze wrote:

On 27.06.2013 02:11, Walter Bright wrote:
 >
 > On 6/26/2013 2:33 PM, Rainer Schuetze wrote:
 >> On 26.06.2013 11:38, Walter Bright wrote:
 >>>
 >>> On 6/26/2013 12:19 AM, Rainer Schuetze wrote:
 >>>>
 >>>> I imagine a few (constrained) templated functions for the
 >>>> different operations defined in the library could also do the
 >>>> job, though it might drown compilation speed. Also getting help
 >>>> from the optimizer to remove redundant calls will need some
 >>>> back doors.
 >>>
 >>> I don't see how this can be done without specific compiler
 >>> knowledge in a memory safe way.
 >>
 >> I currently don't see how it can be memory safe with this
 >> proposal.
 >
 > I'm a little confused about what you are referring to here.

When preparing for dconf I read the "Garbage Collection Handbook" by
Richard Jones, and it very much supported my suspicion that the usual
reference counting cannot be both memory safe and high-performance.

 >>
 >> You have to put the lock around the pair of AddRef and Release, but
 >> if the compiler already splits this into two function calls, this
 >> cannot be done in the implementation.
 >
 > Why is it necessary to put a lock around the pair?

To be more accurate, it is the assignment and the Release that have to
be atomic, in addition to a concurrent read with AddRef. Imagine the
reading thread is suspended while just having read the pointer, but not
incremented the reference count yet. If an assignment with release and deletion 
is performed before the thread resumes, AddRef is called on garbage.

IIRC you also have the GC handbook book on your shelf. Check the
chapters on RC, especially algorithm 18.2 "Eager reference counting with
CompareAndSwap is broken".


 >> Just to clarify, I meant taking a slice of a static array that is a
 >>  field of a refcounted class. Is it forbidden to have a field like
 >> this in a refcounted class or is taking the address through slicing
 >> forbidden?
 >
 > It would be forbidden to obtain a slice of a ref counted object in
 > this way - or even to simply refer to a static array embedded in a
 > ref counted object (in @safe code).

Ok.

 >> Two more notes:
 >>
 >> - I'm not sure it is mentioned, but I think you have to describe
 >> what happens when copying a struct. pre- and post-blit actions have
 >> to be taken if the struct contain pointers to refcounted objects.
 >
 > Yes. It's analogous to copying a struct that has fields which contain
 >  constructors and destructors.

Ok, I tend to forget about the swapping to a temporary when assigning structs.

 > The model I am basing this on is C++'s shared_ptr<>, which makes use
 > of all the various rules around construction, assignment, and
 > destruction. The wrinkles we have are:
 >
 > 1. memory safety

This is the hard part.

 > 2. working with the GC

I don't think that the GC is getting in the way as long as it is
mark-and-sweep. A fully reference counting GC is a different story and can be 
made concurrent, but it is usually not eager and needs to defer actual reference 
counting to avoid locks. Instead it logs accesses to some thread local buffer 
which needs to be processed eventually.