draft proposal for ref counting in D

[Walter Bright]

On 6/26/2013 2:33 PM, Rainer Schuetze wrote:
 > On 26.06.2013 11:38, Walter Bright wrote:
 >>
 >> On 6/26/2013 12:19 AM, Rainer Schuetze wrote:
 >>>
 >>> I imagine a few (constrained) templated functions for the different
 >>> operations defined in the library could also do the job, though it
 >>> might drown compilation speed. Also getting help from the optimizer to
 >>> remove redundant calls will need some back doors.
 >>
 >> I don't see how this can be done without specific compiler knowledge in
 >> a memory safe way.
 >
 > I currently don't see how it can be memory safe with this proposal.

I'm a little confused about what you are referring to here.


 >
 >>>> 3. Assignment to a class reference causes a call to AddRef() on the new
 >>>> value
 >>>> followed by a call to Release() on its original value.
 >>>
 >>> It might be common knowledge, but I want to point out that the usual
 >>> COM implementation (atomic increment/decrement and free when refcount
 >>> goes down to 0) is not thread-safe for shared pointers. That means you
 >>> either have to guard all reads and writes with a lock to make the full
 >>> assignment atomic or have to implement reference counting very
 >>> different (e.g. deferred reference counting).
 >>
 >> Since the implementation of AddRef()/Release() is up to the user,
 >> whether it uses locks or not and whether it supports shared or not is up
 >> to the user.
 >
 > You have to put the lock around the pair of AddRef and Release, but if the 
compiler already splits this into two function calls, this cannot be done in the 
implementation.

Why is it necessary to put a lock around the pair?

 >
 >>
 >>>> 12. AddRef() is not called when passed as the implicit 'this' reference.
 >>>>
 >>>
 >>> Isn't this unsafe if a member function is called through the last
 >>> existing reference and this reference is then cleared during execution
 >>> of this member function or from another thread?
 >>
 >> No. The caller of the function still retains a reference in that thread.
 >
 > Hmmm, I guess I misunderstand the proposal. Assume for example a refcounted 
class R and this code
 >
 > class R : RefCounted
 > {
 >     int _x;
 >     int readx() { return _x; }
 > }
 > int main()
 > {
 >     R r = new R;
 >     return r.readx();
 > }
 >
 > According to 12. there is no refcounting going on when calling or executing 
readx. Ok, now what happens here:
 >
 > class R : RefCounted
 > {
 >     int _x;
 >     int readx(C c)
 >     {
 >         c.r = null; // "standard" rc deletes r here
 >         return _x;  // reads garbage
 >     }
 > }
 > class C
 > {
 >     R r;
 > }
 > int main()
 > {
 >     C c = new C;
 >     c.r = new R;
 >     return c.r.readx(c);
 > }
 >
 > This reads garbage or crashes if there is no reference counting going on when 
calling readx.

I think you're right. Grrrr!

 >
 >>
 >>>
 >>>> 13. Taking the address of, or passing by reference, any fields of an RC
 >>>> object
 >>>> is not allowed in @safe code. Passing by reference an RC field is
 >>>> allowed.
 >>>
 >>> Please note that this includes slices to fixed size arrays.
 >>
 >> As I suggested, arrays would not be supported with this proposal - but
 >> the user can create ref counted array-like objects.
 >
 > Just to clarify, I meant taking a slice of a static array that is a field of 
a refcounted class. Is it forbidden to have a field like this in a refcounted 
class or is taking the address through slicing forbidden?

It would be forbidden to obtain a slice of a ref counted object in this way - or 
even to simply refer to a static array embedded in a ref counted object (in 
@safe code).


 >
 >>>
 >>> I feel I'm hijacking this proposal, but the step to library defined
 >>> read/write barriers seems pretty small. Make AddRef, Release and
 >>> assignment free template functions, e.g.
 >>>
 >>> void ptrConstruct(T,bool stackOrHeap)(T*adr, T p);
 >>> void ptrAssign(T,bool stackOrHeap)(T*adr, T p);
 >>> void ptrRelease(T,bool stackOrHeap)(T*adr);
 >>>
 >>> and we are able to experiment with all kinds of sophisticated GC
 >>> algorithms including RC. Eliding redundant addref/release pairs would
 >>> need some extra support though, I read that LLVM does something like
 >>> this, but I don't know how.
 >>>
 >>
 >> It's pretty invasive into the code generation and performance, and could
 >> completely disrupt the C compatibility of D.
 >
 > I don't see a big difference between a free function and a member function 
call, though the template character of it might hurt compilation performance.
 >
 > Two more notes:
 >
 > - I'm not sure it is mentioned, but I think you have to describe what happens 
when copying a struct. pre- and post-blit actions have to be taken if the struct 
contain pointers to refcounted objects.

Yes. It's analogous to copying a struct that has fields which contain 
constructors and destructors.


 >
 > > 10. Function returns have an AddRef() already done to the return value.
 >
 > - A refcounted reference returned from a function (including new) would have 
to be Released if the return value is ignored or if only used as part of an 
expression.
 >
 >
 >

That's right. Just as if a function returned a struct with a destructor.

The model I am basing this on is C++'s shared_ptr<>, which makes use of all the 
various rules around construction, assignment, and destruction. The wrinkles we 
have are:

1. memory safety
2. working with the GC