draft proposal for ref counting in D

[Walter Bright]

Michel Fortin wrote:


Le 26-juin-2013 à 1:36, Walter Bright  a écrit :

 > On 6/25/2013 6:09 PM, Michel Fortin wrote:
 >> Instead, I think you should create a @noarc attribute for functions: it'll 
prevent the compiler for inserting any of those calls so it becomes the 
responsibility of the author to make those calls (which are then allowed). 
@noarc would be incompatible with @safe, obviously.
 >
 > It's a good point, but adding such an attribute to the function may be too 
coarse, for one, and may cause composition problems, for another. Maybe just 
disallowing it altogether is the best solution.

Well, it's mostly required to write runtime support functions. The attribute 
could be more obscure so people are less tempted to use it, but if you're going 
to implement the ref-counting code you'll need that.

 >> ## More on reference counting
 >>
 >> I feel like I should share some of my thoughts here about a broader use of 
reference counting in D.
 >>
 >> First, we don't have to assume the reference counter has to be part of the 
object. Apple implements reference counting using global hash tables where the 
key is the address. It works very well.
 >>
 >> If we added a hash table like this for all memory allocated from the GC, 
we'd just have to find the base address of any memory block to get to its 
reference counter. I know you were designing with only classes in mind, but I 
want to point out that it is possible to reference-count everything the GC 
allocates if we want to.
 >
 > D would need manual, RC and GC to coexist peacefully.

The problem is how to make the three of those use the same codegen?

- Druntime could have a flag to disable/enable refcounting. It'd make the 
retain/release functions no-ops, but it'd not prevent the GC from reclaiming 
memory as it does today.
- Druntime could have a flag to disable/enable garbage collection (it already 
has). That'd prevent cycles from being collected, but you could use weak 
pointers to work around that or request a collection manually at the appropriate 
time.
- A @noarc (or similar) attribute at the function level could be used to prevent 
the compiler from generating function calls on pointer assignments. You could 
make a whole module @noarc if you want by adding "@noarc:" at the top.

Here's the annoying thing: @noarc is totally safe if reference counting is 
disabled and we rely entirely on the GC. @noarc is unsafe when reference 
counting is enabled.

 >> The downside is that every assignment to a pointer anywhere has to call a 
function. While this is some overhead, it is more predictable than overhead from 
a GC scan and would be preferred in some situation (games I guess). Another 
downside is you have an object retained by being present on the stack frame of a 
C function, it'd have to be explicitly retained from elsewhere.
 >
 > Doesn't this make it impractical to mix vanilla C with D code? An important 
feature of D is this capability, without worrying about a "JNI" style interface.

It's not very different than with the GC today.

If you call a C function by giving it a ref-counted pointer argument, that 
memory block is guarantied to live at least for that call's lifetime (because it 
is retained by the caller). So simple calls to C functions are not a problem.

If the C function puts that pointer elsewhere you'll need to retain it some 
other way, but you have to do this with the GC too. If you're implementing a 
callback called from C you need to care about what you return because the 
caller's C code won't retain it, while with the GC you could manage if C code 
did not store that pointer outside of the stack.

I think that's all you have to worry about.

 > As for D switching to a full refcounted GC for everything, I'm very hesitant 
for such a step. For one thing, reading the clang spec on all the various 
pointer and function annotations necessary is very off-putting.

Don't let Clang intimidate you. The Clang spec is about four to five time more 
complicated than needed because of autoreleased objects and because it supports 
weak pointers. Weak pointers can be implemented as a struct templates (as long 
as we have @noarc). And all those annotations are for special cases, when you 
need to break the rules. You don't use them when doing normal programming, well 
except for __weak.