pinning revisited

[Jason O'Brien]

It's not that any of us need to pin right now, but that the standard
puts anyone using C interop in a hard place: It's not possible to write
forward-compatible gc safe code.  We want to, but we can't, and so we
don't.  Noone is right now, and I'm afraid the more broken code we all
write the less likely it is a more efficient copying collector will ever
be worthwhile, and if it does get done all the more code we have to rewrite.

I have a couple of ideas, but whatever we decide on we should do it soon.

----------------------------------------------------------------------

1) void gcpin(void*) and void gcunpin(void*)
	These functions keep a block of memory from being moved by a copying
collector.  Right now, and to preserve speed in non-copying gc's, I
propose the dummy implementation:
Version(STATIC_GC) { void gcpin(void* ptr){} void gcunpin(void* ptr){} }

This has an extra benefit that if a compiler supports both a copying and
noncopying gc (for whatever reason), all that has to be changed is

Version(COPYING_GC) { //stuff here }
else { //empty versions }

There is a problem with pinned objects being collected though.
How to deal with this would be implementation specific, but wouldn't be
too hard. Either search the pin list as roots, or have roots and pinned
objects be one in the same (gcpin just resolving to addRoot).

Of course, this leaves alot of room for error.  This could lead to some
major memory leaks if you forget to unpin things after interop.  So I
propose a keyword (which is the reason gcpin is named the way it is):
pin(void* ptr)
{
	// interop functions requiring the pin go here
}
This would just resolve to
gcpin(ptr);
//interop functions;
gcunpin(ptr):
but gets rid of the chance to shoot yourself in the foot in most cases.
  You only need to use the manual pins when an object pointer is stored
on the C side, in which case the compiler couldnt know when to unpin it
so you have to do it yourself anyway.

----------------------------------------------------------------------

2)  A pin autoclass.
If combined with a using type scope statement for auto classes, you
could have nearly the same syntax:
using(Pin pin(void* ptr))
{
	// interop functions
}
The alternative being to just have a local pin, but that doesnt give you
  scope clearly telling you where it is and is not pinned, and also
requires the object be pinned until the scope ends (which is potentially
bad for memory consumption)

And if the C side is storing it, you could keep a pin as a class member.
  This is a bit more fool-proof than the option above, but I'm not sure
its even possible with auto classes as they are at the moment, and could
get clumsy if you are pinning more than one object.

One thing to worry about is keeping track of nesting.. (pin() pin()
unpin() should leave the object pinned until another call to unpin()),
though either way could implement this easily.

----------------------------------------------------------------------

Personally, I prefer option 1, the functions being a little easier to
work with when you need asynchronous pinning without worrying about
keeping members to active pins.  I don't see why both couldn't be
implemented, so people can choose whichever they like, as making an
autoclass to wrap the gcpin and gcunpin functions would be simple.

I'm not sure when it is valid for the GC to run, but in the interest of
callback and thread safety, I think it might be necessary for copying gc
implementations to automatically wrap any function call into C with the
appropriate pin.  This way, pointers wont change during the execution of
a C function.  This could easily be done transparently to the user, say:

extern(C) void cFunc(void* ptrNotStored);

then
cFunc(&myClass);
becomes
pin (&myClass) { cFunc(&myClass); }
for heap allocated objects.

Any more complex demands, such as the C side storing a pointer, the user
is responsible for pinning the appropriate data.

any thoughts?  I really think this is urgent, I don't want to see the D
community close itself off from a more efficient gc