D2 weak references

[Jason House]

Sean Kelly Wrote:

> == Quote from Jason House (jason.james.house at gmail.com)'s article
> > Sean Kelly Wrote:
> > >
> > > You're guaranteed that the GC will notify you when it finalizes
> > > the referenced object, so you shouldn't have to query the GC
> > > at all.  Just copy the reference into a GC-scannable location
> > > and then if you haven't been notified by the GC that the object
> > > was finalized the reference is good.
> > Maybe I misunderstood, but I interpreted earlier discussion in this thread to mean the gc did the
> following steps:
> > 1. Pause all threads
> > 2. Scan objects to clear marks
> > 3. Scan objects to add marks
> > 4. Resume all threads
> > 5. Scan objects, if no mark then...
> >     5a. Call object finalizer
> >     5b. Free memory for later use
> > Assuming that is correct, any deref of a weak reference is unsafe between steps 4 and 5a. Even
> though the object may be valid in that window, there's no guarantee that the reference won't be used
> after step 5b... This would lead to random behavior and seg faults. No self-respecting weak reference
> library should allow that to happen.
> > I hope I got something wrong and weak refs are easier to implement than I'm currently thinking.
> 
> Oh I see what you're saying.  So this could happen:
> 
> 4. The GC resume all threads
> 4a. Another thread derefs the WeakRef
> 4b. This thread checks to see if the GC has notified that the
>        object is being collected.  No notification, so it uses the object.
> 5. The GC scans its memory and finalizes the object
> 6. The other thread tries to use the object and explodes
> 
> You're right, it sounds like you'd need to add a step between 4a
> and 4b where the WeakRef notifies the GC that the object it's using
> shouldn't be collected (if a collection is in progress).  In fact, all you
> really need to do is acquire the GCs mutex for a moment.  Once
> you've acquired it then you know that any collection in progress
> must be complete (assuming of course that the GC uses a mutex
> to prevent corruption).
> 
> The horrible thing about all this is it makes using a multithreaded
> WeakRef horribly slow.  Even if the GC facility were provided for
> what you want to do you're talking about acquiring a mutex in
> the GC for every deref operation.  What a mess.

I was hoping for a lock-free query that would return garbage (without crashing) in the rare case that a race occurred. If I get garbage (and says the object is marked) then the finalizer has run when I recheck the weak ref.  Do all druntime queries acquire the GC lock? I'm hoping to do about 100_000+ of these operations per second per core (and do other work too).

As long as memory blocks are not released to the OS while all threads are unpaused, lock-free operations should be possible.

As a side note, does druntime use multiple threads for the garbage collector? Scalability with Tango and a memory intensive application was horrible last time I tried renting an 8 core system. It may have been my fault, but the garbage collector is a convenient scape goat.