D2 weak references

[Jason House]

Leandro Lucarella Wrote:

> Jason House, el 22 de abril a las 21:26 me escribiste:
> > I'm finally looking at the gc source code.  From what I can tell, memory is 
> > allocated in large chunks (pools), and divided into 16 byte chunks.  
> > 
> > Attributes are not all packed together like the BlkInfo of the spec implies, 
> > so some so access to that stuff is safer than I would have thought.
> 
> I'm sorry you had to go through that, I think this could have save you
> a lot of time =)
> http://proj.llucax.com.ar/blog/dgc/blog/tag/understanding%20the%20current%20gc

It really wasn't that bad of a way to familiarize myself. I had enough of an idea about how it coulf work already. It is helpful to compare what I expect in a gc verses what's actually in the gc. I appreciate the link and will make sure to read it.


 
> > Here's a weak ref implementation that I just whipped together under the 
> > assumption it goes into druntime/src/gc/basic/gcx.d.  I think it is sane.  I 
> > haven't scoured the gc's code to ensure everything is legit.  I declared it 
> > as a class mostly because value semantics don't make much sense and I'm too 
> > lazy/tired to do the sample code as a 100% correct struct definition.
> > 
> > My two biggest concerns: 
> > 
> > 1. Is the gc lock held while running a constructor? If not and it's needed 
> > to do what it needs, where can the hook be added to make the construction of 
> > a weak reference special?
> 
> I don't think it's held, I just can't see how it can be held because
> construction is not done in the GC.
> 
> If you need to acquire the GC lock I think you can just do:
> synchronized (GC.gcLock) {
> 	// protected stuff
> }
> The GC lock is a static member of the GC struct in gcx.d.

Efficiency is a big concern of mine with this because I'm going to use it heavily. Locking twice is undesirable, but I may do it in an initial implementation. A lock on deref is unacceptable though.


 
> > 2. Will the assignment to noscan get overwritten / hit a race condition?
> 
> I don't think that should happen except if someone calls
> gc_setAttr/gc_getAttr().
> 
> > It is possible to hide the pointer inside a size_t instead, but given the
> > intimate tying to the GC such tricks just make the code tougher to read.
> 
> Agree. Maybe you can overload the operator new and pass the flags directly
> at allocation time, but I don't know very well how this is plugged in the
> runtime/GC, but since it's a regular gc_malloc() call it should be
> thread-safe.

Interesting idea. I'll look into that.



> Something like:
> 
> class weakRef(T) if (is(T==class))
> {
> 	new(size_t size)
> 	{
> 		return gc_malloc(size, FINALIZE | NO_SCAN);
> 	}
> 	this(T _t)
> 	{
> 		// set up tracking for the weakly referenced object
> 		t = _t;
> 		pool = findPool(t);
> 		rt_attachDisposeEvent(&t, &sweep);
> 	}
> 	// ...
> }
> 
> 
> > class weakRef(T) if (is(T==class))
> > {
> >     this(T _t){
> >         // set up tracking for the weakly referenced object
> >         t = _t;
> >         pool = findPool(t);
> >         rt_attachDisposeEvent(&t, &sweep);
> > 
> >         // tell the gc to not scan the weak reference
> >         weakRefPool = findPool(&this);
> >         weakRefPool.noscan[index(weakRefPool,&this)] = true;
> 
> You can do this by calling setAttr(&this, NO_SCAN).

True.

 
> >     }
> > 
> >     T opCall(){
> >         T tmp = t; // force future conservative GC scan to find the pointer
> >         if ( (tmp is null) || (pool.mark[index] == false) ){
> 
> pool.mark.test(index) is used in the GC code, I don't know if the array
> syntax works. And to use the mark bit trick, I think something in the
> lines of my patch is needed to ensure the mark bit is set for all live
> objects.
> 
> >             t = null;
> >             return null;
> >         }
> >         tmp = t; // avoid race where t was replaced with new data
> 
> I don't understand exactly what kind of race are you trying to avoid with
> this (and how this assignation can avoid any race if there is one).

Consider this sequence:
1. GC runs mark phase and leaves object unmarked.
2. Weakref starts dereferencing and reads its hidden pointer.
3. Sweep calls finalizer and deallocates
4. A new object is allocated in the old location
5. Query for GC mark says we're ok
6. Weakref returns its old copy of the pointer
7. Pointer is used
8. Random behavior


 
> >         return tmp;
> >     }
> > 
> > private:
> >     T t;
> >     Pool* pool;
> >   
> >     size_t index(Pool *p, byte *element){ return (element-p)/16; }
> >     size_t index(){ return index(pool,&t); }
> > 
> >     void sweep( Object o ){
> >         t = null;
> >         pool = null;
> >     }
> > }
> 
> 
> 
> -- 
> Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
> ----------------------------------------------------------------------------
> GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
> ----------------------------------------------------------------------------
> Lo último que hay que pensar es que se desalinea la memoria
> Hay que priorizar como causa la idiotez propia
> Ya lo tengo asumido
> 	-- Pablete, filósofo contemporáneo desconocido