GC Precision

[dsimcha]

== Quote from Andrei Alexandrescu (SeeWebsiteForEmail at erdani.org)'s article
> Sean Kelly wrote:
> > dsimcha Wrote:
> >
> >> == Quote from Sean Kelly (sean at invisibleduck.org)'s article
> >>> dsimcha Wrote:
> >>>> I just realized last night that D's templates are probably powerful enough now
> >>>> to generate bit masks that can be used for precise GC heap scanning.  I'm
> >>>> halfway (emphasis on halfway) thinking of using this to try to hack the GC and
> >>>> make heap scanning fully precise except for the corner case of unions.
> >>>> However, this ties into several things that others in the D community are
> >>>> doing, so I want to gauge people's responses and make sure I'm not wasting
> >>>> effort on something that will be useless in 6 months.
> >>>>
> >>>> 1.  Sean, Leonardo, whoever else may be working on GC implementations, have
> >>>> you by any chance broken ground on precise heap scanning already?
> >>> I've thought about it, but not done anything about it.  The compiler doesn't
> >> provide this information, so precise scanning would require a user-level call.
> >> You'll also have to deal with arrays of structs, by the way.
> >>
> >> Arrays of structs are easy:  Generate a bitmask for one element, and keep reusing
> >> that bitmask until the end of the block.  Am I missing something?
> >
> > Nope.
> One question is, is there enough information for stack variables? My
> understanding from a while ago was that heap data could be reasonably
> analyzed, but stack data has no info associated with it.
> Andrei

That's why I said precise *heap* scanning.  This would solve probably 90+% of the
problem w/ false pointers without requiring any changes with major ripple effects,
i.e. only druntime, not the compiler, would need to be hacked.  Admittedly,
though, unless you came up w/ some pinning scheme for stack variables and unions,
it still wouldn't allow a moving GC.

I personally am much more interested in a decent solution to our GC woes now than
a perfect one at some point indefinitely far into the future.  Right now, when
working with programs that use more than maybe 100-200 MB of memory, false
pointers become such a problem that the GC is almost useless, yet all kinds of
library code still uses the GC heap, which is why I resisted the idea of removing
GC.free() so strongly.  As I see it, the biggest problem is false pointers, with
the fact that every allocation requires a lock in a close second.  These are the
low-hanging fruit.  A moving GC, one that doesn't stop the world on collection,
and one that's fully precise including stack would be nice, but they're several
orders of magnitude less important and would also have more ripple effects.