GC Precision

[Andrei Alexandrescu]

dsimcha wrote:
> == Quote from Andrei Alexandrescu (SeeWebsiteForEmail at erdani.org)'s article
>> Sean Kelly wrote:
>>> dsimcha Wrote:
>>>
>>>> == Quote from Sean Kelly (sean at invisibleduck.org)'s article
>>>>> dsimcha Wrote:
>>>>>> I just realized last night that D's templates are probably powerful enough now
>>>>>> to generate bit masks that can be used for precise GC heap scanning.  I'm
>>>>>> halfway (emphasis on halfway) thinking of using this to try to hack the GC and
>>>>>> make heap scanning fully precise except for the corner case of unions.
>>>>>> However, this ties into several things that others in the D community are
>>>>>> doing, so I want to gauge people's responses and make sure I'm not wasting
>>>>>> effort on something that will be useless in 6 months.
>>>>>>
>>>>>> 1.  Sean, Leonardo, whoever else may be working on GC implementations, have
>>>>>> you by any chance broken ground on precise heap scanning already?
>>>>> I've thought about it, but not done anything about it.  The compiler doesn't
>>>> provide this information, so precise scanning would require a user-level call.
>>>> You'll also have to deal with arrays of structs, by the way.
>>>>
>>>> Arrays of structs are easy:  Generate a bitmask for one element, and keep reusing
>>>> that bitmask until the end of the block.  Am I missing something?
>>> Nope.
>> One question is, is there enough information for stack variables? My
>> understanding from a while ago was that heap data could be reasonably
>> analyzed, but stack data has no info associated with it.
>> Andrei
> 
> That's why I said precise *heap* scanning.  This would solve probably 90+% of the
> problem w/ false pointers without requiring any changes with major ripple effects,
> i.e. only druntime, not the compiler, would need to be hacked.  Admittedly,
> though, unless you came up w/ some pinning scheme for stack variables and unions,
> it still wouldn't allow a moving GC.
> 
> I personally am much more interested in a decent solution to our GC woes now than
> a perfect one at some point indefinitely far into the future.  Right now, when
> working with programs that use more than maybe 100-200 MB of memory, false
> pointers become such a problem that the GC is almost useless, yet all kinds of
> library code still uses the GC heap, which is why I resisted the idea of removing
> GC.free() so strongly.  As I see it, the biggest problem is false pointers, with
> the fact that every allocation requires a lock in a close second.  These are the
> low-hanging fruit.  A moving GC, one that doesn't stop the world on collection,
> and one that's fully precise including stack would be nice, but they're several
> orders of magnitude less important and would also have more ripple effects.

Absolutely! I think that's great work. Thanks for clarifying things for me.

Andrei