GC conservatism -- again

[Steven Schveighoffer]

On Wed, 29 Dec 2010 01:00:01 -0500, Vladimir Panteleev  
<vladimir at thecybershadow.net> wrote:

> On Wed, 29 Dec 2010 05:13:10 +0200, Vladimir Panteleev  
> <vladimir at thecybershadow.net> wrote:
>
>> when the total size of pointers in the managed heap
>
> Correction: total size of *stray* pointers. This is a very important  
> distinction. If you have over 300 MB of stray pointers in your managed  
> heap, then it's almost certain that some binary data is being marked as  
> having pointers. Common culprits are void[] allocations, and large  
> structs/classes that mix pointers and non-pointers.

This is cool stuff.  This is what I normally was considering to be the  
problem with conservative GC.  However, I discovered that the reverse  
relationship also causes problems -- if you have a large block *not*  
containing pointers (say, a char[]), the chances that some 4-byte part of  
a pointer-containing struct "points" to it is relatively high as well.   
The probability of a random integer pointing to a 200MB block is 1/20  
(logically, 200MB is 1/20 of 2^32).  I think this might be a worse problem  
than the unlikely scenario that you allocate 200 or 300 MB of void[] data,  
since that isn't very common.  Essentially, you have no recourse but to  
manually manage that memory, there isn't a magic bullet (such as  
reallocating as ubyte[]).

Especially when it is frowned upon to forcefully deallocate memory...

-Steve