large objects and GC

[Fawzi Mohamed]

On 2008-05-16 23:54:49 +0200, "Vladimir Panteleev" 
<thecybershadow at gmail.com> said:

> On Sat, 17 May 2008 00:32:26 +0300, Fawzi Mohamed <fmohamed at mac.com> wrote:
> 
>> The problem is that if the owner/object that do reference counting are 
>> managed by the gc they might stay uncollected for a quite long time 
>> because they are probably small, so one needs to really do everything 
>> manually, use scope,...
> 
> I don't understand your logic here. The GC does not prioritize objects 
> based on their size. Smaller objects are much less likely to leak 
> because of the proportionally smaller chance of a bogus pointer keeping 
> it "referenced".

If I understood correctly the actual gc (I looked at tango's one, but 
it seems that it is just a slighlty improved version of phobos gc) it 
doesn't know anything about single objects, it just works with pools.
This way the number of object it has to handle stays manageable.
If an object is big it gets its own pool, whereas if it is small it 
gets in a pool with other objects.
Now the pool will stay around as long as any objet into it has references.
When the pools goes away all the finalizers are called and then memory 
is released (not necessarily to the system, but at least to the gc).

This behavior is ok as long as the size of the object is the one the gc 
sees, if the pools have a reasonable size the memory loss stays 
reasonable.
But now look at the typical use of an array initialized from other 
arrays through calculations that need temporary arrays.
Using small wrappers both the result and the temporary arrays are 
likely to be in the same pool.
So as long as the result is kept around all the temporaries used to 
create it stay around.
It is clear that if the temporary actually have a large amount of 
manual allocated memory this result is a waste of resources.
The result is that you don't have to just manually manage the big 
memory allocation, but also the wrappers.
I know that with big objects manual management is probably a good idea, 
but I would like a system that can work reasonably well with a more 
relaxed management.
I think that my proposal achieves this with a small change.

>> One could to add a flag to the garbage collector. This flag would say 
>> to the gc to ignore inner pointer in a region when deciding if the 
>> region should be collected.
>> To have internal pointers one should also keep a pointer to the base object.
> 
> This will be effectively the same as having a "wrapper" class for 
> manually allocated memory. The class destructor, which will be called 
> by the GC, should deallocate the external memory. The wrapper class 
> should only have one field, and thus be very small and it will have the 
> chance of leaking almost equivalent to the method you describe. I see 
> no necessity for reference counting either, you just pass around the 
> reference to the wrapper object.

see the previous point.

>> (but the pointers should be updated when the region is moved)
> 
> A moving garbage collector must also be an exact garbage collector.

I know, it was just in case...

Fawzi