GC and slices

[Sean Kelly]

Lionello Lunesu wrote:
>> When you try to append to a slice a reallocation occurs--growing an 
>> array in place is only possible if your array reference refers to the 
>> head of the memory block.  Also, a rellocation will occur on an append 
>> periodically as the as the block grows.  Up to 4096 bytes (the size of 
>> a memory page) reallocations will occur when the array passes a power 
>> of two boundary--16, 32, 64, etc--beyond 4096 bytes a reallocation 
>> will occur when the array passes a page boundary.  This has to do with 
>> how the GC manages memory, and it is a pretty typical allocator design.
> 
> Hi Sean,
> 
> Thanks for the explanation.. I was thinking about it some more and 
> indeed forgot to test the pointer value after the append :S Pretty 
> stupid of me.
> 
> But a thought occured. Reallocation often involves a copy, but why 
> should that copy be necessary? I mean, the CPU (286+ anyway) has this 
> mapping table from physical to virtual memory. Wouldn't it be possible 
> to allocate a piece of virtual memory, but to let the 'old' pages refer 
> to the old physical memory so no copy is needed? Is this possible in 
> Windows/Linux?

Hrm... Assuming the array was already at least 2048 bytes in length (so 
  it would be living on its own page) and the page(s) allocated could be 
mapped contiguously with the page the array is on then this could work 
as you described.  Currently however, the GC code isn't quite this 
sophisticated.  If the append needs additional memory then an 
allocation/copy occurs regardless of array size or location.  Also, the 
internal GC.realloc routine only works if you pass a pointer to the head 
of a memory block, not a slice.  I'll look into whether it would be 
worthwhile to fix realloc to work regardless.  From there, someone could 
extend realloc further to grow large arrays in place if possible.  I'm 
not sure it's worth the effort, personally, though I suppose it could be 
if an application frequently appends to huge arrays.

> (In windows there's VirtualAlloc, which can be used to grow an existing 
> virtual memory block but I pretty sure that there's a memory copy if 
> that block can not be grown when a new block of virtual address space is 
> reserved.)

I think the allocation and copy would probably occur separately and 
manually within the GC code instead of relying on VirtualAlloc to take 
care of it.


Sean