Slices and GC

[Dmitry Olshansky]

On 05.04.2012 20:35, BLM wrote:
> On Thursday, 5 April 2012 at 15:30:45 UTC, Vladimir Panteleev wrote:
>
>>
>> The GC can't really know which parts of the array you're using. For
>> example, your only reference to the array might be a pointer, and you
>> might be traversing the array in either direction, only keeping count
>> of the remaining bytes until the array boundary.
>>
>> Consider .dup-ing the slices you're going to need, or using std.mmfile
>> to map the file into memory - in that case, the OS won't load the
>> unnecessary parts of the file into memory in the first place.
>
> I had considered using .dup, but I wanted to minimize overhead. I should
> probably look into std.mmfile or pull the data out in smaller chunks
> that the GC can handle individually.

Another idea is to copy out interesting parts of the original chunk to a 
separate storage array. This array will contain your sliced-out data 
just packed more tightly.
If you have a upper bound on % of useful bytes then you can get away 
without extra allocations.

The tricky part is reallocating this storage array, as it will make 
slices that point to it dangling (and keeping GC from deallocation), a 
workaround would be to use pure index-based "slices" that work on this 
block only.

> If the GC can distinguish between pointers and slices, it should
> theoretically be able to prune an array that is only referenced by
> slices, but I'm not sure how well that would fit into the current GC
> system.


-- 
Dmitry Olshansky