shifting array slices

[Steven Schveighoffer]

On Sat, 11 Feb 2012 14:24:53 -0500, H. S. Teoh <hsteoh at quickfur.ath.cx>  
wrote:

> On Sat, Feb 11, 2012 at 11:02:43AM -0800, Jonathan M Davis wrote:
>> On Saturday, February 11, 2012 10:51:36 H. S. Teoh wrote:
>> > This brings up an interesting point: what does the GC do if you have  
>> an
>> > array that's continually appended to, and also shrunk from the front?
>> > That is:
>> >
>> > 	ubyte[] buf;
>> > 	while ( ... ) {
>> > 		consumeData(buf[0]);
>> > 		buf = buf[1 .. $];
>> >
>> > 		ubyte x = getMoreData();
>> > 		buf ~= x;
>> > 	}
>> >
>> > Will such a program "leak memory" in the sense that the memory chunk
>> > allocated to buf will grow larger and larger, even though its
>> > initial segment is never accessed again? Or is the GC smart enough
>> > to only reallocate the needed size when the memory chunk is moved
>> > (at some point when it has no more space to append x)?
>> >
>> > If the GC is smart enough, then this kind of construct could be used
>> > instead of copying overlapping ranges.
>>
>> The memory blocks don't grow. When an array no longer has enough room
>> to increase its size, and you append to it, a new block is allocated,
>> and the array is moved to there. Any slices which refer to the
>> original block still refer to it. And when a garbage collection cycle
>> is run and a memory block has no references to it, it's collected. The
>> only "leaking" that occurs is that once no more slices refer to a
>> particular portion of a memory block, that portion is no longer
>> accessible until the block has been recycled.
>
> I know that. What I was asking is how the GC handles memory block
> reallocations: does it copy the original block in its entirety, or just
> the part referenced by the slice being appended to.
>
>
>> If you haven't read this article yet, you really should:
>>
>> http://www.dsource.org/projects/dcollections/wiki/ArrayArticle
> [...]
>
> After reading this article, I think it's clear that only the portion of
> the block referenced by buf gets copied, correct? And it gets copied to
> the beginning of the new block, so there is no wasted space in the new
> block and the old block will simply be GC'd. Cool.

Yes, most likely.  However, there is one case where it doesn't --  
extending pages.

If an array slice can be extended by consuming the next allocated page  
(this is only for page-sized blocks and above), then instead of copying  
the data, it will just mark the next page as part of the block, and  
continue on.  In this case, the block is not "copied" at all, so it cannot  
deallocate the no-longer referenced data.

This is probably something that can be detected.  For instance, if you are  
appending to a slice of length 5 and its at the end of a 10,000 element  
array, it makes little sense to not copy that data (allowing the original  
block to be freed by the GC).  But the situations are always different.

-Steve