Array slices and copy on write

[Steven Schveighoffer]

On Mon, 04 Apr 2011 14:30:34 -0400, simendsjo <simen.endsjo at pandavre.com>  
wrote:

> On 04.04.2011 15:53, Steven Schveighoffer wrote:
>> 2. The D GC collects entire memory blocks, not portions of them. There
>> is no way for the GC to "collect" a[3]. It can only collect the entire
>> memory block which contains a.
>
> Given the above, I guessed so.
> Then you must copy data to a new array manually in case you want to free  
> excess memory..?
>
> int[] a;
> a.length = 10_000_000; // my initial guess
> // fill a
> a.length = 100; // only 100 used. GC won't ever free the memory not used
> auto b = a[0..100].dup; // copy
> a = null; // and destroy\

-or-

a = a[0..100].dup;

Yes, the GC/array code does not move the array to a smaller block when it  
could (note in the example above, the array code again cannot know that  
nothing has a reference to the full 10,000,000 elements).  Simply because  
the array runtime cannot know what you intend to do with that array.  It  
has that block allocated, it's not going to spend extra cycles trying to  
optimize storage for it just in case that's what you wanted.

People who shrink an array down to 100 then build it back up would  
(rightfully) be complaining if the runtime reallocated to smaller blocks  
on shrinking.

> Or is it possible to mark part this for the GC as unused?

The way the GC works, it *could* do this for arrays that span multiple  
pages (i.e. give back some of the pages to the GC) without moving the  
data, but there is no mechanism that I know of to do that.  Re-allocating  
is a good enough solution, I guess nobody has yet had a very good need to  
do anything else.

>> For those cases where you want to *force* the runtime to consider data
>> not referenced by an array as being 'free', you can use  
>> assumeSafeAppend:
>>
>> b.assumeSafeAppend(); // marks b[3] as being unused
>>
>> b.length += 1; // no copy made.
>
> I cannot find assumeSafeAppend.

It's in object.di (along with the documentation for it):

http://www.digitalmars.com/d/2.0/phobos/object.html#assumeSafeAppend

> Thanks a lot for your very thorough and detailed answer. Cleared up a  
> couple of blind spots.

No problem.  Arrays are a very tricky business in D, and I'm still finding  
new ways of looking at them :)  Having modified most of the array append  
code, I can say they are certainly one of the more deceptively complex  
parts of the runtime.

-Steve