Feature suggestion: in-place append to array

[Steven Schveighoffer]

On Thu, 04 Mar 2010 12:43:32 -0500, grauzone <none at example.net> wrote:

> Steven Schveighoffer wrote:
>> On Thu, 04 Mar 2010 11:43:27 -0500, grauzone <none at example.net> wrote:
>>
>>> Some sort of "resetAndReuse" function to clear an array, but enabling  
>>> to reuse the old memory would be nice:
>>>
>>> int[] a = data;
>>> a = null;
>>> a ~= 1; //reallocates (of course)
>>> a.length = 0;
>>> a ~= 1; //will reallocate (for safety), used not to reallocate
>>> resetAndReuse(a);
>>> assert(a.length == 0);
>>> a ~= 1; //doesn't reallocate
>>>
>>> This can be implemented by setting both the slice and the internal  
>>> runtime length fields to  0.
>>>
>>> Additionally, another function is necessary to replace the old  
>>> preallocation trick:
>>>
>>> //preallocate 1000 elements, but don't change actual slice length
>>> auto len = a.length;
>>> a.length = len + 1000;
>>> a.length = len;
>>>
>>> As I understood it, this won't work anymore after the change. This can  
>>> be implemented by enlarging the array's memory block without touching  
>>> any length fields.
>>>
>>> I'm sure the function you had in mind does one of those things or both.
>>  proposed usage (as checked in a couple days ago):
>>  int[] a;
>> a.setCapacity(10000); // pre-allocate at least 10000 elements.
>> foreach(i; 0..10000)
>>    a ~= i; // no reallocation
>> a.length = 100;
>> a.shrinkToFit(); // resize "allocated" length to 100 elements
>> a ~= 5; // no reallocation.
>
> What shrinkToFit() does is not really clear. Does it reallocate the  
> memory block of the array such, that no space is wasted? Or does it  
> provide (almost) the same functionality as my resetAndReuse(), and make  
> the superfluous trailing memory available for appending without  
> reallocation?

Sorry, should have added:

assert(a.length == 101);

Basically, shrinkToFit shrinks the "allocated" space to the length of the  
array. To put it another way, you could write your resetAndReuse function  
as follows:

void resetAndReuse(T)(ref T[] arr)
{
   arr.length = 0;
   arr.shrinkToFit();
}

I want to avoid assuming that shrinking the length to 0 is the only usable  
idiom.

>
> I think a resetAndReuse is really needed. I have found it can prevent  
> "GC thrashing" in many cases. E.g. when caching frequently re-evaluated  
> data in form of arrays (free previous array, then allocate array that  
> isn't larger than the previous array).

Yes, it is useful in such cases.  The only questionable part about it is  
that it allows for stomping in cases like:

auto str = "hello".idup;
auto str2 = str[3..4];
str2.shrinkToFit();
str2 ~= "a";
assert(str == "hella");

So it probably should be marked as unsafe.

Again, the name shrinkToFit isn't my favorite, ideas welcome.

-Steve