Builtin array and AA efficiency questions

[Random D user via Digitalmars-d-learn]

Ah missed your post before replying to H.S. Teoh (I should 
refresh more often).
Thanks for reply.

On Thursday, 15 October 2015 at 19:50:27 UTC, Steven 
Schveighoffer wrote:
>
> Without more context, I would say no. assumeSafeAppend is an 
> assumption, and therefore unsafe. If you don't know what is 
> passed in, you could potentially clobber data.
>
> In addition, assumeSafeAppend is a non-inlineable, runtime 
> function that can *potentially* be low-performing.

Yeah I know that I want to overwrite the data, but still that's 
probably a lot of calls to assumeSafeAppend. So I agree.

> instance, you call it on a non-GC array, or one that is not 
> marked for appending, you will most certainly need to take the 
> GC lock and search through the heap for your block.
>

What does marked for appending mean. How does it happen or how is 
it marked?

> The best place to call assumeSafeAppend is when you are sure 
> the array has "shrunk" and you are about to append. If you have 
> not shrunk the array, then the call is a waste, if you are not 
> sure what the array contains, then you are potentially stomping 
> on referenced data.

So assumeSafeAppend is only useful when I have array whose length 
is set to lower than it was originally and I want to grow it back 
(that is arr.length += 1 or arr ~= 1).

>
> An array uses a block marked for appending, assumeSafeAppend 
> simply sets how much data is assumed to be valid. Calling 
> assumeSafeAppend on a block not marked for appending will do 
> nothing except burn CPU cycles.
>
> So yours is not an accurate description.

Related to my question above.
How do you get a block not marked for appending? a view slice?

Perhaps I should re-read the slice article. I believe it had 
something like capacity == 0 --> always allocates. Is it this?

>
>> A.3) If A.2 is true, are there any conditions that it reverts 
>> to
>> original behavior? (e.g. if I take a new slice of that array)
>
> Any time data is appended, all references *besides* the one 
> that was used to append now will reallocate on appending. Any 
> time data is shrunk (i.e. arr = arr[0..$-1]), that reference 
> now will reallocate on appending.
>

Thanks. IMO this is very concise description of allocation 
behavior.
I'll use this as a guide.

> So when to call really sort of requires understanding what the 
> runtime does. Note it is always safe to just never use 
> assumeSafeAppend, it is an optimization. You can always append 
> to anything (even non-GC array slices) and it will work 
> properly.

Out of curiosity. How does this work? Does it always just 
reallocate with gc if it's allocated with something else?

>
> This is an easy call then:
>
> array.reserve(100); // reserve 100 elements for appending
> array ~= data; // automatically manages array length for you, 
> if length exceeds 100, just automatically reallocates more data.
> array.length = 0; // clear all the data
> array.assumeSafeAppend; // NOW is the best time to call, 
> because you can't shrink it any more, and you know you will be 
> appending again.
> array ~= data; // no reallocation, unless previous max size was 
> exceeded.
>

Thanks. This will probably cover 90% of cases.
Usually I just want to avoid throwing away memory that I already 
have.
Which is slow if it's all over your codebase.
Like re-reading or recomputing variables that you already have.
One doesn't hurt but a hundred does.

>> B.1) I have a temporary AA whose lifetime is limited to a 
>> known span
>> (might be a function or a loop with couple functions). Is 
>> there way to
>> tell the runtime to immeditially destroy and free the AA?
>
> There isn't. This reminds me, I have a lingering PR to add 
> aa.clear which destroys all the elements, but was waiting until 
> object.clear had been removed for the right amount of time. 
> Perhaps it's time to revive that.

Should array have clear() as well?
Basically wrap array.length = 0; array.assumeSafeAppend();
At least it would then be symmetric (and more intuitive) with 
built-in containers.

>
> -Steve