In-place extension of arrays only for certain alignment?

[Ali Çehreli]

Thank you for the quick response.

On 8/16/22 12:31, Steven Schveighoffer wrote:
 > On 8/16/22 2:11 PM, Ali Çehreli wrote:
 >> Related to my DConf 2022 lightning talk, I am noticing that D
 >> runtime's in-place array extension optimization is available only for
 >> array data that are at certain memory alignments.
 >>
 >
 > No, it's based on 2 factors:
 >
 > 1. Is it a page-size-or-greater block?

I assume the length of the new block.

 > 2. Is there a free page after it?

Makes sense.

 > The reason why your `bad` version fails is because when it must
 > reallocate, it still is only allocating 1 element.

That part I still don't understand. The same block of e.g. 16 bytes 
still has room. Why not use that remaining portion?

Here is a simpler test with better-than-expected results. Array c is 
what I want to do. In this test it works and I don't even need to call 
assumeSafeAppend() (this must be what you call "end slice"):

import std.stdio;

void main() {
   ubyte[] a;
   a ~= 0;
   a.length = 0;
   a.assumeSafeAppend();         // Needed
   assert(a.capacity == 15);

   // Essentially, the same as above
   ubyte[] b;
   b ~= 0;
   b = b[0..0];
   b.assumeSafeAppend();        // Needed
   assert(b.capacity == 15);

   ubyte[] c;
   c ~= 0;
   c = c[1..$];
   // c.assumeSafeAppend();
   assert(c.capacity == 14);
}

 > metadata (e.g. typeinfo for destruction and append capacity).

I think it is 16 bytes total (on 64 bits): void* + size_t and I see this 
when I print .ptr: The change is always 0x10.

 > Note that once you reach page size, the optimization can happen, *even
 > if you are only appending to an end slice*. Here's something to try:
 > when the capacity is less than the "magic" number of elements, `reserve`
 > that number of elements. Then the "drop one element each loop" should
 > use the optimization.

.reserve sounds promising but it will sometimes allocate memory and move 
elements even if I will not really need e.g. more than just one more 
element. (In my case, I may not know how many will be needed.) In other 
words, I really don't know how much to reserve.

What I seem to need is this function:

void increaseCapacityWithoutAllocating(T)(ref T[] arr) {
   // ...
}

Only the runtime seems to be able to implement that function. Can I call 
something in the runtime similar to how assumeSafeAppend() calls 
_d_arrayshrinkfit() in object.d?

 >
 > -Steve

Ali