C interface provides a pointer and a length... wrap without copying?

[Jonathan M Davis via Digitalmars-d-learn]

On Sunday, March 12, 2017 02:47:19 cy via Digitalmars-d-learn wrote:
> On Saturday, 11 March 2017 at 23:43:54 UTC, Nicholas Wilson wrote:
> > A string *is* a pointer length pair, an immutable(char)[].
>
> Yes, but surely there's some silly requirement, like that the
> pointer must only ever point to garbage collected memory, or
> something?

No. A dynamic array is just a struct with a pointer and a length. Aside from
avoiding accessing memory that is no longer valid or avoiding allocations,
it doesn't matter one whit what memory it points to.

char[5] a;
char[] b = a;

is perfectly valid as is slicing memory that comes from malloc or some other
crazy place. Most dynamic array operations don't even need to care about
what memory they refer to. If you access an element, it does the math and
dereferences it like you'd get with naked pointer arithmetic. The difference
is that it also does bounds checking for you, because it knows the length.
If you slice the array, then you just get an array with an adjusted pointer
and/or length.

The only times that the GC gets involved are when doing anything involving
appending. If you call capacity, reserve, or ~=, then the GC gets involved.
In those cases, the GC looks at the pointer to determine whether it can
append. If it finds that it's GC-allocated memory, then it will look to see
what room there is in the GC-allocated block after the array. If you're
calling capacity, it will just return how large the array can grow to
without being reallocated. If you're calling reserve or ~= it checks to see
whether the capacity is great enough to grow into that space. If not, it
will allocate a new block of memory, copy the array's elements into that,
and set the array to point to it. In the case of ~=, it will also grow the
array into that memory and put the new elements in it, whereas in the case
of reserve, it just does the reallocation. If there was enough space, then
~= will just expand the array into that space without reallocating, and
reserve will do nothing.

If you have a dynamic array that was not allocated by the GC (be it a slice
of a static array or malloc-ed memory or whatever), then its capacity will
be 0. So, capacity will tell you 0, and ~= and reserve will always result in
the array being reallocated.

I would suggest that you read this excellent article:

http://dlang.org/d-array-article.html

though I would point out that it uses the wrong terminology in that it
refers to the GC-allocated buffer as the dynamic array rather than T[], and
it refers to T[] as a slice, whereas the official terminology is that T[] is
the dynamic array and that buffer doesn't have an official name, and while
T[] _is_ a slice of memory (assuming that it's not null), slice refers to a
lot of other stuff in D (e.g. slicing a container gives you range over that
container, so it's a slice of the container, but it's not T[]). And your use
case here is a perfect example of why T[] is the dynamic array and not the
GC-allocated buffer.

Dynamic arrays simply don't care about the memory that they refer to,
because they don't manage their own memory. ~=, reserve, and capacity do
care about what memory a dynamic array refers to, but they work exactly the
same way regardless of what the dynamic array refers to. It's just a
question of when reallocations do or do not occur.

The big concern with slicing static arrays or malloc-ed memory to get a
dynamic array is that it's then up to you to ensure that that dynamic array
does not outlive the memory that it refers to. So, there is some danger
there, but that's no different from operating on raw pointers, and operating
on dynamic arrays gives better safety thanks to bounds-checking, and it also
works with appending, though that will cause the dynamic array to then refer
to GC-allocated memory instead of the original memory.

- Jonathan M Davis