On Thu, Oct 15, 2015 at 04:47:35PM +0000, Random D user via Digitalmars-d-learn wrote:
> So I was doing some optimizations and I came up with couple basic
> questions...
>
> A)
> What does assumeSafeAppend actually do?
It adjusts the size of the allocated block in the GC so that subsequent
appends will not reallocate.
Basically, whenever you try to append to an array and the end of the
array is not the same as the end of the allocated GC block, the GC will
conservatively assume that somebody else has an array (i.e. slice) that
points to the data between the end of the array and the end of the
block, so it will allocate a new block and copy the array to the new
block before appending the new data. Calling assumeSafeAppend "shrink
fits" the allocated GC block to the end of the array, so that the GC
won't reallocate, but simply extend the block to accomodate the new
element.
> A.1) Should I call it always if before setting length if I want to
> have assumeSafeAppend semantics? (e.g. I don't know if it's called
> just before the function I'm in)
Probably, otherwise the GC may sometimes reallocate when you don't want
it to.
> A.2) Or does it mark the array/slice itself as a "safe append" array?
> And I can call it once.
Not that I know of.
> A.3) If A.2 is true, are there any conditions that it reverts to
> original behavior? (e.g. if I take a new slice of that array)
[...]
> What I'm trying to do is a reused buffer which only grows in capacity
> (and I want to overwrite all data). Preferably I'd manage the current
> active size of the buffer as array.length.
[...]
> There's just so much magic going behind d arrays that it's a bit
> cumbersome to track manually what's actually happening. When it
> allocates and when it doesn't.
> So, I already started doing my own Buffer type which gives me explicit
> control, but I wonder if there's a better way.
This is probably the best way to do it, since the built-in arrays do
have a lot of "interesting" quirks that probably don't really do what
you want.
The thought that occurs to me is that you could still use the built-in
arrays as a base for your Buffer type, but with various operators
overridden so that it doesn't reallocate unnecessarily. So you'd keep a
T[] as the underlying array, but keep track of .length separately and
override the ~ and ~= operators so that they update Buffer.length
instead of the .length of the underlying array. Only when Buffer.length
is greater than .length, you'd increment .length so that the GC will
reallocate as needed. Similarly, you might want to override the slicing
operators as well so that they also return Buffer types instead of T[],
so that the user doesn't accidentally get access to the raw T[] and
cause unnecessary reallocations.
> B.1) I have a temporary AA whose lifetime is limited to a known span
> (might be a function or a loop with couple functions). Is there way to
> tell the runtime to immeditially destroy and free the AA?
>
> I'd like to assist the gc with manually destroying some AAs that I
> know I don't need anymore. I don't really want to get rid of gc, I
> just don't want to just batch it into some big batch of gc cycle work,
> since I know right then and there that I'm done with it.
>
> For arrays you can do:
> int[] arr;
> arr.length = 100;
> delete arr; // I assume this frees it
Unfortunately, delete has been deprecated, and may not be around for
very much longer.
> but for AAs:
> int[string] aa;
> delete aa; // gives compiler error Error: cannot delete type int[string]
>
> I could do aa.destroy(), but that just leaves it to gc according to docs.
Perhaps what you could do is to trigger GC collection after setting the
AA to null:
aa = null; // delete references to GC data
GC.collect(); // run collection cycle to free it
I'm not sure if it's a good idea to run collection cycles too often,
though, it will have performance impact.
> Maybe I should start writing my own hashmap type as well?
If you want to manually delete data, you probably want to implement your
own AA based on malloc/free instead of the GC. The nature of GC doesn't
lend it well to manual management.
> B.2) Is there a simple way to reuse the memory/object of the AA?
>
> I could just reuse a preallocated temp AA instead of alloc/freeing it.
Not that I know of... unfortunately, the current AA implementation
doesn't allow overriding of the allocator; it's hardcoded to use the
default GC. This may change in the distant future, but I don't see it
happening anytime soon.
The only thing I can think of is to implement this manually, e.g., by
wrapping your AA in a type that keeps a size_t "generation counter",
where if any value in the AA is found to belong to a generation that's
already past, it pretends that the value doesn't exist yet. Something
like this:
struct AA(K,V) {
static struct WrappedValue {
size_t generation;
V value;
}
V[WrappedValue] impl;
size_t curGeneration = 1;
size_t length;
// Overload aa[key]
V opIndex(K key) {
auto p = key in impl;
if (p is null || p.generation < curGeneration)
throw RangeError(...);
return p.value;
}
// Overload aa[key] = value
void opIndexAssign(V value, K key) {
auto p = key in impl;
if (p is null || p.generation < curGeneration)
length++;
impl[key].generation = curGeneration;
impl[key].value = value;
}
// Overload key in aa
V* opBinary(string op : "in")(K key) {
auto p = key in impl;
if (p !is null && p.generation < curGeneration)
return null;
return &p.value;
}
// Overload aa.get(...);
V get(K key, V defaultValue) {
auto p = key in impl;
if (p is null || p.generation < curGeneration) {
return defaultValue;
}
return p.value;
}
// Overload aa.remove(...);
void remove(K key) {
auto p = key in impl;
if (p is null)
return;
// Mark entry as outdated so that it will be
// reused next time. Don't actually call
// impl.remove() so that GC won't collect it.
p.generation = 0;
}
@property bool empty() { return length == 0; }
void destroy() {
// Invalidate all existing entries
curGeneration++;
// N.B. doesn't set impl.length to 0 (which will
// cause reallocation).
length = 0;
}
}
Of course, this assumes that your keys will have a lot of overlap
between calls to .destroy, otherwise your AA will eat up a lot of memory
unnecessarily. If that's not the case, it's probably better to just let
the GC do its job, or implement your own AA with malloc/free.
T
--
Why waste time reinventing the wheel, when you could be reinventing the engine? -- Damian Conway