Problems with GC, trees and array concatenation

[Paolo Invernizzi]

Oskar Linde wrote:
> Paolo Invernizzi skrev:
>> Oskar Linde wrote:
>>
>>> There will almost always be some spurious pointers. Allocating large 
>>> arrays (such as what happens behind the scenes when doing array 
>>> concatenation) will always carry a risk of allocated memory being hit 
>>> by a spurious pointer.
>>
>> I don't understand. Where are the spurious pointers coming in the 
>> above example?
> 
> Stack data (and registers) for instance. Possibly other sections of data 
> from the runtime or libraries.

Again, I don't understand. The only library imported in the example is 
the std.stdio, but I guess that the leak are not coming from that 
(Babele, can you confirm? I'm on OS X...).

And, as I don't see how the above program can put on the stack something 
  similar to a pointer (everything it's null, or IT'S a pointer), it 
remains only the runtime... *sigh*

Are really the registers to be considered as a possible font of leaks?

>>> Is the cause of the memory leak, because it allocates a huge chunk of 
>>> memory that is left to the GC to free. The chance of those huge 
>>> chunks being hit by a spurious pointer seems quite high.
>>
>> This is a nonsense. What's the meaning of having a garbage collector 
>> if, huge or not huge, it cannot free a block of memory is a so pretty 
>> linear example? For sure, I'm missing something...
> 
> The GC is not guaranteed to free all blocks of memory, but the usage 
> patterns in most applications make the amount of noncollectable memory 
> bound by a constant factor.

I simply cannot see how the usage pattern above can produce leak.
Must we avoid dynamic array concatenation?

> I was not able to reproduce the unbounded memory growth in the original 
> example, and if it is really there, it seems to me it may very likely be 
> a bug, as the program, as you say, never should produce any considerable 
> amount of "random" data.
> 
> Unbounded growth should normally only appear if the noncollected memory 
> blocks themselves contain spurious pointers. I am not too familiar with 
> the D GC and allocator, but as far as I know, care is taken to make sure 
> no uninitialized (random) data is introduced to the GC.

Yep, that's the point! I can understand it's better to keep under 
control big chunks of data full of random values, but  it seems to me 
that the fact nobody can understand in an easy way that Babele example 
WILL leaks it's a little frightening.

> I'd be interested to hear on what platform the original sample gives 
> unbounded memory leakage.

DMD 1.014 on XP, I guess..

Cheers, Paolo