Problems with GC, trees and array concatenation

[Babele Dunnit]

Oskar Linde Wrote:

> The GC is not guaranteed to free all blocks of memory, but the usage 
> patterns in most applications make the amount of noncollectable memory 
> bound by a constant factor.

That is good theory, but this means that this version of DMD is unuseful for production software. HOW MUCH is that upper bound? Will it be enough to have 2 GB of RAM? or 4 GB? I am running a genetic algorithm, which ideally should run "forever"... nonetheless, I wrote some Java SW (which I basically dont like) which is running server-side since last 3 years... OK, OK, we all know how Java works, and the Java GC is more mature, and Sun has a lot of money, blah blah.. 

Thinking at it again, a 10-children individual and two 200-individual population will amount to 1.480KB when properly garbage collected, but will raise to a (relatively) impressive 31MB when using array concatenation (I stopped it after 700.000 iterations). Try It Yourself. And it seems to me that these are not *big* arrays. And all pointers are null. And GC works OK when you do *NOT* use concatenation (even on "real" trees, I can assure you... I got some 300-nodes, 20-level-deep trees in my program here, and when not using concatenation everything gets collected smoothly)

So, I should say, there is a bug in ARRAYS CONCATENATION, not in GARBAGE COLLECTION... Array concatenation introduces something in memory layout so that, later, the garbage collection cannot reclaim that memory zone...

> I'd be interested to hear on what platform the original sample gives 
> unbounded memory leakage.

Windows XP, DMD 1.014 (point 5 of my original post)

>From another post:

>So in conclusion, I am quite sure the following line:
>              indi[] = testPop1.individuals ~ testPop2.individuals;
>Is the cause of the memory leak, because it allocates a huge chunk of
>memory that is left to the GC to free. The chance of those huge chunks
>being hit by a spurious pointer seems quite high.

again: if you DO NOT concatenate those two arrays, everything is properly garbage collected. So, why on earth the GC should be able to collect the "original" arrays and not the "copied" ones?? And it happens also with very small vectors (see above), and still I do not see from where those spurious pointer should come from (holy smoke, Walter was so kind to make D initialize everything...)

Seems to me that DEFINITELY there is something in array concatenation code which fools the GC - the Windows version of it, I mean...

Ciao,
Babele