either me or GC sux badly (GC don't reuse free memory)

[Steven Schveighoffer via Digitalmars-d]

On 11/12/14 6:04 AM, ketmar via Digitalmars-d wrote:
> Hello.
>
> let's run this program:
>
>    import core.sys.posix.unistd;
>    import std.stdio;
>    import core.memory;
>
>    void main () {
>      uint size = 1024*1024*300;
>      for (;;) {
>        auto buf = new ubyte[](size);
>        writefln("%s", size);
>        sleep(1);
>        size += 1024*1024*100;
>        buf = null;
>        GC.collect();
>        GC.minimize();
>      }
>    }
>
> pretty innocent, right? i even trying to help GC here. but...
>
>    314572800
>    419430400
>    524288000
>    629145600
>    734003200
>    core.exception.OutOfMemoryError@(0)
>
> oooops.
>
> by the way, this is not actually "no more memory", this is "i'm out of
> address space" (yes, i'm on 32-bit system, GNU/Linux).
>
> the question is: am i doing something wrong here? how can i force GC to
> stop eating my address space and reuse what it already has?
>
> sure, i can use libc malloc(), refcounting, and so on, but the question
> remains: why GC not reusing already allocated and freed memory?
>

I think I might know what's going on.

You are continually adding 100MB to the allocation size. Memory is 
contiguous from the OS, but can get fragmented inside the GC.

So let's say, you allocate 300MB. Fine. It needs more space from the OS, 
allocates it, and assigns a pool to that 300MB. Now, you add another 
100MB. At this point, it can't fit into the original pool, so it 
allocates another 400MB. BUT, it doesn't merge the 300MB into that (I 
don't think), so when it adds another 100MB, it has a 300MB space, and a 
400MB space, neither of which can hold 500MB. And it goes on and on. 
Keep in mind also that it is a frequent error that people make to set a 
pointer to null and expect the data will be collected. For example, buf 
could still be in a register.

I would be interested in how much memory the GC has vs. how much is 
actually used.

-Steve