either me or GC sux badly (GC don't reuse free memory)

ketmar via Digitalmars-d digitalmars-d at puremagic.com
Wed Nov 12 08:06:22 PST 2014


On Wed, 12 Nov 2014 10:51:31 -0500
Steven Schveighoffer via Digitalmars-d <digitalmars-d at puremagic.com>
wrote:

> On 11/12/14 6:04 AM, ketmar via Digitalmars-d wrote:
> > Hello.
> >
> > let's run this program:
> >
> >    import core.sys.posix.unistd;
> >    import std.stdio;
> >    import core.memory;
> >
> >    void main () {
> >      uint size = 1024*1024*300;
> >      for (;;) {
> >        auto buf = new ubyte[](size);
> >        writefln("%s", size);
> >        sleep(1);
> >        size += 1024*1024*100;
> >        buf = null;
> >        GC.collect();
> >        GC.minimize();
> >      }
> >    }
> >
> > pretty innocent, right? i even trying to help GC here. but...
> >
> >    314572800
> >    419430400
> >    524288000
> >    629145600
> >    734003200
> >    core.exception.OutOfMemoryError@(0)
> >
> > oooops.
> >
> > by the way, this is not actually "no more memory", this is "i'm out of
> > address space" (yes, i'm on 32-bit system, GNU/Linux).
> >
> > the question is: am i doing something wrong here? how can i force GC to
> > stop eating my address space and reuse what it already has?
> >
> > sure, i can use libc malloc(), refcounting, and so on, but the question
> > remains: why GC not reusing already allocated and freed memory?
> >
> 
> I think I might know what's going on.
> 
> You are continually adding 100MB to the allocation size. Memory is 
> contiguous from the OS, but can get fragmented inside the GC.
> 
> So let's say, you allocate 300MB. Fine. It needs more space from the OS, 
> allocates it, and assigns a pool to that 300MB. Now, you add another 
> 100MB. At this point, it can't fit into the original pool, so it 
> allocates another 400MB. BUT, it doesn't merge the 300MB into that (I 
> don't think), so when it adds another 100MB, it has a 300MB space, and a 
> 400MB space, neither of which can hold 500MB. And it goes on and on. 
> Keep in mind also that it is a frequent error that people make to set a 
> pointer to null and expect the data will be collected. For example, buf 
> could still be in a register.
> 
> I would be interested in how much memory the GC has vs. how much is 
> actually used.
i posted the second samle where i'm doing `GC.free()` to reclaim
memory. as i said, RES is jumping between "almost nothing" and "several
GB", as sample allocates and frees. but VIRT is growing constantly.

i believe that GC just can't merge segments, so it keep asking for more
and more address space for new segments, leaving old ones unused and
unmerged. this way GC has alot of free memory, but when it can't
allocate another segment, it throws "out of memory error".

if i'll use libc malloc() for allocating, everything works as i
expected: address space consumtion is on par with allocation size.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20141112/eb6cd173/attachment-0001.sig>


More information about the Digitalmars-d mailing list