[Issue 5623] New: Slow GC with large heaps

[d-bugmail at puremagic.com]

http://d.puremagic.com/issues/show_bug.cgi?id=5623

           Summary: Slow GC with large heaps
           Product: D
           Version: D2
          Platform: Other
        OS/Version: Windows
            Status: NEW
          Keywords: patch, performance
          Severity: normal
          Priority: P2
         Component: druntime
        AssignedTo: nobody at puremagic.com
        ReportedBy: dsimcha at yahoo.com


--- Comment #0 from David Simcha <dsimcha at yahoo.com> 2011-02-20 12:10:57 PST ---
Created an attachment (id=918)
The patch.

Here's a GC benchmark and its performance on the stock GC.

import std.stdio, std.datetime, core.memory, std.conv;

void main(string[] args) {
    if(args.length < 2) {
        stderr.writeln("Need size.");
        return;
    }

    immutable mul = to!size_t(args[1]);
    auto ptr = GC.malloc(mul * 1_048_576, GC.BlkAttr.NO_SCAN);

    auto sw = StopWatch(autoStart);
    GC.collect();
    immutable msec = sw.peek.msecs;
    writefln("Collected a %s megabyte heap in %s milliseconds.",
        mul, msec);
}

Outputs for various sizes (pre-patch):

Collected a 10 megabyte heap in 1 milliseconds.
Collected a 50 megabyte heap in 4 milliseconds.
Collected a 200 megabyte heap in 16 milliseconds.
Collected a 500 megabyte heap in 41 milliseconds.
Collected a 1000 megabyte heap in 80 milliseconds.
Collected a 5000 megabyte heap in 397 milliseconds.
Collected a 10000 megabyte heap in 801 milliseconds.
Collected a 30000 megabyte heap in 2454 milliseconds.
Collected a 50000 megabyte heap in 4096 milliseconds. 

Attached is a patch that creates separate large and small object pools, only
stores the GC bits for the large object pool at pagesize offsets, not 16-byte
offsets, and stores, rather than computes, offsets for B_PAGEPLUS pages.  So
far, all unit tests for Phobos, dstats and std.parallelism/parallelfuture pass
with this enabled.  

Here are the new benchmarks (post-patch):

Collected a 10 megabyte heap in 0 milliseconds.
Collected a 50 megabyte heap in 0 milliseconds.
Collected a 250 megabyte heap in 1 milliseconds.
Collected a 500 megabyte heap in 0 milliseconds.
Collected a 1000 megabyte heap in 1 milliseconds.
Collected a 5000 megabyte heap in 3 milliseconds.
Collected a 10000 megabyte heap in 6 milliseconds.
Collected a 30000 megabyte heap in 16 milliseconds.
Collected a 50000 megabyte heap in 26 milliseconds.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------