A smaller GC benchmark

[bearophile]

This tiny benchmark focus on GC performance and shows similar results to the "binary trees" benchmark I have shown here few weeks ago, but it's quite shorter, so it may be more useful. This code isn't meant to represent normal programs with thousands of classes, etc, it just shows something quite limited. This is an adaptation of the "Object Test" that you can find used and discussed here:
http://www.twistedmatrix.com/users/glyph/rant/python-vs-java.html
http://blog.snaplogic.org/?p=55
http://programming.reddit.com/info/24ynh/comments
Note that I test the GC of Phobos only, I don't use Tango yet.


Object Test timings (on PIII @ 500 MHz), best of 3 runs (seconds, approximate Mbytes memory used), n=1_000, m=10_000:
               seconds  MB
  DMD class:   18.95    1.7
  GDC class:   17.91    1.8
  DMD struct:  11.77    1.7
  GDC struct:  12.31    1.8
  Python:      37.10    3.1  (ShedSkin version)
  Psyco:       15.68    3.5
  ShedSkin      6.35    1.6
  Java (1):     2.19    7.3
  Java (2):     2.10    7.8

See below for the sources. You can see the Java is about 9 (~= 18.95 / 2.10) times faster than the program produced by DMD. Even Psyco (the JIT for the turtle-slow language Python) gives faster performance (and the Python GC is simple, it's just a reference count plus cycle detection). You can also see Java (as usual) uses much more RAM (7.8/1.7 ~= 4.6).

Note that with some manual GC optimization the running time for the DMD class benchmark can become about one-two seconds smaller, but the memory used can be about 4 MB:

import std.gc;
class ObjectTest { ObjectTest next; }
void main() {
    for (uint k = 0; k < 50; k++) {
        std.gc.disable();
        for (uint i = 0; i < 20; i++) {
            auto root = new ObjectTest();
            for (uint j = 0; j < 10_000; j++) {
                root.next = new ObjectTest();
                root = root.next;
            }
        }
        std.gc.fullCollect();
    }
}

-------------------

COMPILED/RUN WITH:
  gcc version 3.4.5 (mingw special) (gdc 0.24, using dmd 1.020)

  DMD v1.024

  gcc version 3.4.2 (mingw-special)

  javac 1.6.0_03
  java version "1.6.0_03"

  Python 2.5.1

  ShedSkin 0.0.25 (shedskin.sourceforge.net , a Python to C++ compiler)

-------------------

COMPILATION/RUNNING PARAMETERS:
  GDC used as:
    gdc -O3 -s -frelease -finline-functions -ffast-math -fomit-frame-pointer -funroll-loops -march=pentiumpro obj_test_class.d -o obj_test_class_gdc

  DMD used as:
    dmd -O -release -inline obj_test_class.d -ofobj_test_class_dmd.exe

  Java compiled with:
    javac ObjectTest.java

  Java (1) run with:
    java ObjectTest

  Java (2) run with:
    java -Xms20m -Xbatch ObjectTest

  SS and Psyco compiled and run without flags.

-------------------

SOURCES:

// obj_test_class.d
class ObjectTest { ObjectTest next; }
void main() {
    for (uint i = 0; i < 1_000; i++) {
        auto root = new ObjectTest();
        for (uint j = 0; j < 10_000; j++) {
            root.next = new ObjectTest();
            root = root.next;
        }
    }
}

-------------------

// obj_test_struct.d
struct ObjectTest { ObjectTest* next; }
void main() {
    for (uint i = 0; i < 1_000; i++) {
        auto root = new ObjectTest();
        for (uint j = 0; j < 10_000; j++) {
            root.next = new ObjectTest();
            root = root.next;
        }
    }
}

-------------------

// ObjectTest.java
public class ObjectTest {
    public ObjectTest next;
    public static void main(String[] args) {
        for (int i = 0; i < 1000; i++) {
            ObjectTest root = new ObjectTest();
            for (int j = 0; j < 10000; j++) {
                root.next = new ObjectTest();
                root = root.next;
            }
        }
    }
}

-------------------

# obj_test.py
from psyco.classes import __metaclass__
class ObjectTest: pass
def main():
    for i in xrange(1000): # N
        root = ObjectTest()
        for j in xrange(10000): # M
            root.next = ObjectTest()
            root = root.next
import psyco; psyco.full()
main()

-------------------

# obj_test_ss.py
class ObjectTest: pass
def main():
    for i in xrange(1000): # N
        root = ObjectTest()
        for j in xrange(10000): # M
            root.next = ObjectTest()
            root = root.next
main()


Bear hugs,
bearophile