Large Arrays and GC

dsimcha dsimcha at yahoo.com
Wed May 7 21:41:04 PDT 2008


After further testing, I've found an exact threshold for this bug.  When an array
of uints gets to 48_693_248 bytes (12_173_312 elements) this problem occurs, after
26 iterations at the threshold, or less for larger arrays.  Anything below that,
even one element smaller, and memory usage is stable over at least hundreds of
iterations.  It appears that the number of bytes is the key, since a ulong[] will
allow the same number of bytes (1/2 the elements) before causing problems, and a
ushort[] will allow twice as many elements (same number of bytes) without
crashing.  Furthermore, using equivalent sized floats instead of ints (float
instead of uint, double instead of ulong) or using signed ints, has no effect.

== Quote from dsimcha (dsimcha at yahoo.com)'s article
> Because of some difficulties encountered using D for large arrays (See previous
> posts about array capacity fields), I produced the following test case that seems
> to be a reproducible bug in D 2.0.13.  The following program keeps allocating a
> huge array in a function and then letting all references to this array go out of
> scope.  This should result in the array being freed as soon as more memory is
needed.
> import std.stdio, std.gc;
> void main(){
>     uint count=0;
>     while(true) {
>         test();
>         fullCollect();
>         writefln(++count);
>     }
> }
> void test() {
>     uint[] stuff=new uint[15_000_000];
> }
> This produced an out of memory error after 21 iterations on my machine w/ 2 GB of
> RAM.  Using an array size of 10_000_000 instead of 15_000_000, its memory usage
> stabilized at 350 megs, which seems rather large since a uint[10_000_000] should
> only use 40 megs, plus maybe another 40 for overhead.  Furthermore, it takes
> several iterations for the memory usage to reach this level.  Using a larger array
> size, such as 100_000_000, made this test run out of memory after even fewer
> iterations.  Furthermore, changing from a uint[] to real[] with a size of
> 15_000_000 made it run out after 8 iterations instead of 21.




More information about the Digitalmars-d mailing list