Where is my memory?

[anonymous via Digitalmars-d]

On Sunday, 22 March 2015 at 09:42:41 UTC, Ozan Süel wrote:
> Hi!
> I'm working on a Big Data project, where a huge amount of RAM 
> is needed. Using D I've run into a - let's called it - memory 
> leak. I tested this with following code:
>
> 	foreach(i; 0..1000) {
> 		int[] ints;
> 		foreach(j; 0..1000) {
> 			ints ~= uniform(0, 100);
> 		}
> 	}
>
> Without the first foreach this code use only 220KB,
> with the first foreach this code needs 2,5MB.
> (220KB x 1000 is something around 2,5MB).

220KB x 1000 would be around 200MB.

> But why is GC (Garbage Collector) not running? Following the 
> explanations in http://wiki.dlang.org/Memory_Management memory 
> usage should be something around  220KB.

The GC is running. 1000 x 1000 ints would be 4MB. That alone is 
more than the 2.5MB you're observing.

> I used GC.minimize, slow down the loop, replaced 
> uniform...doesn't work.
> (By the way: After a while my LINUX server killed my big data 
> project because running out of RAM & SWAP space)

GC.minimize alone can't do anything if the memory isn't 
collected. Use GC.collect(), too.

Also, if you know the final size of the array beforehand, you can 
`reserve` it. That avoids a lot of copying (and garbage creation) 
when appending.

		int[] ints;
		ints.reserve(1000); /* <- */
		foreach(j; 0..1000) {
			ints ~= uniform(0, 100);
		}