Memory issues. GC not giving back memory to OS?

[Cristian Becerescu]

Hi!

A little bit of context first:

I was using DPP and I noticed huge amounts of RAM being used.
So I used valgrind massif and found out that 98% of the process’ 
memory (~6GB) was allocated for arrays / Appender with mmap.

I then performed a simple test where I incrementally appended 
2^30 integers (4GB) to a dynamic array (memory measurements are 
the same for Appender).
-> Memory used (peak; increasing towards the end of execution): 
~7GB
-> capacity == 1.107 * size (at the end of the program)

This is a bit odd, because 1.107 * 2^30 is roughly 4.4GB, and the 
peak memory consumption was 7GB. Apparently, the GC can correctly 
collect the memory when manually calling collect() at the end of 
appending, but that memory (we are talking 7 - 4.4 = 2.6GB) is 
never given back to the system. At least this is our intuition 
after making those observations.

I have created a gist with the test code and results (thanks Edi 
for augmenting the test code to profile the GC): 
https://gist.github.com/cbecerescu/e6606a8530c56ae06c52e5b1cd32b31f

Just some notes:
- if reserving 2^30 elements for the array (or Appender) 
beforehand, memory peaks are at 4GB
- C++'s std::vector, without reservation, never gets beyond 4GB 
and has size == capacity at the end