Memory leak in rare cases using foreach and parallel

[Guillaume Lathoud]

On Thursday, 17 October 2019 at 14:44:51 UTC, Guillaume Lathoud 
wrote:
> On Thursday, 17 October 2019 at 06:43:38 UTC, Seb wrote:
>
>> Do you have by any chance a small reproducible example?
>
> No. The use case was difficult to reduce, to reproduce the bug 
> in a usable manner
> If I manage to get one I'll definitely post it!

I still could not reproduce this in a small example.

But here is an additional hint that helped me as well, and could 
help someone having similar memory usage issues, in the case of 
running long tasks using `parallel()`: run

core.memory.GC.collect();
core.memory.GC.minimize();

at the end of each task. See the example below.

Best regards,
Guillaume
.

#!/usr/bin/env rdmd

import core.memory;
import core.thread;
import std.parallelism;
import std.range;
import std.stdio;

double[] do_one()
{
   auto arr = new double[ 50_000_000 ];

   Thread.sleep( 500.msecs );

   return arr;
}

void main()
{
   foreach (i; parallel( 100.iota ))
     {
       do_one; // Some long task (several seconds or minutes)

       // The next two lines divided the peak memory usage by a
       // factor of about 2 or 3
       core.memory.GC.collect();
       core.memory.GC.minimize();
     }

   writeln( "Done. Sleeping so that you can get the peak memory 
usage using `grep VmHWM /proc/<id>/status or similar..." );
   Thread.sleep( 10.seconds );
}