D Language Foundation Monthly Meeting Summary for December 2022

[Ali Çehreli]

On 1/23/23 12:06, H. S. Teoh wrote:

 > `.parallel` -- it's basically zero cost.  And for script-like helper
 > utilities, .parallel is just the thing you need to get the job done in
 > the shortest amount of time possible.  No need for anything more
 > elaborate.

Yes! :)

As a heads up to those who haven't tried it yet, there are cases that 
may benefit from reducing the work unit size from its default value of 100.

Especially when it's about file processing and there are a few files 
that take disproportionate amount of processing time, then the thread 
that is working on the largest file(s) would be holding on to 99 others 
to process them later in serial fashion. This may happen when the other 
e.g. 20 threads have already finished their tasks.

So, I recommend experimenting with smaller work unit sizes; I currently 
use 1 for such file processing. Something like this:

   auto tp = new TaskPool(totalCPUs / 2);   // Thread count
   foreach (e; tp.parallel(elements, 1)) {  // Work unit size
     // ...
   }
   tp.finish(); // Don't forget

as seen here:

   https://youtu.be/dRORNQIB2wA?t=1692

Ali