std.parallelism: How to wait all tasks finished?

[Russel Winder]

On Mon, 2014-02-03 at 00:00 +0000, Cooler wrote:
> I have several tasks. Each task may or may not create another 
> task. What is the best way to wait until all tasks finished?

What you are describing here is a classic fork/join architecture. The
tasks are structured as a tree with synchronization handled by the
sub-nodes. 

As far as I am aware std.parallelism focuses on data parallelism which
is a scatter/gather (aka map/reduce) model of just a single layer.

All the code fragments in the thread have, I believe, been predicated on
working with a thread pool as an explicit global entity. I think the
problems have stemmed from taking this viewpoint.

I would suggest following the way the Java fork/join framework (based on
Doug Lea's original) works. There is an underlying global thread pool,
but the user code uses the fork/join abstraction layer in order to
create the tree of synchronization dependencies. In this case instead of
working with tasks directly there needs to be a type whose job it is to
be a non-leaf node in the tree that handles synchronization whilst
nonetheless creating tasks and submitting them to the pool.

This is clearly something that could turn into an addition to
std.parallelism or be std.forkjoin.

Sorry I have no actual code to offer, but the overall design of what is
needed is well understood, at least in the Java context. C++ has a long
way to go to catch up, as does D.

The other thing that then sits on this is lazy stream parallelism, which
is what Java 8 is adding to the mix.

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder at ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel at winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder