Thoughts on parallel programming?

[Fawzi Mohamed]

On 11-nov-10, at 20:10, Russel Winder wrote:

> On Thu, 2010-11-11 at 18:24 +0100, Tobias Pfaff wrote:
> [ . . . ]
>> Unfortunately I only know about the standard stuff, OpenMP/OpenCL...
>> Speaking of which: Are there any attempts to support lightweight
>> multithreading in D, that is, something like OpenMP ?
>
> I'd hardly call OpenMP lightweight.  I agree that as a meta-notation  
> for
> directing the compiler how to insert appropriate code to force
> multithreading of certain classes of code, using OpenMP generally  
> beats
> manual coding of the threads.  But OpenMP is very Fortran oriented  
> even
> though it can be useful for C, and indeed C++ as well.
>
> However, given things like Threading Building Blocks (TBB) and the
> functional programming inspired techniques used by Chapel, OpenMP
> increasingly looks like a "hack" rather than a solution.

I agree I think that TBB offers primitives for many parallelization  
kinds, and is more clean and flexible than OpenMP, but in my opinion  
it has a big weakness: it cannot cope well with independent tasks.  
Coping well wit both nested parallelism and independent tasks is a  
crucial thing to have a generic solution that can be applied to  
several problems.
This is missing as far as I know also from Chapel.
I think that having a solution that copes well with both nested  
parallelism and independent tasks is an excellent starting on which to  
build almost all other higher level parallelization schemes.
It is important to handle this centrally, because the number of  
threads that one should spawn should ideally stay limited to the  
number of  execution units.