Parallel programming

[Markus Koskimies]

On Tue, 15 Jul 2008 19:34:24 -0400, bearophile wrote:

> How much time do we have to wait to see some parallel processing
> features in D? People are getting more and more rabid because they have
> few ways to use their 2-4 core CPUs. Classic multithreading is useful,
> but sometimes it's not easy to use correctly.

A very short answer; for true parallel processing, 2-4 processors is 
nothing. The success of CFL (Control-Flow Languages) like C, C++, D, 
Pascal, Perl, Python, BASICs, Cobol, Comal, PL/I, whitespace, malbolge, 
etc. etc. is that they follow the underlaying paradigm of computer.

There has been many efforts to declare languages, that are implicitly 
parallel. The most used approach is to use DFL (Data-Flow Language) 
paradigms, and the most well-know of these is definitely VHDL. Others are 
e.g. NESL and ID. Then there are several languages that are either in-
between like functional programming languages (Haskell, Erlang) or 
reductive languages (like make and Prolog).

Short references:

http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=714561
http://portal.acm.org/citation.cfm?id=359579&dl=GUIDE
http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=630241

Especially Hartenstein's articles are good to read, if you are trying to 
understand, why we are still using CFL & RASP, and why parallel 
architectures have failed.

No, the future will show us not any more parallelism at source level. 
Instead, (a) the compilers start to understand source better, to 
parallelize inner kernels of loops automatically, and (b) there will be 
even more layers between source we are writing and the instructions/
configurations processors are executing, and thus the main purpose of 
source language is not any more to follow the underlaying paradigm, but 
productivity - how easy it is to humans to express things; and CFL-
languages are far from their counterparts in this area. Comparing CFL/DFL 
at compiler level, see e.g.

http://csdl2.computer.org/persagen/DLAbsToc.jsp?resourcePath=/dl/
proceedings/&toc=comp/proceedings/
fccm/1995/7086/00/7086toc.xml&DOI=10.1109/FPGA.1995.477423


If I would asked to say what is the way of writing future programs, I 
would say it is MPS (Message Passing Systems), refer to e.g. Hewitt's 
Actor Model (1973). Furthermore, I would predict processors to start to 
do low-level reconfigurations, e.g. RSIP (Reconfigurable Instruction Set 
Processor) -paradigm. Look google for GARP and the awesome performance 
increasements it can offer for certain tasks.