Thoughts on parallel programming?

[Russel Winder]

On Thu, 2010-11-11 at 02:24 +0000, jfd wrote:
> Any thoughts on parallel programming.  I was looking at something about Chapel
> and X10 languages etc. for parallelism, and it looks interesting.  I know that
> it is still an area of active research, and it is not yet (far from?) done,
> but anyone have thoughts on this as future direction?  Thank you.

Any programming language that cannot be used to program applications
running on a heterogeneous collection of processors, including CPUs and
GPUs as computational devices, on a single chip, with there being many
such chips on a board, possibly clustered, doesn't have much of a
future.  Timescale 5--10 years.

Intel's 80-core, 48-core and 50-core devices show the way server,
workstation and laptop architectures are going.  There may be a large
central memory unit as now, but it will be secondary storage not primary
storage.  All the chip architectures are shifting to distributed memory
-- basically cache coherence is too hard a problem to solve, so instead
of solving it, they are getting rid of it.  Also the memory bus stops
being the bottleneck for computations, which is actually the biggest
problem with current architectures.

Windows, Linux and Mac OS X have a serious problem and will either die
or be revolutionized.  Apple at least recognize the issue, hence they
pushed OpenCL.

Actor model, CSP, dataflow, and similar distributed memory/process-based
architectures will become increasingly important for software.  There
will be an increasing move to declarative expression, but I doubt
functional languages will ever make the main stream.  The issue here is
that parallelism generally requires programmers not to try and tell the
computer every detail how to do something, but instead specify the start
and end conditions and allow the runtime system to handle the
realization of the transformation.  Hence the move in Fortran from lots
of "do" loops to "whole array" operations.

MPI and all the SPMD approaches have a severely limited future, but I
bet the HPC codes are still using Fortran and MPI in 50 years time.

You mentioned Chapel and X10, but don't forget the other one of the
original three HPCS projects, Fortress.  Whilst all three are PGAS
(partitioned global address space) languages, Fortress takes a very
different viewpoint compared to Chapel and X10.

The summary of the summary is:  programmers will either be developing
parallelism systems or they will be unemployed.

<shameless-plug>
To hear more, I am doing a session on all this stuff for ACCU London
2010-11-18 18:30+00:00
http://skillsmatter.com/event/java-jee/java-python-ruby-linux-windows-are-all-doomed
</shameless-plug>

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder at ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel at russel.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20101111/b27f1baa/attachment.pgp>