Unofficial wish list status.(Jul 2008)

[Me Here]

Robert Fraser wrote:

> Me Here Wrote:
> > Web servers and other IO-bound processes are not the target of
> > multithreading.
> 
> How do functional programming styles apply better to I/O-bound systems? 
> I couldn't imagine writing a query processor using monads for all I/O.
> 

No. You missed my (badly worded) point. I perhaps should have said:

	Of course, webservers and other IO-bound processes...

I was referring back to my earlier point that "Side-effects are par for the
course in software.". And the fact that many, maybe even a majority of strong
applications for threading aren't IO-bound, which makes it /seem/ as if
functional techniques lend themselves to the transparent threading of
compute-bound processing.

But, as I then pointed out, the problem with the way D2 handles the mutation of
arrays (strings and others), is that it flies in the face of those bread&butter
concurrency algorithms that manipulate large datasets (images and the like)
with add&multiply algorithms, by setting multiple threads to operate on
different segments of the same datastructure. These would be completely
hamstrung by the unnecessary and uncontrolled replication.

The best way to handle concurrent access to shared resources, is to avoid
concurrent access. For IO and other external shared resources, you serialise
access to them through queuing or message passing to a single thread that acts
on the behalf of others.

For memory, you avoid concurrent access by partitioning the dataset. Ie, you
pass each thread a constrained alias to just that part of the dataset you wish
it to operate upon. Each thread can then operate upon its partition at full
speed, without locking, and without restrictions upon the operations it can
perform. Replicating entire arrays everytime something mutates an element
within it, is a complete nonsense. It basically throws away the very advantage
of having shared memory in the first place. And shared memory is the very
reason detre of threading.

I love functional techniques. Perl's map, grep, reduce and kin are one of its
greatest assets, but using purity to attempt to simplify threading fails
dismally. It creates work and impacts performance for *everyone not using
threads*, and those using them whenever they *don't need the protection*, in
order to "simplify" things for those that are using threads and know they would
otherwise need to take precautions.

It's like drawing solid white lines across major roads before every side
turning and on-ramp, to force the main flow to check if someone wants to turn
on to the major road, so that those turning onto the major road don't have to
check if anything is coming. The priorities are backwards.

This isn't a rail against functional techniques or languages. Its an informed
reality check to weight the many costs against the few benefits, and to
consider the rarity of the occasions when those few benefits will actually
occur.

I'll say it again. D was my great white hope for compiled language performance
and integrity checking, rational semantics and threading. But this issue, and
to some degree, the whole templating thing, seem to have side-tracked the
development process and the original clean vision of D and (IMO) are leading it
off in directions that are utopic in vision, but imprqactical in reality.

b.
--