Getting started with threads in D

[Russel Winder]

On Sun, 2012-06-17 at 03:15 +0200, Henrik Valter Vogelius Hansson wrote:
> Hi again!
> 
> I have looked around a little with what D offers but don't know 
> really what I should use since D offers several ways to use 
> threads. Some more high level than others. Don't really also know 
> which one would be suitable for me.

My take on this is that as soon as an applications programmer talks
about using threads in their program, they have admitted they are
working at the wrong level.  Applications programmers do not manage
their control stacks, applications programmers do not manage their
heaps, why on earth manage your threads. Threads are an implementation
resource best managed by an abstraction.

Using processes and message passing (over a thread pool, as you are
heading towards in comments below) has proven over the last 30+ years to
be the only scalable way of managing parallelism, so use it as a
concurrency technique as well and get parallelism as near as for free as
it is possible to get.

Ancient models and techniques such as actors, dataflow, CSP, data
parallelism are making a resurgence exactly because explicit shared
memory multi-threading is an inappropriate technique. It has just taken
the world 15+ years to appreciate this.

> A little background could help. I am a game developer and during 
> my semester I want to experiment with making games in D. I use 
> threads to separate some tasks that can easily work in parallel 
> with each other. The most common being a Logic/Graphics 
> separation. But as development progresses I usually add more 
> threads like inside graphics I can end up with 2 or 3 more 
> threads.

I can only repeat the above: don't think in terms of threads and shared
memory, think in terms of processes and messages passed between them.

> I want to avoid Amdahl's law as much as possible and have as 
> small synchronization nodes. The data exchange should be as basic 
> as possible but still have room for improvements and future 
> additions.

Isn't the current hypothesis that you can't avoid Amdahl's Law? If what
you mean is that you want to ensure you have an embarrassingly parallel
solution so that speed up is linear that seems entirely reasonable, but
then D has a play in this game with the std.parallelism module.  It uses
the term "task" rather than process or thread to try and enforce an
algorithm-focused view. cf. http://dlang.org/phobos/std_parallelism.html

> The Concurrency library looked very promising but felt like the 
> synchronization wouldn't be that nice but it would provide a 
> random-access to the data in your code. Correct me of course if I 
> am wrong. Is there a good thread pool system that could be used? 
> Does that system also handle solving dependencies in the 
> work-flow? This is what we use at my work more or less.

What makes you say synchronization is not that nice?

Random access, data, threads and parallelism in the same paragraph
raises a red flag of warning!

std.concurrency is a realization of actors so there is effectively a
variety of thread pool involved. std.parallelism has task pools
explicitly. 

> In worst case scenario I will just use the basic thread class and 
> implement my own system above that. Then there is the question, 
> is there any pitfalls in the current library that I should be 
> aware of?

I am sure D's current offerings are not perfect but they do represent a
good part of the right direction to be travelling.  What is missing is a
module for dataflow processing(*) and one for CSP.  Sadly I haven't had
time to get stuck into doing an implementation as I had originally
planned 18 months or so ago: most of my time is now in the Python and
Groovy arena as that is where the income comes from.  cf. GPars
(http://gpars.codehaus.org) and Python-CSP – though the latter has
stopped moving due to planning a whole new Python framework for
concurrency and parallelism.


(*) People who talk about "you can implement dataflow with actors and
vice versa" miss the point about provision of appropriate abstractions
with appropriate performance characteristics.
 
-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder at ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel at winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://lists.puremagic.com/pipermail/digitalmars-d-learn/attachments/20120617/ee9de6af/attachment.pgp>