Getting started with threads in D

Henrik Valter Vogelius Hansson groogy at groogy.se
Fri Jun 22 11:17:18 PDT 2012


On Sunday, 17 June 2012 at 07:23:38 UTC, Russel Winder wrote:
> On Sun, 2012-06-17 at 03:15 +0200, Henrik Valter Vogelius 
> Hansson wrote:
>> Hi again!
>> 
>> I have looked around a little with what D offers but don't 
>> know really what I should use since D offers several ways to 
>> use threads. Some more high level than others. Don't really 
>> also know which one would be suitable for me.
>
> My take on this is that as soon as an applications programmer 
> talks
> about using threads in their program, they have admitted they 
> are
> working at the wrong level.  Applications programmers do not 
> manage
> their control stacks, applications programmers do not manage 
> their
> heaps, why on earth manage your threads. Threads are an 
> implementation
> resource best managed by an abstraction.
>
> Using processes and message passing (over a thread pool, as you 
> are
> heading towards in comments below) has proven over the last 30+ 
> years to
> be the only scalable way of managing parallelism, so use it as a
> concurrency technique as well and get parallelism as near as 
> for free as
> it is possible to get.
>
> Ancient models and techniques such as actors, dataflow, CSP, 
> data
> parallelism are making a resurgence exactly because explicit 
> shared
> memory multi-threading is an inappropriate technique. It has 
> just taken
> the world 15+ years to appreciate this.
>
>> A little background could help. I am a game developer and 
>> during my semester I want to experiment with making games in 
>> D. I use threads to separate some tasks that can easily work 
>> in parallel with each other. The most common being a 
>> Logic/Graphics separation. But as development progresses I 
>> usually add more threads like inside graphics I can end up 
>> with 2 or 3 more threads.
>
> I can only repeat the above: don't think in terms of threads 
> and shared
> memory, think in terms of processes and messages passed between 
> them.
>
>> I want to avoid Amdahl's law as much as possible and have as 
>> small synchronization nodes. The data exchange should be as 
>> basic as possible but still have room for improvements and 
>> future additions.
>
> Isn't the current hypothesis that you can't avoid Amdahl's Law? 
> If what
> you mean is that you want to ensure you have an embarrassingly 
> parallel
> solution so that speed up is linear that seems entirely 
> reasonable, but
> then D has a play in this game with the std.parallelism module.
>  It uses
> the term "task" rather than process or thread to try and 
> enforce an
> algorithm-focused view. cf. 
> http://dlang.org/phobos/std_parallelism.html
>
>> The Concurrency library looked very promising but felt like 
>> the synchronization wouldn't be that nice but it would provide 
>> a random-access to the data in your code. Correct me of course 
>> if I am wrong. Is there a good thread pool system that could 
>> be used? Does that system also handle solving dependencies in 
>> the work-flow? This is what we use at my work more or less.
>
> What makes you say synchronization is not that nice?
>
> Random access, data, threads and parallelism in the same 
> paragraph
> raises a red flag of warning!
>
> std.concurrency is a realization of actors so there is 
> effectively a
> variety of thread pool involved. std.parallelism has task pools
> explicitly.
>
>> In worst case scenario I will just use the basic thread class 
>> and implement my own system above that. Then there is the 
>> question, is there any pitfalls in the current library that I 
>> should be aware of?
>
> I am sure D's current offerings are not perfect but they do 
> represent a
> good part of the right direction to be travelling.  What is 
> missing is a
> module for dataflow processing(*) and one for CSP.  Sadly I 
> haven't had
> time to get stuck into doing an implementation as I had 
> originally
> planned 18 months or so ago: most of my time is now in the 
> Python and
> Groovy arena as that is where the income comes from.  cf. GPars
> (http://gpars.codehaus.org) and Python-CSP – though the 
> latter has
> stopped moving due to planning a whole new Python framework for
> concurrency and parallelism.
>
>
> (*) People who talk about "you can implement dataflow with 
> actors and
> vice versa" miss the point about provision of appropriate 
> abstractions
> with appropriate performance characteristics.
> 

Aight been reading a lot now about it. I'm interested in the 
TaskPool but there is a problem and also why I have to think 
about threads. OpenGL/DirectX contexts are only valid for one 
thread at a time. And with the task pool I can't control what 
thread to be used with the specified task right? At least from 
what I could find I couldn't. So that's out of the question. The 
concurrency library is... I don't know. I most usually do a very 
fast synchronization swap(just swap two pointers) while the 
concurrency library seems like it would halt both threads for a 
longer time. Or am I viewing this from the wrong direction? 
Should I do it like lazy evaluation maybe? If you need code 
examples of what I am talking about I can give you that. Though I 
don't know the code-tag for this message board.

I will still use the task pool I think though all OpenGL calls 
will have to be routed so they are all done on the same thread 
somehow.

The message box for the threads in concurrency, are they thread 
safe? Let's say we have two logic tasks running in parallel and 
both are sending messages to the graphics thread. Would that 
result in undefined behavior or does the concurrency library 
handle this kind of scenario for you?


More information about the Digitalmars-d-learn mailing list