Threadpools, difference between DMD and LDC

[Philippe Sigaud via Digitalmars-d-learn]

On Mon, Aug 4, 2014 at 6:38 PM, Russel Winder via Digitalmars-d-learn
<digitalmars-d-learn at puremagic.com> wrote:

> Are these std.concurrent threads or std.parallelism tasks?
>
> A std.parallelism task is not a thread. Like Erlang or Java Fork/Join
> framework, the program specifies units of work and then there is a
> thread pool underneath that works on tasks as required. So you can have
> zillions of tasks but there will only be a few actual threads working on
> them.

That's it. Many tasks, a few working threads. That's what I'm
converging to. They are not particularly 'concurrent', but they can
depend on one another.

My only gripes with std.parallelism is that I cannot understand
whether it's interesting to use the module if tasks can create other
tasks and depend on them in a deeply interconnected graph. I mean, if
I have to write lots of scaffolding just to manage dependencies
between task, I might as well built it on core.thread and message
passing directly. I'm becoming quite enamored of message passing,
maybe because it's a new shiny toy for me :)

That's for parsing, btw. I'm trying to write a n-core engine for my
Pegged parser generator project.



>> Most likely those threads either do nothing or are short living
>> so you don't get actually 10 000 threads running simultaneously.
>
> I suspect it is actually impossible to start this number of kernel
> threads on any current kernel

So, what happens when I do

void doWork() { ... }

Tid[] children;
foreach(_; 0 .. 10_000)
    children ~= spawn(&doWork);

?

I mean, it compiles and runs happily.
In my current tests, I end the application by sending all thread a
CloseDown message and waiting for an answer from each of them. That
takes about 1s on my machine.

> I have no current data, but it used to be that for a single system it
> was best to have one or two more threads than the number of cores.
> Processor architectures and caching changes so new data is required. I
> am sure someone somewhere has it though.

I can add that, depending on the tasks I'm using, it's sometime better
to use 4, 6, 8 or 10 threads, repeatedly for a given task. I'm using a
Core i7, Linux sees it as an 8-core.
So, well, I'll try and see.