Oh, my GoD! Goroutines on D

Russel Winder russel at winder.org.uk
Tue May 19 09:55:53 UTC 2020


On Tue, 2020-05-19 at 08:42 +0000, Bienlein via Digitalmars-d wrote:
> On Saturday, 16 May 2020 at 20:06:47 UTC, mw wrote:
> > On Tuesday, 29 March 2016 at 17:10:02 UTC, Jin wrote:
> > > http://wiki.dlang.org/Go_to_D
> > 
> > Any performance comparison with Go? esp. in real word scenario?
> > 
> > Can it easily handle hundreds of (go)routines?
> 
> Go can easily have some ten thousand green threads. I once did a 
> test run to see how many. On a 4 GB machine 80.000 green threads 
> aka goroutines were created till out of memory occured.

It wouldn't surprise me if std.parallelism couldn't do something analogous.
There are tasks that are executed (potentially with work stealing) by threads
in a threadpool. The problem is that the std.parallelism API is dedicated to
data parallelism rather than the process/channels concurrency/parallelism of
CSP (the theoretical foundation for goroutines – sort of but you have to read
Rob Pike's articles to see why).

I am fairly certain, but have not yet checked, that the idea of task is very
similar in std.parallelism and jin.go – which is based on vibe.d's event loop
as I understand it (but could be wrong). Vibe.d allows a threadpool, though I
suspect most people use it single threaded. So Vibe.d and std.parallelism have
a lot in common. I'll bet there is much that could be factored out and shared,
but realistically this isn't going to happen.

> Communicating sequential processes as in Go relies on being able 
> to create a large amount of threads. With a paradigm of threads 
> doing blocking takes on channels any application would otherwise 
> quickly run out of threads. In D something similar could be done 
> using fibers. Using fibers is also the approach chosen in Java 
> extending the JVM to have CSP-stye concurrency as in Go, see 
> https://www.youtube.com/watch?v=lIq-x_iI-kc

I wonder if we should use "thread" for heavyweight (OS?) thread, "fibre" for
fibre (including lightweight threads from before threads became threads), and
task for the things that get put into the job queues of a threadpool. D
already has threadpool in std.parallelism, perhaps it needs extracting (as in
Java) so it can be the basis of Vibe.d – or vice versa, obviously.

> Then you also need continuations. Lets say inside a function a 
> blocking take is done on two channels in a row. The first channel 
> has some input, the next one has not. In between comes a context 
> switch. When switching back the value taken from the first 
> channel has to be put back into the context. This is why 
> continuations are needed.

Are you sure? Isn't the whole point of tasks (processes in CSP jargon) and
channels is that you don't need continuations. A CSP computation should not
even understand the idea of a context switch. If it matters, has perhaps, the
concept of fibre intruded in a way that violates the abstraction?

> Really nice work! Please keep it going :-)
-- 
Russel.
===========================================
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20200519/6a6d3557/attachment.sig>


More information about the Digitalmars-d mailing list