concurrency call to arms

[John Belmonte]

On Thursday, 16 August 2018 at 23:33:04 UTC, H. S. Teoh wrote:
> However, it would seem to require language support, no?  It's 
> going to be a tough sell to Walter & Andrei if it requires 
> language support. (Though IMO it's worth it.)

To implement scoped nursery and cancellation?  I hope it could be 
done with libraries given D's flexibility.  At the very least 
they could be prototyped with scope exits.

async/await might need syntax.  Part of async/await is just 
knowing what functions and call sites can context switch, so you 
can get that with decorators and clear library API.  But the 
other part is compiler help-- e.g. any function with await must 
be declared async, enforced by the compiler.  But I suspect a D 
library could do some compile time magic here too.

One point is that the new control structures are valid regardless 
of how threads are implemented:   OS threads, coroutines with 
implicit context switch, coroutines with explicit context switch, 
etc.  What seems by far the most promising is the last one since 
it further simplifies reasoning about concurrent programs.  And 
that's exactly what Trio + Python async/await provide.

> But we might be able to work around this with a mechanism 
> similar to @safe/@system/@trusted, to isolate potentially 
> encapsulation-breaking code from code that's been vetted to not 
> have unwanted concurrent side-effects.  Then within the realm 
> of "well-behaved" code, we can reap the benefits of the new 
> concurrency model without being crippled by the potential of 
> interacting with "badly-behaved" code.

It's an important point.  As argued in the first article, as soon 
as parts of the program are spawning tasks ad-hoc, the benefits 
break down.  Hence GOTO has been eliminated or neutered in most 
languages, using the author's analogy.  Similarly D can use the 
approach of @safe etc. so that we know what parts of the program 
will behave correctly on cancellation or exception.

> Not sure what you're referring to by "GC dirty laundry", but 
> IMO, the GC gets a lot of undeserved hate from GC-phobic folk 
> coming from C/C++.

I totally agree with the importance of a GC.  I'm referring to GC 
stop-the world latency.  E.g. Go language has a concurrent GC now 
at around 500 usec pause per GC and due to drop significantly 
more (https://blog.golang.org/ismmkeynote).