concurrency call to arms

[Russel Winder]

On Tue, 2018-08-28 at 03:36 +0000, John Belmonte via Digitalmars-d
wrote:
> On Wednesday, 22 August 2018 at 16:49:01 UTC, Russel Winder wrote:
> > Have you tried asyncio in the Python standard library? Is Trio 
> > better?
> 
> The library that Guido admits is a disaster?  
> https://twitter.com/gvanrossum/status/938445451908472832
> 
> Trio and libraries like it have evolved out of frustration with 
> asyncio.

When I originally wrote the comment I was thinking of asyncore, which
is even worse than asyncio.

[…]
> 
> It's far more than "ensuring join".  It ensures that tasks 
> created by children are similarly constrained.  It allows the 
> normal try-catch of exceptions coming from any level of child 
> tasks.  It ensures that when a task has an error, all sibling 
> tasks are cancelled and the resulting multi-exception is 
> propagated to the parent.

Any fork/join framework will provide such guarantees.

> > – except Python is single threaded so the above is time
> > division multiplexing of tasks.
> 
> No, tasks can optionally be real threads (obviously constrained 
> by GIL).  Soon they can be tasks from the "multiprocessing" 
> library as well (separate process, either local or remote).
> 
> These are details of Trio's implementation and Python.  As 
> mentioned, the control structures apply to any concurrency model 
> (implicit or explicit context switching, OS or light threads, 
> etc.)

But that is the point, this is Python specific, and yet the motivating
example is a misunderstanding of how Go is used. This inconsistency
seriously undermines the general argument.

I have no problem with using with statement and context managers to
enforce fork/join abstractions in Python. But to say this is a new
general abstraction is  false claim.

[…]
> async/await is a means to explicitly mark points of context 
> switching in API's and code.  It applies to threads, coroutines, 
> remote processing, or anything else which may allow something 
> else to modify system state while you're waiting.

Certainly it is a language structure to support yield to the
appropriate scheduler, the question is whether a language structure is
required or it can be handled with library features. I suspect language
feature makes things easier of implementation.

> Local to a single thread, async/await with coroutines happens to 
> be wonderful because it eliminates a ton of explicit locking, 
> fretting about race conditions, etc. required of the programmer.  
> It's a building block-- e.g. you can then combine such threads in 
> careful ways with message passing within and among CPU cores to 
> get the best of all worlds.

I've never really worried about single threaded concurrency, nor used
C#, so I have no actual data to apply to this argument. In a multi-
threaded context, you can do all that is needed using processes and
channels. Having said this executors in single or multi threaded
contexts work fine at the library level without language constructs.

> While Go can certainly implement the nursery and cancellation 
> control structures, there is no turning back on Go's implicit 
> context switching (i.e. any old function call can cause a context 
> switch).  The human is back to fretting about locks and race 
> conditions, and unable to prove that even the smallest of 
> programs is correct.

Not if the user sticks to using processes and channels as is the idiom
on Go. Go has all the locks stuff, but if you use channels as the means
of communication between processes, the programmer never needs explicit
locks since everything is handled bu blocking on channels.

-- 
Russel.
===========================================
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20180828/91d37df7/attachment.sig>