An idea; Coroutines

[Richard Andrew Cattermole (Rikki)]

Coroutines! Oh coroutines.

Recently Walter has shown some interest in me describing my 
library implementation thereof, due to my belief that they should 
be used for asynchronous tasks and hence need to be part of the 
language.

Before I begin I just want to say, that while I have a library 
implementation working, this is not the only way to do 
coroutines, nor is the proposed syntax.

-----------

So to begin with, let's define what the library author considers 
the library API to be.

```d
struct InstantiableCoroutine(ResultType, Args...) {
     this(T:__coroutine)(immutable T co);

     bool isNull();
     bool isInstantiated();
     InstantiableCoroutine makeInstance(Args args);

     GenericCoroutine opCast(T=GenericCoroutine)();
     Future opCast(T=Future!ResultType)();

     GenericCoroutine waitingOn();
     COResult!ResultType result();

     bool isComplete();
     void unsafeContinue() @system;
}
```

This is the premise that the other two API representations are 
built from.

What we wish to describe in this API, is a type that can be 
instantiated with new instances, after having a descriptor passed 
in via say a closure or from function pointer (whose function is 
being compiled also).
It needs to be able to tell the user some information of it, if 
it is instantiated and if so, is it currently complete or waiting 
on another coroutine.
Along with the respective values.
Lastly we must be able to continue execution once we are no 
longer waiting on our condition.

Of note is the ``GenericCoroutine`` and ``Future``. Take notice 
of the template arguments that are provided by the ``opCast``'s. 
Their respective methods are a subset of 
``InstantiableCoroutine``'s.

For my implementation I use structs and reference counting.
I find that this works quite well.
However due to the possibility of cyclic relationships, a GC may 
be desired if you want a more naive solution.

-----------

The ``GenericCoroutine`` is where the real magic happens.
It is what all the coroutine API's boil down to without templates.

When you do dependency analysis on what coroutine is waiting on 
another, and when no longer waiting on another sent off to the 
worker pool to execute, this is the abstraction that is used 
internally to the event loop.

-----------

Next up is a specialized coroutine implementation that I call 
"future-completion".

A future completion uses the same API as above, but one very key 
difference.
The descriptor state does not come from the user.
It comes from a prebuilt library function that only wants to know 
the return type of the future.

It will only complete, after it has been externally triggered as 
such.
This can be done for say a socket read based upon such functions 
as ``poll``.

For reference counting this helps break up cycles since you can 
store the trigger separately from the reference counted 
abstraction.
So it does a lot of jobs, I have found it to be irreplacable.

As a feature, it is quite easily one of my most genius ideas ever.
Simply because it is an ordinary coroutine, that integrates into 
the worker dependency state for coroutines, and does not require 
the user to know that it is special.

-----------

For a language feature, it should not be tied to a given library 
implementation.
There is no reason for it to be.

Done properly and with enough understanding of the compiler, it 
should be possible to write code similar to what I posted 
earlier, without using any attributes to say that this is a 
coroutine object that can be implicitly constructed.
It can see that it can be because of the constructor template 
check.

This leads us to wanting to describe the user experience:

```d
struct ListenSocket {
     static ListenSocket 
listen(A...)(InstantiableCoroutine!(Socket, A) coroutine, ushort 
port);
}

ListenSocket.listen((Socket socket) /* @notls */ {
     writeln(socket.read(4));

     // is equivalent to

     /+
     Future!(ubyte[]) __temp0 = socket.read(4);
     await __temp0;
     assert(__temp0.isComplete);
     writeln(__temp0.result);
     +/
}, port: 8080);
```

It looks sequential.

The user knows nothing about the coroutines happening underneath.

This is quite honestly the holy grail of asynchronous programming 
that we as a field have been studying and trying to make work 
since the 1950's.
See[0] for more information.

We can do it.

There is nothing stopping us except political will.

[0] 
https://www.amazon.com.au/Concurrent-Programming-C-R-Snow/dp/0521339936
[1] 
https://github.com/Project-Sidero/eventloop/tree/master/source/sidero/eventloop/coroutine
[2] 
https://github.com/Project-Sidero/eventloop/blob/master/source/sidero/eventloop/tasks/future_completion.d
[3] 
https://github.com/Project-Sidero/eventloop/blob/master/source/sidero/eventloop/internal/workers.d#L169

-----------

My code while not ready for users, does work.
See [1] for coroutines API, builder has the unit test.
[2] for future completions logic (yes there is unittest in 
there!).
And [3] for the internal coroutine dependency state.


For those interested in the state that the compiler would 
generate consider:

```d
struct CO_Object_xx(LibraryType) {
     enum Functions = [&__stage1, &__stage2];
     alias UserVars = AliasSeq!(...);
     alias ResultType = ...;

     static struct State {
         Stage stage;
         LibraryType waitingOn;

         UserVars vars;
         ResultType result;

         version(D_BetterC) {
         } else {
             Exception resultException;
         }

         ~this(); // cleanup if required

         bool isComplete() {
             return this.stage == Stage.CompleteValue || 
this.stage == Stage.CompleteException;
         }
     }

     enum Stage {
         Stage_1,
         Stage_2,
         ReadyToStart,
         CompleteValue,
         CompleteException,
     }

     void __stage1(scope State* state) {
     }

     void __stage2(scope State* state) {
     }
}
```