Promises in D

Sebastiaan Koppe mail at skoppe.eu
Sat Apr 10 14:34:01 UTC 2021


On Thursday, 8 April 2021 at 11:55:37 UTC, Vladimir Panteleev 
wrote:
> On Thursday, 8 April 2021 at 11:13:55 UTC, Sebastiaan Koppe 
> wrote:
>> Yes, but be aware that the callee of .start() has the 
>> obligation to keep the operational state alive until *after* 
>> one of the three receiver's functions are called.
>
> Sorry, what does operational state mean here? Does that refer 
> to the root sender object (which is saved on the stack and 
> referenced by the objects implementing the intermediate 
> steps/operations)? Or something else (locals referred by the 
> lambdas performing the asynchronous operations, though I guess 
> in that case DMD would create a closure)?

Operational state is a term from the proposal, it is what is 
returned when you call `.connect(receiver)` on a Sender. It 
contains all the state needed to start the Sender, often 
including the receiver itself.

It is this state that requires an allocation when you are doing 
Futures. With senders/receivers it lives on the callee's stack. 
With that comes the responsibility to keep it alive.

In practice it is a non-issue though. You are unlikely to call 
`.start()` yourself, instead you often push the responsibility 
all the way up to `void main`, where you do a `sync_wait` to 
ensure all is done.

There are cases where you want to make the operational state live 
on the heap though (because it gets too big), and there are ways 
to do that.

> Also, does this mean that this approach is not feasible for 
> `@safe` D?

I certainly tried, but there are likely some safety-violations 
left. Undoubtedly some of those could be resolved by a more 
safety-capable engineer than me; I sometimes feel it is more 
complicated to write @safe code correctly than lock-free 
algorithms - which are notoriously hard - and sometimes it is not 
possible to express the wanted semantics.

Even so, even if there is some large unsafe hole in this library, 
I rather have it than not. There is a lot of upside in being able 
to write asynchronous algorithms separate from the async tasks 
themselves. Just like the STL separated the algorithms from the 
containers, senders/receivers separate the algorithms from the 
async tasks. That is so valuable to me I gladly take a little 
possible unsafety. Although obviously I certainly welcome any 
improvements on that front!

>> Often, instead of calling `.start` you would call 
>> `.sync_wait`, or just return the sender itself (and have the 
>> parent take care of it).
>
> I'm finding it a bit difficult to imagine how that would look 
> like on a larger scale. Would it be possible to write e.g. an 
> entire web app where all functions accept and return senders, 
> with only the top-level function calling `.start`?

Yes, except the top-level function would call `.sync_wait`. The 
main reason is because that awaits completion.

The key part is expressing the web server as a Sender, and then 
run it till completion. A web server is a bit special in that it 
spawns additional sub-tasks as part of its execution. You can use 
a `Nursery()` for that, which is a Sender itself, but allows 
adding additional senders during its execution. Then you just 
model each request as a Sender and add it to the Nursery. They 
can be short lived or long lived tasks. When it is time for 
shutdown the StopToken is triggered, and that will stop the 
listening thread as well as any running sub-tasks as well (like 
open requests or websockets, etc.).

> Or is there perhaps a small demo app making use of this as a 
> demonstration? :)

Nothing public at the moment sorry, but I plan to open source our 
webserver in time.

>> Hmm, I see. But isn't that the limitation of async/await 
>> itself? I suppose the solution would be to build refcounts on 
>> top of the value, such that the promise hold a reference to 
>> the value (slot), as well as any un-called continuations. 
>> Which would tie the lifetime of the value to that of the 
>> promise and all its continuations.
>
> Logically, at any point in time, a promise either has un-called 
> continuations, OR holds a value. As soon as it is fulfilled, it 
> schedules all registered continuations to be called as soon as 
> possible. (In reality there is a small window of time as the 
> scheduler runs these continuations before they consume the 
> value.)

I think it is possible to attach a continuation after the promise 
has already completed.

```
promise = getFoo();
getBar().then(x => promise.then(y => print(x*y));
```

The one thing I miss most from promises though, is cancellation. 
With senders/receivers you get that (almost) for free, and it is 
not at all difficult to properly shutdown (parts of) your 
application (including sending shutdown notifications to any 
clients).


More information about the Digitalmars-d mailing list