Promises in D

[Vladimir Panteleev]

On Thursday, 8 April 2021 at 11:13:55 UTC, Sebastiaan Koppe wrote:
> On Thursday, 8 April 2021 at 09:31:53 UTC, Vladimir Panteleev 
> wrote:
>> I see, thanks! So, if I understand correctly - to put it in 
>> layman terms, senders/receivers is just a structured way to 
>> chain together callables, plus propagating errors (as with 
>> promises), plus cancellation. I understand that `setValue` 
>> just calls the next continuation with its argument (as opposed 
>> to storing the value somewhere as its name might imply), which 
>> means that the value may reside on the stack of the sender's 
>> start function, and remain valid only until `setValue` exits.
>> The API is also somewhat similar, and I understand the main 
>> distinction is that starting execution is explicit (so, and 
>> the end of your `.then` chain, there must be a `.start()` call 
>> OSLT).
>
> Yes, but be aware that the callee of .start() has the 
> obligation to keep the operational state alive until *after* 
> one of the three receiver's functions are called.

Sorry, what does operational state mean here? Does that refer to 
the root sender object (which is saved on the stack and 
referenced by the objects implementing the intermediate 
steps/operations)? Or something else (locals referred by the 
lambdas performing the asynchronous operations, though I guess in 
that case DMD would create a closure)?

Also, does this mean that this approach is not feasible for 
`@safe` D?

> Often, instead of calling `.start` you would call `.sync_wait`, 
> or just return the sender itself (and have the parent take care 
> of it).

I'm finding it a bit difficult to imagine how that would look 
like on a larger scale. Would it be possible to write e.g. an 
entire web app where all functions accept and return senders, 
with only the top-level function calling `.start`?

Or is there perhaps a small demo app making use of this as a 
demonstration? :)

>> I see how you could write a fiber-based executor/scheduler, 
>> but, I don't see how you could use these as a base for a 
>> synchronous fiber API like async/await. With delegates (and 
>> senders/receivers), there is a known finite lifetime of the 
>> value being propagated. With async/await, the value is 
>> obtained as the return value of `await`, which does not really 
>> provide a way to notify the value's source of when it is no 
>> longer needed.
>
> Hmm, I see. But isn't that the limitation of async/await 
> itself? I suppose the solution would be to build refcounts on 
> top of the value, such that the promise hold a reference to the 
> value (slot), as well as any un-called continuations. Which 
> would tie the lifetime of the value to that of the promise and 
> all its continuations.

Logically, at any point in time, a promise either has un-called 
continuations, OR holds a value. As soon as it is fulfilled, it 
schedules all registered continuations to be called as soon as 
possible. (In reality there is a small window of time as the 
scheduler runs these continuations before they consume the value.)

We *could* avoid having to do reference counting or such with 
promises if we were to:

1. Move the value into the promise, thus making the promise the 
value's owner

2. Call continuations actually immediately (not "soon" as 
JavaScript promises do)

3. Define that continuation functions may only use the value 
until they return.

With these modifications, it is sufficient to make the promise 
itself reference-counted (or, well, non-copyable). When it is no 
longer referenced / goes out of scope, all consumers of the value 
will have been called, and no more can be registered.

However, these modifications unfortunately do make such promises 
unusable for async/await. Here, the continuation is the fragment 
of the `async` function from that `await` and only until the next 
`await` (or the return). We can't really make any assumptions 
about the lifetime of the value in this case. (I think the same 
applies to fibers, too?)

The "call soon" requirement is interesting because it does help 
avoid an entire class of bugs, where something N levels deep 
removes the rug from under something N-10 levels deep, so I guess 
it's a trade-off between performance and potential correctness.

> Ultimately this is all caused by the promise's design. 
> Specifically the fact that you can `.then` the same promise 
> twice and get the same value. Senders/Receivers don't have 
> this. You get the value/error/done exactly once. Calling start 
> again is not allowed.

Yeah, I see. They don't hold a copy of the value at all, but are 
just a protocol for passing them around to the next processing 
step.