My two cents

[Adam Wilson]

On 10/23/17 16:47, Nathan S. wrote:
> On Monday, 23 October 2017 at 22:22:55 UTC, Adam Wilson wrote:
>> Additionally, MSFT/C# fully recognizes that the benefits of
>> Async/Await have never been and never were intended to be for
>> performance. Async/Await trades raw performance for an ability to
>> handle a truly massive number of simultaneous tasks.
>
> Could you clarify this? Do you mean it's not supposed to have better
> performance for small numbers of tasks, but there is supposed to be some
> high threshold of tasks/second at which either throughput or latency is
> better?

It's pretty complicated. In general, for a small number of tasks, it 
will take longer to execute those task with Async than without (Async 
has an overhead, about 10ms IIRC). So each individual call to an 
awaitable function will take longer than a call to a blocking function. 
In fact, MSFT recommends that if you are reasonably certain that most of 
the time it will take less than about 10ms to just use the blocking 
methods, as they have less total overhead.

The main benefit of Async is in throughput. It allows the physical CPU 
to handle many incoming requests. So while each individual request may 
take longer, the overall utilization of the CPU is much higher.

Async, has different benefits and drawbacks depending on where it's 
applied. For example, in UI apps, it allows the main program thread to 
keep responding to system events while waiting for long-wait IO (the 
thread is not suspended).

Some background that may help understanding is that in blocking IO, what 
is really happening underneath your blocking call is that the runtime is 
creating a callback and then calling the OS's thread suspend method 
(e.g. ThreadSuspend in Windows), then when the callback is called, the 
thread is resumed and the data passed to into it via the callback. This 
means that the calling thread cannot execute anything while it's 
waiting. This is why UX apps appear to freeze when using blocking IO 
calls. The reason this is done is because the application has no way to 
say "I'm going to put this task off to the side and keep executing". The 
thread does not know to start some other processing while waiting.

Async allows the app to put that task to the aside and do something 
else. At a low level the process in .NET it does this by:
1. Serializing the stack frame of the currently executing method to the 
heap (yes, Async is a GC feature, just like lambdas) at an await.
2. Pulling the next completed task from the heap.
3. Rebuilding the stack frame for that method on any available thread.
4. Continue execution of that stack.

Obviously this makes a lot of sense for UI apps, where blocking the main 
thread can be disastrous, but why internet applications are inherently 
multi-threaded, so why do we care?

The answer is thread context switching. A context switch is the most 
expensive common CPU operation by an order of magnitude, ranging from 
2k-1m cycles. Whereas on modern CPUs a main RAM read is 100-150 cycles 
and a NUMA different socket read is 300-500 cycles. In .NET the 
TaskScheduler creates a predetermined number of threads (one per core 
IIRC) and begins scheduling tasks on those threads. Remembering that 
each task is really just a block on the heap, in the worst case that 
will take 500 cycles. Whereas if we had to switch to a different thread, 
it could be up to 1m cycles. That is a noticeable difference.

Context switches are painfully expensive but in the traditional model it 
was all we had. Task based systems allow us to circumvent the task 
switch. But they're cumbersome to use without compiler support. For 
example, the Task Parallel Library was added in .NET 4.0 which includes 
Task and Task<T> and ALL of the constructs that are used in Async/Await, 
however, it was not until .NET 4.5 and the arrival of the Async/Await 
keywords (and the compiler lowerings that they enabled) that people 
started using Tasks in any significant way.

(Source for access times: 
http://ithare.com/wp-content/uploads/part101_infographics_v08.png)

-- 
Adam Wilson
IRC: LightBender
import quiet.dlang.dev;