Asynchronicity in D

[Piotr Szturmaj]

dsimcha wrote:
> == Quote from Piotr Szturmaj (bncrbme at jadamspam.pl)'s article
>> Max Klyga wrote:
>>> I've been thinking on things I can change in my GSoC proposal to make it
>>> stronger and noticed that currently Phobos does not address asynchronous
>>> I/O of any kind.
>>>
>>> A number of threads on thid newsgroup mentioned about this problem or
>>> shown ways other languages address asynchronicity.
>>>
>>> I want to ask D community about plans on asynchronicity in Phobos.
>>> Did somenone in Phobos team thought about possible design?
>>> How does asynchronicity stacks with ranges?
>>> What model should D adapt?
>>> etc.
>>>
>> Yes, asynchronous networking API would be more scalable. If you're
>> collecting information about async IO, please take a look at libevent
>> and libev, also NT's completion ports, FreeBSD's kqueue and Linux epoll.
>> Protocols implemented using event-driven APIs should scale to thousands
>> of connections using few working threads. Moreover async protocols could
>> be wrapped to be synchronous (but not other way around) and used just
>> like well known blocking API's.
>> Basically, while using async IO, you request some data to be written and
>> then wait for completion event (usually by callback function). Then you
>> request some allocated buffer to be read and then you wait until network
>> stack fills it up. You do not wait for blocking operation like with
>> using send() or recv(), instead you may do some useful processing
>> between events.
>
> Forgive any naiveness here, but isn't this just a special case of future promise
> parallelism?  Using my proposed std.parallelism module:
>
> auto myTask = task(&someNetworkClass.recv);
>
> // Use a new thread, but this could also be executed on a task
> // queue to keep the number of threads down.
> myTask.executeInNewThread();
>
> // Do other stuff.
>
> auto recvResults = myTask.yieldWait();
> // Do stuff with recvResults
>
> If I understand correctly (though it's very likely I don't since I've never
> written any serious networking code before) such a thing can and should be
> implemented on top of more general parallelism primitives rather than being baked
> directly into the networking design.

Asynchronous tasks are great thing, but async networking IO aka 
overlapped IO is something different. Its efficency comes from direct 
interaction with operating system. In case of tasks you need one thread 
for each task, whereas in overlapped IO, you just request some well 
known IO operation, which is completed by the OS in the background. You 
don't need any threads, besides those which handle completion events.
Here is a good explanation of how it works in WinNT: 
http://en.wikipedia.org/wiki/Overlapped_I/O