Asynchronicity in D

[Jonas Drewsen]

On 01/04/11 01.07, dsimcha wrote:
> == Quote from Jonas Drewsen (jdrewsen at nospam.com)'s article
>> On 31/03/11 23.20, Max Klyga wrote:
>>> On 2011-03-31 22:35:43 +0300, Jonas Drewsen said:
>>>
>>>> On 31/03/11 18.26, Andrei Alexandrescu wrote:
>>>>> snip
>>>>
>>>> I believe that we would need both the threaded async IO that you
>>>> describe but also a select based one. The thread based is important
>>>> e.g. in order to keep buffering incoming data while processing
>>>> elements in the range (the OS will only buffer the number of bytes
>>>> allowed by sysadmin). The select based is important in order to handle
>>>> _many_ connections at the same time (think D as the killer app for
>>>> websockets). As Robert mentions fibers would be nice to take into
>>>> consideration as well.
>>>>
>>>> What I also see as an unresolved issue is non-blocking handling in
>>>> http://erdani.com/d/phobos/std_stream2.html which fits in naturally
>>>> with this topic I think.
>>>>
>>>> I may very well agree mentoring if we get a solid proposal out of this.
>>>
>>> I'm very glad to hear this. Now my motivation doubled!
>>>>
>>>> /Jonas
>>>
>>> Any comments, if this proposal be more focused on asyncronous networking
>>> or should it address asyncronisity in Phobos in general?
>>>
>>> I researched a little about libev and libevent. Both seem to have some
>>> limitations on Windows platform.
>>>
>>> libev can only be used to deal with sockets on Windows and uses select,
>>> which limits libev to 64 file handles per thread.
>> Actually it seems the limit is OS version dependent and for NT it is
>> 32767 per process: http://support.microsoft.com/kb/111855
>
> Again forgive my naiveness, as most of my experience with concurrency is
> concurrency to implement parallelism, not concurrency for its own sake.  Shouldn't
> 32,000 threads be more than enough for anything?  I can't imagine what kinds of
> programs would really need this level of concurrency, or how bad performance on
> any specific thread would be when you have this many.  Right now in my Task
> Manager the program with the most threads is explorer.exe, with 28.

There doesn't have to be a thread for each socket. Actually many servers 
have very few threads with many sockets each. 32000 sockets is not 
unimaginable for certain server loads e.g. websockets or game servers. 
But I know it is not that common.

/Jonas