Network server design question

[Brad Roberts]

A reasonably common way to handle this is that the event loop thread only detects events (readable, 
writable, etc) and passes them off to worker threads to process (do the reading and parsing, do the 
writing, etc).  In general, I wouldn't recommend one thread per active connection, but if you're 
_sure_ that you're constrained to those low sorts of numbers, then it might well be the easiest path 
to go for your app.  You definitely want to move the actual i/o out of your event loop thread. to 
let those other cores take on that job, freeing up your single threaded part to do a little work as 
possible.  It's your bottleneck and that resource needs to be protected.

On 8/4/13 12:38 PM, Marek Janukowicz wrote:
> I'm writing a network server with some specific requirements:
> - 5-50 clients connected (almost) permanently (maybe a bit more, but
> definitely not hundreds of them)
> - possibly thousands of requests per seconds
> - responses need to be returned within 5 seconds or the client will
> disconnect and complain
>
> Currently I have a Master thread (which is basically the main thread) which
> is handling connections/disconnections, socket operations, sends parsed
> requests for processing to single Worker thread, sends responses to clients.
> Interaction with Worker is done via message passing.
>
> The problem with my approach is that I read as much data as possible from
> each ready client in order. As there are many requests this read phase might
> take a few seconds making the clients disconnect. Now I see 2 possible
> solutions:
>
> 1. Stay with the design I have, but change the workflow somewhat - instead
> of reading all the data from clients just read some requests and then send
> responses that are ready and repeat; the downside is that it's more
> complicated than current design, might be slower (more loop iterations with
> less work done in each iteration) and might require quite a lot of tweaking
> when it comes to how many requests/responses handle each time etc.
>
> 2. Create separate thread per each client connection. I think this could
> result in a nice, clean setup, but I see some problems:
> - I'm not sure how ~50 threads will do resource-wise (although they will
> probably be mostly waiting on Socket.select)
> - I can't initialize threads created via std.concurrency.spawn with a Socket
> object ("Aliases to mutable thread-local data not allowed.")
> - I already have problems with "interrupted system call" on Socket.select
> due to GC kicking in; I'm restarting the call manually, but TBH it sucks I
> have to do anything about that and would suck even more to do that with 50
> or so threads
>
> If anyone has any idea how to handle the problems I mentioned or has any
> idea for more suitable design I would be happy to hear it. It's also
> possible I'm approaching the issue from completely wrong direction, so you
> can correct me on that as well.
>