Network server design question

[Marek Janukowicz]

I'm writing a network server with some specific requirements:
- 5-50 clients connected (almost) permanently (maybe a bit more, but 
definitely not hundreds of them)
- possibly thousands of requests per seconds
- responses need to be returned within 5 seconds or the client will 
disconnect and complain

Currently I have a Master thread (which is basically the main thread) which 
is handling connections/disconnections, socket operations, sends parsed 
requests for processing to single Worker thread, sends responses to clients. 
Interaction with Worker is done via message passing.

The problem with my approach is that I read as much data as possible from 
each ready client in order. As there are many requests this read phase might 
take a few seconds making the clients disconnect. Now I see 2 possible 
solutions:

1. Stay with the design I have, but change the workflow somewhat - instead 
of reading all the data from clients just read some requests and then send 
responses that are ready and repeat; the downside is that it's more 
complicated than current design, might be slower (more loop iterations with 
less work done in each iteration) and might require quite a lot of tweaking 
when it comes to how many requests/responses handle each time etc.

2. Create separate thread per each client connection. I think this could 
result in a nice, clean setup, but I see some problems:
- I'm not sure how ~50 threads will do resource-wise (although they will 
probably be mostly waiting on Socket.select)
- I can't initialize threads created via std.concurrency.spawn with a Socket 
object ("Aliases to mutable thread-local data not allowed.")
- I already have problems with "interrupted system call" on Socket.select 
due to GC kicking in; I'm restarting the call manually, but TBH it sucks I 
have to do anything about that and would suck even more to do that with 50 
or so threads

If anyone has any idea how to handle the problems I mentioned or has any 
idea for more suitable design I would be happy to hear it. It's also 
possible I'm approaching the issue from completely wrong direction, so you 
can correct me on that as well.

-- 
Marek Janukowicz