Network server design question

[John Colvin]

On Sunday, 4 August 2013 at 19:37:40 UTC, Marek Janukowicz wrote:
> I'm writing a network server with some specific requirements:
> - 5-50 clients connected (almost) permanently (maybe a bit 
> more, but
> definitely not hundreds of them)
> - possibly thousands of requests per seconds
> - responses need to be returned within 5 seconds or the client 
> will
> disconnect and complain
>
> Currently I have a Master thread (which is basically the main 
> thread) which
> is handling connections/disconnections, socket operations, 
> sends parsed
> requests for processing to single Worker thread, sends 
> responses to clients.
> Interaction with Worker is done via message passing.
>
> The problem with my approach is that I read as much data as 
> possible from
> each ready client in order. As there are many requests this 
> read phase might
> take a few seconds making the clients disconnect. Now I see 2 
> possible
> solutions:
>
> 1. Stay with the design I have, but change the workflow 
> somewhat - instead
> of reading all the data from clients just read some requests 
> and then send
> responses that are ready and repeat; the downside is that it's 
> more
> complicated than current design, might be slower (more loop 
> iterations with
> less work done in each iteration) and might require quite a lot 
> of tweaking
> when it comes to how many requests/responses handle each time 
> etc.
>
> 2. Create separate thread per each client connection. I think 
> this could
> result in a nice, clean setup, but I see some problems:
> - I'm not sure how ~50 threads will do resource-wise (although 
> they will
> probably be mostly waiting on Socket.select)
> - I can't initialize threads created via std.concurrency.spawn 
> with a Socket
> object ("Aliases to mutable thread-local data not allowed.")
> - I already have problems with "interrupted system call" on 
> Socket.select
> due to GC kicking in; I'm restarting the call manually, but TBH 
> it sucks I
> have to do anything about that and would suck even more to do 
> that with 50
> or so threads
>
> If anyone has any idea how to handle the problems I mentioned 
> or has any
> idea for more suitable design I would be happy to hear it. It's 
> also
> possible I'm approaching the issue from completely wrong 
> direction, so you
> can correct me on that as well.

Take a look at how vibe.d approaches the problem: 
http://vibed.org/