std.socket - problems closing socket

[simendsjo]

On 03.10.2011 16:16, Regan Heath wrote:
> On Mon, 03 Oct 2011 12:57:56 +0100, simendsjo <simendsjo at gmail.com> wrote:
(...)
> To help me understand (I know nothing about fastcgi or nginx) can you
> clarify...
> 1. Your D code is the client side, connecting to the web server and
> sending GET/POST style requests?
> 2. You get these ABORTED and RESET errors on the client side?
> 3. As #3 even after doing as I described, shutdown(SEND), recv, then close?
>
> If yes to all the above, then it sounds like the web server/fastcgi is
> closing the socket without reading all the data you're sending, which
> probably means you're sending something it's not expecting. I would
> start by verifying exactly what data you're sending, and that it's all
> expected by the remote end.


Yes. I've coded the client as follows:
1) start listening socket
2) wait for incoming connections or incoming data
3) receive(). If a socket returns 0 or -1, close it and process next 
with data
4) read fastcgi request from server
5) write fastcgi response
6) write fastcgi EndRequest (the server should now end the request)
7) if the application should close the request, send shutdown(send)
8) accept incoming connection
9) back to 2)

FastCGI connections works in one of two ways: the server is responsible 
for closing the connections (supports mulitplexing) or the application 
should close the connection after a request has been sent.
For the latter I send SocketShutdown.SEND after writing EndRequest in 
step 6), but it doesn't really matter as nginx doesn't support 
multiplexing. It closes the connection after each request anyway. I see 
the same result no matter what option I use.

I'm running the exact same request and writing the exact same response 
for all queries, so there shouldn't be any unknown fields.

I also only get an error on <1/5 of the requests, and even when the 
error occurs, the response has been written completely to the browser.


>> I'm doing socket.shutdown(SocketShutdown.SEND) now after sending all
>> my data and reading until I receive 0 or -1. (doesn't really matter as
>> sending the FastCGI EndRequest makes the server shut it down as it
>> doesn't handle multiplexing)
>
> So, the socket closure is initiated by fastcgi/the web server. This
> supports the theory that it's not reading some of your data, because
> it's not expecting it, and this is likely the cause of the ABORT/RESET
> errors you're seeing.
>
>> I have tried with linger too, but it doesn't help:
>> socket.setOption(SocketOptionLevel.SOCKET, SocketOption.LINGER,
>> std.socket.linger(1, 30));
>
> The default LINGER options should be fine, as-is. But, double check the
> D socket code just in case it is setting different LINGER options by
> default (I haven't used it, or looked myself, sorry).


Linger is default off, but it doesn't help to turn it on. RCV/SNDTIMO is 
also set to 0.


>> Could this be caused by some bad settings on the webserver?
>
> It is possible, but I would double check your requests first. There may
> be a setting, or settings for aborting connections which take too long,
> or fail to send certain data, or connect from the wrong IP, or... If
> your requests are otherwise working, then I suspect you're sending some
> 'extra' data which is not being read.


The requests are handled in ~1msec, so there shouldn't be any timeouts. 
The default timeout on nginx for fastcgi is 60 seconds too.
I can easily process ~200 requests per second (and nginx and my server 
doesn't break a sweat, it's my curl spammers that's using all the cpu)


>> PS: Seems my computer can handle about 16000 TIME_WAIT before it
>> starts "hanging".
>
> You'll be running out of operating system handles or similar at that
> point :p
>

Yup. I'll probably never have that problem in a production environment 
though :)