`Socket.receive` providing arbitrary packet sizes and hanging without sending EOF

[Unazed Spectaculum]

On Thursday, 14 December 2017 at 20:27:36 UTC, Ali Çehreli wrote:
> On 12/14/2017 11:55 AM, Unazed Spectaculum wrote:
>
>> This is the only function which has a call to `receiveAll`
>
>
> I marked my changes with [Ali]:
>
> import std.stdio;
> import std.socket;
> import std.conv;
> import std.json;
>
> ubyte[] receiveBytes(T)(T socket, size_t receiveCount)
> {
>     ubyte[] buffer = new ubyte[receiveCount];
>     size_t count = socket.receive(buffer);
>     // [Ali] Return null when there is no data
>     if (count == Socket.ERROR || count == 0) {
>         return null;
>     }
>     return buffer[0 .. count];
> }
>
> string receiveAll(T)(T socket, size_t segmentSize = 1024)
> {
>     ubyte[][] data;
>     size_t count = 0;
>
>     do
>     {
>         debug(1) writefln("Chunk %s", count);
>         auto part = receiveBytes(socket, segmentSize);
>         // [Ali] Done when there is no data
>         if (!part) {
>             break;
>         }
>         data ~= part;
>         writeln(data[count]);
>         if (!data)
>             break;
>
>     } while(data[count++]);
>
>     char[] stringData;
>
>     foreach (elem; data)
>         stringData ~= elem;
>
>     debug(1) writeln(`Exiting "receiveAll"`);
>
>     return to!string(stringData);
> }
>
> void main()
> {
>     auto socket = new TcpSocket();
>     // [Ali]    setupSocket(socket, "0.0.0.0", 6969);
>     // The following had worked in a test program:
>     socket.setOption(SocketOptionLevel.SOCKET, 
> SocketOption.REUSEADDR, true);
>     socket.bind(new InternetAddress(6969));
>     socket.listen(1);
>
>     writefln("Listening: %s", socket.localAddress);
>
>     while(true)
>     {
>         Socket client = socket.accept();
>         debug(1) writefln("Client: %s", client.remoteAddress);
>
>         auto data = receiveAll(client);
>         writeln(data);
>         JSONValue json;
>
>         try {
>             json = parseJSON(data);
>         } catch (JSONException e) {
>             debug(1) writefln("Failed parsing data as JSON, 
> aborting.\n|| %s", e);
>             client.close();
>             continue;
>         } catch (Exception e) {
>             debug(1) writefln("Client caused exception:\n||%s", 
> e);
>             client.close();
>             continue;
>         }
>
>         /+ [Ali] Not important for this test
>         if (!verifyValues(json))
>         {
>             debug(1) writefln("Client missed out important key 
> fields: %s", client.remoteAddress);
>             client.close();
>             continue;
>         }
>         +/
>
>         debug(1) writeln("Client transacted successful JSON 
> packet.");
>
>         writefln("%s:\n\tFilename: %s\n\tMethod: %s\n\tData 
> length: %d",
>                 client.remoteAddress,
>                 json["filename"],
>                 json["method"],
>                 data.length
>             );
>
>         // [Ali] Not important for this test
>         /+
>         if (json["method"].str == "store")
>             storeData(json["filename"].str, json["data"].str);
>         else if (json["method"].str == "retrieve")
>             retrieveData(client, json["filename"].str);
>         +/
>         client.close();
>     }
> }
>
> 1) Start the program
>
> 2) $ telnet 0.0.0.0 6969
> Trying 0.0.0.0...
> Connected to 0.0.0.0.
> Escape character is '^]'.
>
> 3) Paste expected json to terminal:
>
> {"filename":"/tmp/foo","method":"bar"}
>
> 4) Press the escape character
>
> ^]
>
> 5) Press Ctrl-D (Ctrl-Z on Windows) to terminate the connection
>
> telnet> Connection closed.
>
> Here is the output from the program:
>
> Listening: 0.0.0.0:6969
> [123, 34, 102, 105, 108, 101, 110, 97, 109, 101, 34, 58, 34, 
> 47, 116, 109, 112, 47, 102, 111, 111, 34, 44, 34, 109, 101, 
> 116, 104, 111, 100, 34, 58, 34, 98, 97, 114, 34, 125, 13, 10]
> {"filename":"/tmp/foo","method":"bar"}
>
> 127.0.0.1:47704:
> 	Filename: "\/tmp\/foo"
> 	Method: "bar"
> 	Data length: 40
>
> I think it works! :)
>
> Ali

I've tried to integrate your code with mine; to no avail, I've 
directly tested your code; and yet again to no avail, so this 
should conclude the thread as it pinpoints an error with my 
actual OS.

Just in case, as a last resort in case I'm actually that dumb, 
here's me reproducing your steps:

1) Starting program
unazed at unazed  /home/d/storage-server  dmd -debug -run app.d
Listening: 0.0.0.0:6969

2) telnet to the server
  unazed at unazed  ~  telnet 0.0.0.0 6969
Trying 0.0.0.0...
Connected to 0.0.0.0.
Escape character is '^]'.

3) paste '{"filename":"/tmp/foo","method":"bar"}'
  unazed at unazed  /home/d/storage-server  dmd -debug -run app.d
Listening: 0.0.0.0:6969
Client: 127.0.0.1:55014
Chunk 0
[123, 34, 102, 105, 108, 101, 110, 97, 109, 101, 34, 58, 34, 47, 
116, 109, 112, 47, 102, 111, 111, 34, 44, 34, 109, 101, 116, 104, 
111, 100, 34, 58, 34, 98, 97, 114, 34, 125, 13, 10]
Chunk 1
[pause]

4-5) press ESC + CTRL+D (I'm on Linux)
Chunk 1
[27]
Chunk 2
[pause]

So, big thanks , and sorry for taking longer to respond; I just 
get anxious about solving things sometimes and now it's kind of 
annoying I have to do some more research to fix this issue.