question on buffer passed in to recvFrom

[Jonathan M Davis]

On Friday, December 07, 2012 17:31:00 Matthew Caron wrote:
> Hey folks.
> 
> Given a configured socket, called server, and the following code:
> 
>      Address fromAddress;
>      void[] buffer[2048];
>      long bytes;
> 
>      bytes = server.receiveFrom(buffer, fromAddress);
> 
> All works grand.
> 
> However, if I try to do:
> 
>      Address fromAddress;
>      void[] buffer;
>      long bytes;
> 
>      bytes = server.receiveFrom(buffer, fromAddress);
> 
> bytes comes back as 0, presumably because the buffer is too small.
> 
> Might anyone be able to explain why the standard library can't
> automatically size the buffer for me, before calling the underlying C
> library code, then return to me the correct size buffer which is
> automatically garbage collected?
> 
> Thanks in advance.

The underlying C API requires that you give it a buffer and tell it the size of 
that buffer. It then returns how many bytes were actually read (which could be 
the same or less than the size of the buffer). There is no concept of having 
read the entire message. The API has no concept of that. You either ask for 
more data and get it if it's there or get none because there is none (which 
may not mean that you've reached the end of the message - the data may just 
not have arrived yet). And the D API can't possibly know the size of the 
message any more than the C API does. On top of that, if you ask for enough 
data, you could get the end of one message and the beginning of the next one 
at the same time, depending on what you're doing. So, it only makes sense for 
it to return you up to the amount that you asked for and not more. It can't 
possibly know what the correct amount is.

On top of that, we try and avoid stray allocations in the standard library. 
Some functions clearly have to allocate data (e.g. a function like 
std.array.split which returns a brand-new array), but many of them can easily 
avoid it. And in this case, it's extremely easy to avoid allocations by having 
the caller allocate the array. And that array could even be a static one, 
avoiding heap allocations entirely. In many cases though, the buffer used by a 
socket gets reused, and allocating with every call would be inefficient for no 
real gain.

- Jonathan M Davis