streaming redux

[Steven Schveighoffer]

On Thu, 30 Dec 2010 16:49:15 -0500, Dmitry Olshansky  
<dmitry.olsh at gmail.com> wrote:

> [snip]
>> -----
>> Question: Should we allow read to return an empty slice even if atEnd  
>> is false? If we do, we allow non-blocking streams with burst transfer.  
>> However, naive client code on non-blocking streams will be inefficient  
>> because it would essentially implement busy-waiting.
>>
>> Why not return an integer so different situations could be designated?   
>> It's how the system call read works so you can tell no data was read  
>> but that's because it's a non-blocking stream.
>>
>> I realize it's sexy to return the data again so it can be used  
>> immediately, but in practice it's more useful to return an integer. For  
>> example, if you want to fill a buffer, you need a loop anyways (there's  
>> no guarantee that the first read will fill the buffer), and at that  
>> point, you are just going to use the length member of the return value  
>> to advance your loop.
>>
>> I'd say, return -1 if a non-blocking stream returns no data, 0 on EOF,  
>> positive on data read, and throw an exception on error.
>>
> Maybe it's only me but I would prefer non-blocking IO not mixed with  
> blocking in such a way. Imagine function that takes an  
> UnbufferedInputTransport, how should it indicate that it expects only a  
> non-blocking IO capable transport? Or the other way around. Checking  
> return codes hardly helps anything, and it means supporting both types  
> everywhere, which is a source of all kind of  weird problems.
>  From my (somewhat limited) experience, code paths for blocking and  
> non-blocking IO are quite different, the latter are performed by  
> *special* asynchronous calls which are supported by all modern OSes for  
> things like files/sockets.
>
> Then my position would be:
> 1) All read/write methods are *blocking*, returning empty slices on EOF.
> 2) Transport that supports asynchronous IO should implement extended  
> interfaces like
> interface AsyncInputTransport: UnbufferedInputTransport{
>      void asyncRead(ubyte[] buffer, void delegate(ubyte[] data)  
> callback=null);
> }
> interface AsyncOutputTransport: UnbufferedOutputTransport{
>      void asyncWrite(ubyte[] buffer, void delegate(ubyte[] data)  
> callback=null);
> }
> Where callback (if not null) is called with a slice of buffer containing  
> actual read/written bytes on IO completion.
> Any calls to read/asyncRead while there is asynchronous IO operation  
> going on should throw, of course.

On Linux, you set the file descriptor to blocking or non-blocking, and  
read(fd) returns errno=EWOULDBLOCK when no data is available.  How does  
this fit into your scheme?  I.e. if you call read() on a  
AsyncInputTransport, what does it do when it gets this error?

It's quite possible that there is some API I'm unaware of for doing  
non-blocking and blocking I/O interleaved, but this has been my experience.

-Steve