Streaming transport interfaces: input

[Steven Schveighoffer]

On Thu, 14 Oct 2010 16:47:13 -0400, Steven Schveighoffer  
<schveiguy at yahoo.com> wrote:

> On Thu, 14 Oct 2010 14:43:56 -0400, Andrei Alexandrescu

>> How? Denis' implementation has two copies in the mix. (I'm not counting  
>> .dup etc.) Anyhow, let's do this - write down your interfaces so I can  
>> comment on them. We talk "oh that's a buffering interface" and "that  
>> requires buffering" and "that's an extra copy" and so on but we have  
>> little concrete contenders. I put my cards on the table, you put yours.
>
> I'll see if I can put something together.

Here's a rough outline:

enum Anchor
{
     Begin,
     Current,
     End
}

interface Seek
{
     ulong seek(long delta, Anchor whence);
     final ulong tell() { return seek(0, Anchor.Current); }
     bool seekable(); // define as false if seeking is not supported, true  
if it
                      // is supported (this doesn't necessarily mean a seek  
will
                      // succeed).
}

interface InputTransport : Seek
{
     size_t read(ubyte[] data); // returns 0 on EOF.
}

// defined to implement either a D buffered object or wrap a FILE *.
//
interface BufferedInputTransport : Seek
{
     size_t read(ubyte[] data); // returns 0 on EOF.

     //
     // read data into the buffer until the delegate returns other than ~0
     //
     // The delegate is passed the entire buffer so far, with the start of  
the
     // new data just read.  It returns other than ~0 when it determines  
the end
     // of the data in question.
     //
     ubyte[] readUntil(uint delegate(ubyte[] data, uint start) process);

     //
     // same as readUntil except append to the given arr, Any excess
     // data will be pushed into the internal buffer.
     //
     size_t appendUntil(uint delegate(ubyte[] data, uint start) process,  
ref ubyte[] arr)

     // various buffer functions.
     @property size_t bufsize();
     @property size_t readable();
     // etc.
}

The way I see it working is, there are two implementations for  
BufferedInputTransport:  FILEInputTransport and DBufferInputTransport.   
There are numerous implementations of InputTransport, each of which can be  
passed to the DBufferInputTransport, which uses its own buffer  
implementation.  For example, a network socket, file, inter-thread stream,  
an array, etc.

This way, you can play nice with C's stdio when necessary (i.e. for  
stdin/stdout/stderr) and avoid the FILE limitations and performance issues  
otherwise.

-Steve