Streaming library

[Denis Koroskin]

On Thu, 14 Oct 2010 01:32:49 +0400, Steven Schveighoffer  
<schveiguy at yahoo.com> wrote:

> Before responding directly, I'll say I think this is on the right  
> track.  IMO, buffering should be transparent when it can be, meaning you  
> should be able to have control over the buffering.  The abstraction  
> should look like this:
>
> specific application -> buffered streams -> OS abstraction -> Low level  
> calls.
>
> Denis' proposal covers up to OS abstraction.  What we need on top of  
> that is a buffer layer.
>
> On Wed, 13 Oct 2010 12:16:38 -0400, Denis Koroskin <2korden at gmail.com>  
> wrote:
>
>> I'll explain my I/O streams implementation below in case you didn't  
>> read my message (I've changed some stuff a little since then). My  
>> Stream interface is very simple:
>>
>> // A generic stream
>> interface Stream
>> {
>>      @property InputStream input();
>>      @property OutputStream output();
>>      @property SeekableStream seekable();
>>      @property bool endOfStream();
>>      void close();
>> }
>>
>> You may ask, why separate Input and Output streams? Well, that's  
>> because you either read from them, write from them, or both.
>> Some streams are read-only (think Stdin), some write-only (Stdout),  
>> some support both, like FileStream. Right?
>
> I feel we can possibly make this a compile-time decision.  Can we do  
> something like this:
>
> interface Stream : InputStream, OutputStream {}
>
>> Not exactly. Does FileStream support writing when you open file for  
>> reading? Does it support reading when you open for writing?
>> So, you may or may not read from a generic stream, and you also may or  
>> may not write to a generic stream. With a design like that you can make  
>> a mistake: if a stream isn't readable, you have no reference to invoke  
>> read() method on.
>
> Essentially, it's near zero the times that I decide at runtime whether  
> I'm opening a file for reading, writing or both.  So why not build that  
> into the type, and then we have the compiler to tell us when something  
> can't be used for reading or writing?
>
>> Similarly, a stream is either seekable, or not. SeekableStreams allow  
>> stream cursor manipulation:
>>
>> interface SeekableStream : Stream
>> {
>>      long getPosition(Anchor whence = Anchor.begin);
>>      void setPosition(long position, Anchor whence = Anchor.begin);
>> }
>
> A seekable interface is one of those things that's really hard to get  
> right.  In Tango, we eventually got rid of the seekable interface and  
> just added seek methods to all the low level stream interfaces.  The  
> rationale is that most of the time seekability is not a requirement you  
> can set when opening a file.  You open a file for reading or writing,  
> but not for seeking.  So it's almost necessary that seekability is a  
> runtime decision (because the OS decides it outside of your control).
>
> There are, of course, streams that will not be seekable (netowork  
> sockets), but you just throw an exception when seeking such a stream.   
> The only thing I'd create is a way to determine seekability without  
> throwing an exception (i.e. a canSeek property).  But most of the time  
> you know whether a stream is seekable without having to check.
>

Essentially, there is no difference between bool canSeek and  
SeekableStream seekable(). However, it's almost always a good idea to  
check whether a stream actually supports seeking before doing so, and  
seekable() statically enforces you to do so.

>>
>> InputStream doesn't really has many methods:
>>
>> interface InputStream
>> {
>> 	// reads up to buffer.length bytes from a stream
>> 	// returns number of bytes read
>> 	// throws on error
>> 	size_t read(ubyte[] buffer);
>>
>> 	// reads from current position
>> 	AsyncReadRequest readAsync(ubyte[] buffer, Mailbox* mailbox = null);
>> }
>
> I'd say void[] is better here, since you aren't creating the buffer,  
> you're accepting it.  Using ubyte makes for awkward casts when you are  
> reading binary data into specific structures.
>
> ditto for OutputStream.
>
> -Steve


-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/