std.stream replacement

[Steven Schveighoffer]

On Tue, 05 Mar 2013 14:12:58 -0500, Dmitry Olshansky  
<dmitry.olsh at gmail.com> wrote:

> 05-Mar-2013 22:49, Steven Schveighoffer пишет:

>> Everything else is read directly in (I have some heuristics to
>> try and prevent tiny reads, so if you want to say read 4 bytes, it will
>> first fill the stream buffer, then copy 4 bytes).
>
> This seems a bit like C one iff it's a smart libc. What if instead you  
> read more then requested into target buffer (if it fits)? You can tweak  
> the definition of read to say "buffer no less then X bytes, the actual  
> amount is returned" :)
>
> And if one want the direct and dumb way of get me these 4 bytes - just  
> let them provide fixed buffer of 4 bytes in total, then std.io can't  
> read more then that. (Could be useful to bench OS I/O layer and such)
> Another consequence is that std.io wouldn't need to allocate internal  
> buffer eagerly for tiny reads (in case they actually show up).
>

The way I devised it is a "processor" delegate.  Basically, you provide a  
delegate that says "yep, this is enough".  While it's not enough, it keeps  
extending and filling the extended buffer.

Which buffer is used is your call, if you want it to use it's internal  
buffer, then it will, extending as necessary (I currently only use D  
arrays and built-in appending/extending).

Here is the a very simple readline implementation (only supports '\n',  
only supports UTF8, the real version supports much more):

const(char)[] readline(InputStream input)
{
    size_t checkLine(const(ubyte)[] data, size_t start)
    {
        foreach(size_t i; start..data.length)
           if(data[i] == '\n')
              return i+1; // consume this many bytes
        return size_t.max; // no eol found yet.
    }

    auto result = cast(const(char)[]) input.readUntil(&checkLine);
    if(result.length && result[$-1] == '\n')
       result = result[0..$-1];
    return result;
}

Note that I don't have to care about management of the return value, it is  
handled for me by the input stream.  If the user intends to save that for  
later, he can make a copy.  If not, just process it and move on to the  
next line.

There is also an appendUntil function which takes an already existing  
buffer and appends to it.

Also note that I have a shortcut for what is probably a very common  
requirement -- read until a delimiter is found.  That version accepts  
either a single ubyte or a ubyte array.  I just showed the above for  
effect.

input.readUntil('\n');

also will work (for utf-8 streams).

-Steve