Streaming library

[s50]

I think copy-less interfaces basically can be implemented on
top of chunked(or block-oriented) read(or buffer) also w/o extra
inefficiencies.
Recently in phobos ML, someone has suggested a archetypal
buffering interface allows us to build upper layers which have
copy-less interfaces. If it would be polished more, probably
we can find a point of compromise.

2010/10/14 Andrei Alexandrescu <SeeWebsiteForEmail at erdani.org>:
> On 10/13/2010 06:23 PM, Denis Koroskin wrote:
>>
>> On Thu, 14 Oct 2010 03:06:30 +0400, Andrei Alexandrescu
>> <SeeWebsiteForEmail at erdani.org> wrote:
>>>
>>> Gnu offers two specialized routines:
>>> http://www.gnu.org/s/libc/manual/html_node/Line-Input.html. It is many
>>> times more efficient than anything that can be done in client code
>>> using the stdio API. I'm thinking along those lines.
>>>
>>
>> I can easily implement similar interface on top of chunked read:
>> ubyte[] readLine(ubyte[] lineBuffer); or bool readLine(ref ubyte[]
>> lineBuffer);
>
> You can't.
>
>> I've quickly looked through an implementation, too, and it's still
>> filling a buffer first, and then copying character byte-by-byte to the
>> output string (making realloc when needed) until a delimiter is found.
>> It is exactly as efficient as implemented externally.
>
> Except you don't have an interface to copy byte by byte. Oops...
>
>> It does the same
>> amount of copying and memory allocations. "Many times more efficient" is
>> just an overestimation.
>
> It's not. I measured because it was important in an application I was
> working on. It's shocking how some seemingly minor changes can make a big
> difference in throughput.
>
>> BTW, did you see my message about std.concurrency?
>
> Yes, but I'll need to leave the bulk of it to Sean. Thanks.
>
>
> Andrei
>