Streaming library

[Andrei Alexandrescu]

On 10/13/2010 06:23 PM, Denis Koroskin wrote:
> On Thu, 14 Oct 2010 03:06:30 +0400, Andrei Alexandrescu
> <SeeWebsiteForEmail at erdani.org> wrote:
>> Gnu offers two specialized routines:
>> http://www.gnu.org/s/libc/manual/html_node/Line-Input.html. It is many
>> times more efficient than anything that can be done in client code
>> using the stdio API. I'm thinking along those lines.
>>
>
> I can easily implement similar interface on top of chunked read:
> ubyte[] readLine(ubyte[] lineBuffer); or bool readLine(ref ubyte[]
> lineBuffer);

You can't.

> I've quickly looked through an implementation, too, and it's still
> filling a buffer first, and then copying character byte-by-byte to the
> output string (making realloc when needed) until a delimiter is found.
> It is exactly as efficient as implemented externally.

Except you don't have an interface to copy byte by byte. Oops...

> It does the same
> amount of copying and memory allocations. "Many times more efficient" is
> just an overestimation.

It's not. I measured because it was important in an application I was 
working on. It's shocking how some seemingly minor changes can make a 
big difference in throughput.

> BTW, did you see my message about std.concurrency?

Yes, but I'll need to leave the bulk of it to Sean. Thanks.


Andrei