Streaming library

[Denis Koroskin]

On Thu, 14 Oct 2010 03:06:30 +0400, Andrei Alexandrescu  
<SeeWebsiteForEmail at erdani.org> wrote:

> On 10/13/10 17:23 CDT, Denis Koroskin wrote:
>> On Thu, 14 Oct 2010 02:01:24 +0400, Andrei Alexandrescu
>> <SeeWebsiteForEmail at erdani.org> wrote:
>>
>>> On 10/13/10 16:05 CDT, Denis Koroskin wrote:
>>>> On Thu, 14 Oct 2010 00:19:45 +0400, Andrei Alexandrescu
>>>> <SeeWebsiteForEmail at erdani.org> wrote:
>>>
>>> To substantiate my brief answer:
>>>
>>>> This library code needs to be put somewhere. I just believe it belongs
>>>> to line-reader, not a generic stream. By putting line reading into a
>>>> stream interface, you want make it more efficient.
>>>
>>> I assume you meant "won't" instead of "want". So here you're saying
>>> that line-oriented I/O does not belong in the interface because it
>>> won't make things more efficient.
>>>
>>> But then your line reading code is extremely efficient by using the
>>> interface you yourself embraced.
>>>
>>
>> By adding readln() to the stream interface you will only move that code
>> from ByLine to Stream implementation. The code would be still the same.
>>
>> How can you make it any more efficient? I've read fgets() source code
>> that comes with Microsoft CRT, and it does exactly the same what I did
>> (i.e. fill buffer, read byte-by-byte, copy to output string).
>> It also puts restrictions on line size while I didn't (that's why I had
>> to use an Appender). I also did a line copy (dup) so that I could return
>> immutable string.
>>
>> You see, it's not the Stream interface that make that code less
>> efficient, it's the additional functionality over C API it provides.
>
> Gnu offers two specialized routines:  
> http://www.gnu.org/s/libc/manual/html_node/Line-Input.html. It is many  
> times more efficient than anything that can be done in client code using  
> the stdio API. I'm thinking along those lines.
>

I can easily implement similar interface on top of chunked read:
ubyte[] readLine(ubyte[] lineBuffer); or bool readLine(ref ubyte[]  
lineBuffer);

I've quickly looked through an implementation, too, and it's still filling  
a buffer first, and then copying character byte-by-byte to the output  
string (making realloc when needed) until a delimiter is found.
It is exactly as efficient as implemented externally. It does the same  
amount of copying and memory allocations. "Many times more efficient" is  
just an overestimation.

BTW, did you see my message about std.concurrency?

>>> Andrei
>>>
>>> P.S. I think I figured the issue with your fonts: the header
>>> Content-type contains "charset=KOI8-R". That charset propagates
>>> through all responses. Does anyone know how I can ignore it?
>>
>> I've changed that to utf-8. Did it help?
>
> Yes, looking great. Thanks!
>
>
> Andrei