[phobos] Fwd: [Issue 4025] New: Making network with the std.stdio.File interface

[Andrei Alexandrescu]

Somehow the quotes got messed pretty badly during this exchange. I'll 
trim most of the content, please refer to the original message if you 
come to this later.

On 04/08/2010 03:01 PM, Steve Schveighoffer wrote:
> But I'm saying, the times where we need to intermingle with C are
> only for the standard handles, it seems that's what you're saying
> also, but you worded it in a way that makes it sound like you
> disagree with me...  Confused.

I worded it in a way that clarifies it's about more than printf.

> Yes, if your application processes 1024 bytes at a time, it is easier
> to use a range.  That's not the application I'm referring to.  The
> application I'm talking about is when you need to read a different
> amount of bytes per read, such as a varying length packet.  This is
> not an uncommon situation.

I think the best way to make progress is to compare the range I 
suggested with another one. For example, if having rawRead as a range 
primitive is necessary, by all means let's make rawRead part of it.

> Let's look at that version with your range:
>
> while(!input.empty()) { input.bufsize = numtoread; input.popFront();
> auto data = input.front();
>
> // process data. }
>
> and with File's rawRead:
>
> ubyte buf[MAXSIZE]; ubyte[] data; while((data =
> input.rawRead(buf[0..numtoread])).length) { // process data. }
>
> And look, we can use the stack for buffering!  Plus, we don't have to
> worry about whether the data buffer will be overwritten, we control
> what buffer is used by the input object, so we can manage that less
> defensively.

This is a weak argument. Buffer allocation is hardly a bottleneck for 
streaming application. The code above would not work in SafeD.

> Also, let's not forget that you can easily bolt an input range
> interface on top of a file interface (as evidenced by byChunk), but
> you can't do the opposite.  For example, reading a packet at a time
> from a network/file stream given a length can easily be implemented
> with a range on top of a File struct, but not easily with a range on
> top of a range.

Indeed. A range that adapts variable-length packets to fixed-length 
packets would need to know some more details. I'm not sure we need such 
an abstraction, but if we do, we can define it.

> So essentially, the idea is to double-buffer the data, once inside
> the range (to support the front/popFront regime) and once for your
> application, so you can build up enough "chunks" to read the data
> correctly?

We are in agreement that we shouldn't be doing that.

>> We need to figure out all this stuff together, but so far I'm not
>> at all convinced that seekable ranges are awkward.
>
> I may not have explained myself well, I don't have a big problem with
> seekable ranges for certain applications, I just don't think they are
> the primitive that should be used for all applications.

I confess I'm not sure what you want and how to get from where we are to 
where you want us to be.


Andrei