[phobos] Fwd: [Issue 4025] New: Making network with the std.stdio.File interface

[Steve Schveighoffer]

Sorry, I meant to reply to the group, but my email client just sent it to Andrei.

-Steve



----- Original Message ----
> From: Steve Schveighoffer <schveiguy at yahoo.com>
> To: Andrei Alexandrescu <andrei at erdani.com>
> Sent: Thu, April 8, 2010 8:14:50 AM
> Subject: Re: [phobos] Fwd: [Issue 4025] New: Making network with the std.stdio.File interface
> 
> ----- Original Message ----

> From: Andrei Alexandrescu <
> ymailto="mailto:andrei at erdani.com" 
> href="mailto:andrei at erdani.com">andrei at erdani.com>
> Subject: Re: 
> [phobos] Fwd: [Issue 4025] New: Making network with the std.stdio.File 
> interface
> 
> On 03/30/2010 06:03 AM, Steve Schveighoffer 
> wrote:
> > While I believe ranges can 
> > be useful for 
> streams, they are not the
> > best interface for all 
> > 
> applications.  For example, if I have a
> > protocol that reads 2 
> bytes 
> > to get a length, and then reads length
> > bytes 
> from the stream, a range of 
> > those units is probably a good
> 
> > abstraction.  But I don't want to 
> > resort to C calls to 
> create that
> > abstraction -- there should be a nice D 
> > 
> layer in between.  I should
> > not have to create my own 
> buffering 
> > solution.  I/O performance is
> > more 
> important IMO than interface 
> > when it comes to streams.  
> This
> > does not mean big-O complexity, I'm 
> > talking 
> about raw performance.
>
> I think the interface you described can 
> be 
> easily modeled as a range of ubyte[]. Again, a range of ubyte[] 
> doesn't have to 
> return the same number of elements each step.

I 
> agree that a range of such packets is possible, but the problem is, how does one 
> build such a range?  If
all you have as an interface to a network socket 
> is a range, how do you
make *that* range spit out what you need?  In 
> other words, you want a D
abstraction to the syscalls, I think we all agree 
> on that.  I don't
think a range abstraction is good enough for all 
> purposes.

In my example, I'd want at least a "read this many bytes" 
> interface, so
I could read the initial length, then read the remaining bytes 
> in the
packet using that length.

If you follow through the logic of 
> how such a system would be built, I think the best abstraction is a layer that 
> abstracts out the read/write functionality (unbuffered), and then build 
> ranges/buffered i/o on top of that.  The abstraction can be compile-time, 
> we don't need to do interfaces here.

> > I hope we can see a 
> 
> > design before you commit to doing it this way.
> > For 
> example, a zip 
> > library uses a range as a source, what does 
> the
> > file range look like 
> > that satisfies the range 
> properties and also is
> > efficient?  Just 
> > seeing 
> the API should be enough to judge.
>
>Feel free to suggest one. I 
> 
> won't be able until I study the zip file format.

I don't plan 
> to.  Zip files are not my forte.  My question was about the statement 
> "We concluded that the zip file library should know nothing about files or file 
> streams. It should work with ranges."  Does someone here know what 
> functional requirements the zip format needs for input?

A simple answer 
> here would be: "A x range of type T"  where you substitute x for input, 
> forward, etc. and T for the type returned by front.  And I'm not talking 
> about the zip library, I'm talking about the generic file/network stream.  
> It can't know that it's being used by a zip 
> function.

-Steve