[phobos] Fwd: [Issue 4025] New: Making network with the std.stdio.File interface
Steve Schveighoffer
schveiguy at yahoo.com
Tue Mar 30 04:03:39 PDT 2010
Can you detail the reader/writer operations? Because popNext and front ain't gonna cut it for streams. Most of the time the element size you want is defined by the application (and not easily abstracted), not the range, but the range is in charge of the element size (via front).
While I believe ranges can be useful for streams, they are not the best interface for all applications. For example, if I have a protocol that reads 2 bytes to get a length, and then reads length bytes from the stream, a range of those units is probably a good abstraction. But I don't want to resort to C calls to create that abstraction -- there should be a nice D layer in between. I should not have to create my own buffering solution. I/O performance is more important IMO than interface when it comes to streams. This does not mean big-O complexity, I'm talking about raw performance.
I hope we can see a design before you commit to doing it this way. For example, a zip library uses a range as a source, what does the file range look like that satisfies the range properties and also is efficient? Just seeing the API should be enough to judge.
And are there plans to make a good abstracted library for streams that custom ranges can be built upon?
-Steve
----- Original Message ----
> From: Walter Bright <walter at digitalmars.com>
>
> I had a long conversation with Andrei about a similar topic, i.e. dealing with
> zip files. We concluded that the zip file library should know nothing about
> files or file streams. It should work with ranges. Similarly, designs like "make
> this memory buffer look like a file stream" are completely wrong for modern
> designs. It should be "make a file look like a range" and "make a memory buffer
> look like a range" and then have the reader/writer operate on ranges.
The
> same should hold for network access.
Steve Schveighoffer wrote:
> I
> have no problem with Adam joining. As far as the network library, do we
> want to increase our dependency on the C FILE* structure/api? I would not
> want that. I don't know a lot of network solutions that put a socket fd
> inside a FILE *, there's probably good reason for that.
>
> As long
> as there's a non-buffered solution that allows socket-like functions accessible,
> I'm cool with File wrapping a socket. That is, there should be a way to
> create a socket with the API that doesn't use File.
>
> I hope we
> will eventually get a D-only solution for File. Good news is, any future
> improvements to File automatically translate to what Adam is working on.
>
>
> -Steve
>
>
>
> ----- Original Message
> ----
>
>> From: Andrei Alexandrescu <
> ymailto="mailto:andrei at erdani.com"
> href="mailto:andrei at erdani.com">andrei at erdani.com>
>> To:
> Discuss the phobos library for D <
> href="mailto:phobos at puremagic.com">phobos at puremagic.com>
>>
> Sent: Sun, March 28, 2010 7:21:20 PM
>> Subject: [phobos] Fwd: [Issue
> 4025] New: Making network with the std.stdio.File interface
>>
>
>> I'm quite hopeful about this development. If we have anything for
> the network in D2, that would be great, and integration with File will reuse a
> ton of work.
>>
>
> I'm considering
> extending Adam an offer to join Phobos. He has
>> donated a
> Windows machine for building and running Phobos and generally seems to know what
> he's talking about. If he'll join the project, my hope is to attract him to
> dedicate more time to it.
>>
>
> Please
> cast your vote by replying
>> (silence = abstaining, i.e.
> you're not for or against).
>>
>
>
>
> Andrei
>
> -------- Original Message
>>
> --------
>>
> Subject: [Issue 4025] New: Making
> network with the std.stdio.File
>> interface
>>
>
> Date: Sun, 28 Mar 2010 22:51:45 +0000 (UTC)
> From:
>
>> ymailto="mailto:
> href="mailto:d-bugmail at puremagic.com">d-bugmail at puremagic.com"
> href="mailto:
> href="mailto:d-bugmail at puremagic.com">d-bugmail at puremagic.com">
> ymailto="mailto:d-bugmail at puremagic.com"
> href="mailto:d-bugmail at puremagic.com">d-bugmail at puremagic.com
>>
>
> Organization:
>> Digital
> Mars
>>
> To:
>> href="mailto:
> ymailto="mailto:digitalmars-d-bugs at puremagic.com"
> href="mailto:digitalmars-d-bugs at puremagic.com">digitalmars-d-bugs at puremagic.com">
> ymailto="mailto:digitalmars-d-bugs at puremagic.com"
> href="mailto:digitalmars-d-bugs at puremagic.com">digitalmars-d-bugs at puremagic.com
>>
>
> Newsgroups:
>>
> digitalmars.D.bugs
>>
>
>
> http://d.puremagic.com/issues/show_bug.cgi?id=4025
>
>
>
>> Summary: Making network with the
> std.stdio.File interface
>>
>
> Product: D
>
>>
> Version: 2.041
>>
>
>
>> Platform:
> Other
>>
> OS/Version:
> Linux
>
>>
> Status: NEW
>>
>
>
>> Severity: enhancement
>>
>
> Priority:
>>
> P2
>>
>
> Component: Phobos
>
>>
> AssignedTo: href="mailto:
> href="mailto:nobody at puremagic.com">nobody at puremagic.com">
> ymailto="mailto:nobody at puremagic.com"
> href="mailto:nobody at puremagic.com">nobody at puremagic.com
>>
>
>
>> ReportedBy:
> href="mailto:
> href="mailto:destructionator at gmail.com">destructionator at gmail.com">
> ymailto="mailto:destructionator at gmail.com"
> href="mailto:destructionator at gmail.com">destructionator at gmail.com
>>
>
>
>
> ---
>> Comment #0 from Adam
> D. Ruppe <
>> href="mailto:
> ymailto="mailto:destructionator at gmail.com"
> href="mailto:destructionator at gmail.com">destructionator at gmail.com">
> ymailto="mailto:destructionator at gmail.com"
> href="mailto:destructionator at gmail.com">destructionator at gmail.com>
> 2010-03-28 15:51:43 PDT ---
>>
> I've written a
> small module that opens a network
>> connection, then wraps it
> in
>>
> the File struct, allowing you to use the
> byLine,
>> etc., ranges on it, as well as
>>
>
> writef/readln/etc.
>
> It is Linux only,
>
>> but should be pretty easy to port to other operating
> systems.
>>
> Ideally, I'd
>>
> like to eventually be able to use File for talking
>>
>
> to processes too, on all D
>>
> platforms.
>>
>
> Here's the code I have
> for
>> now:
>>
>
>
> ==============
>
>
>
> public import
> std.stdio;
> import
>> std.string;
>>
>
>
> import std.conv;
>
>
> version(linux):
>
> import
>>
> std.c.linux.linux;
>>
> import
> std.c.linux.socket;
>
> alias std.c.linux.socket
>
>> sock;
>>
> alias std.c.linux.linux
> linux;
>
> enum int PF_INET = 2;
> enum int
>
>> AF_INET = PF_INET;
>>
>
>
> extern(C) FILE* fdopen(int, const(char)*);
>
> File
>
>> openNetwork(string host, ushort port) {
>>
>
> hostent* h;
>
>>
> sockaddr_in addr;
>>
>
> h
> =
>>
> gethostbyname(std.string.toStringz(host));
>>
>
> if(h is
>> null)
>>
>
> throw new
>>
> StdioException("gethostbyname");
>>
>
>
> int s = socket(PF_INET,
>> SOCK_STREAM,
> 0);
>>
> if(s == -1)
>
>
>> throw new
> StdioException("socket");
>>
>
>
>
>> scope(failure)
>>
>
> close(s);
>
>
>
>> addr.sin_family = AF_INET;
>>
>
> addr.sin_port =
>>
> htons(port);
>>
>
> std.c.string.memcpy(&addr.sin_addr.s_addr,
>> h.h_addr,
> h.h_length);
>>
>
>
> if(sock.connect(s, cast(sockaddr*)
>> &addr, addr.sizeof)
> == -1)
>>
> throw
> new
>> StdioException("Connect failed");
>>
>
>
> FILE* fp = fdopen(s,
>>
> "w+".ptr);
>>
>
> File
> f;
>
> auto imp = new
>>
> File.Impl(fp, 1, host ~ ":" ~ to!string(port));
>>
>
>
> f.p =
>> imp;
>>
>
>
> return
>>
> f;
>>
> }
>
> _______________________________________________
> phobos mailing
>
>> list
>>
>
>
>
>> href="mailto:
> href="mailto:phobos at puremagic.com">phobos at puremagic.com">
> ymailto="mailto:phobos at puremagic.com"
> href="mailto:phobos at puremagic.com">phobos at puremagic.com
>>
>
> http://lists.puremagic.com/mailman/listinfo/phobos
>
>
>
>
> _______________________________________________
> phobos mailing
> list
>
> href="mailto:phobos at puremagic.com">phobos at puremagic.com
>
> href="http://lists.puremagic.com/mailman/listinfo/phobos" target=_blank
> >http://lists.puremagic.com/mailman/listinfo/phobos
>
>
>
>
_______________________________________________
phobos
> mailing list
> href="mailto:phobos at puremagic.com">phobos at puremagic.com
> href="http://lists.puremagic.com/mailman/listinfo/phobos" target=_blank
> >http://lists.puremagic.com/mailman/listinfo/phobos
More information about the phobos
mailing list