Should "std.net.curl" be moved from Phobos to Deimos?

[Adam D. Ruppe]

On Wednesday, 27 November 2013 at 01:30:26 UTC, Andrei
Alexandrescu wrote:
> A new interface is also on the table, but that brings the 
> additional burden of defining your own design. std.net.curl has 
> already been approved.

Aye, but that'd be worth it. There's a few pieces we should get
together:

1) a url module. My cgi.d has one that may be usable with a
little testing:
https://github.com/adamdruppe/misc-stuff-including-D-programming-language-web-stuff/blob/master/cgi.d#L2022

about 150 lines long, including some unittest. Also has some
functions that build on std.uri to make encoding multiple values
easier - working with associative arrays instead of just strings.

This is useful since we can then pass type-safe urls to various
functions and get consistent parsing and creation. The basedOn()
method in my code there is meant to do relative links in a
browser fetch situation. Regardless, all this is fairly useful
independent of network functions too.

2) Some kind of SSL sockets, including adding ssl to an already
open socket (needed for STARTTLS on smtp at least). I did a
partial implementation as a subclass of Phobos' Socket with
openssl. Should also do a Windows impl and probably a gnutls as
well.

I think this would be ideally picked at runtime, with dlopen().
If gnutls is available, make that work. Otherwise, use openssl,
and if none are there, throw upon ssl requests. On Windows, it
can just use the built in ssl facilities the operating system
provides.

I'd probably make this a new module that extends std.socket
rather than editing std/socket.d.

3) async (overlapped) I/O on those sockets. This might need
std.socket to be edited to add the functionality.



Then, we go into the curl replacement:


4) The http module. Offers simple functions (get(), post() that
just work), a reusable client object, and advanced functions, all
in sync and async versions.

My http.d can be a starting point, though I have a new api in
mind. I'll perhaps slap together a first draft over the next
week, if my other work leaves enough time. I have to write a book
chapter on Phobos soon too! lol maybe time to scratch the
std.net.curl portion.

5) The smtp module. Offers the low level send functions, and a
higher level MIME message builder class. I've written the latter
already.
https://github.com/adamdruppe/misc-stuff-including-D-programming-language-web-stuff/blob/master/email.d#L27

though it uses my html to text module, which uses my dom module,
which uses my characterencodings module. That could be easily
pulled out though, it would just require the user to set the
plain text version of an html email themselves.

As you can see there too, it currently uses std.net.curl to
actually send the email, since my smtp function (written but it
isn't great) doesn't support the starttls command that Amazon SES
uses and i needed that for a work project...

My module can also read email, but I don't think that's needed in
the std.net.email.

6) The ftp module. I haven't written ftp code for a long time, it
would perhaps be better if someone else wrote this, though if
nobody else does, I'm sure I can refresh myself well enough with
a lil time. But the beauty of modules is we can divide and
conquer!

7) Any other protocols we care about. Personally, I'd kinda love
some ssh and scp code... but I'm afraid actually writing that is
something I don't want to do, perhaps we can do a wrapper for
libssh. Perhaps not appropriate for phobos due to dependency and
licensing but I'd still love someone to write it and post a link
here!

8) If we want to, there could be a combined module that reads a
runtime url and dispatches to the appropriate protocol, like the
std.net.curl.download and upload can do. But tbh I don't even
care much about this. It is trivial once the other modules are
written anyway.

9) Maybe my OAuth module's client code can be cleaned up and 
added too, as an add-on to the http functionality.


Each piece here is pretty independent, except the first three,
and I have enough written already for them to get started at
least.