Should "std.net.curl" be moved from Phobos to Deimos?

[Andrei Alexandrescu]

On 11/25/13 4:36 PM, Jonathan M Davis wrote:
> The number one problem on Windows is the fact that libcurl does not come with
> Windows and that you have to get a version of it build which is compatible
> with dmd has proven to be a huge hurdle for Windows developers.

Why is "compatible with dmd" an issue? Do we make it any harder for 
libcurl than other clients? Could you please explain?

> There may be
> other problems, but that's the biggest. Distributing libcurl with dmd could fix
> that, but IIRC, there was some reason why we couldn't do that (or maybe it was
> just that Walter didn't want to - I can't remember).

Walter is opposed only to building libcurl from source for distributing 
it, which is an understandable stance.

> On Linux, I believe that the biggest problem is that dmd then has to be built
> on your specific distro for it to work with your distro, so the libraries in
> the zip file were only working on debian-based systems. Regardless of why
> libcurl is not distributed with dmd on Windows, we couldn't distribute for
> Linux to fix this problem, because that would conflict with the system's
> libcurl.

So what is the classic recommended solution for such cases? We can't be 
the first people who are having this issue on Unix.

> There may be other problems, but I think that those are the two bigs ones, and
> they've come up several times (particularly the issues with libcurl and
> Windows).

I've asked the question "what exactly is the problem" several times and 
you came closest to answering it, but a bunch of stuff is still unclear 
to me.

> There's also Don's recent rant on why curl was ever supported in the first
> place, but I guess that he just wasn't paying enough attention to it given
> that he only complained about it recently rather than during the review:
>
> http://forum.dlang.org/thread/CADpwU1cu=0r+NJ+4GN9P9Fd_XisK1dsduYrrkjpGAvM0YJ_tUA@mail.gmail.com

First, it's not like we slipped curl in while nobody was looking. 
Second, I'm sure curl has its shortcomings, but I'd be hard pressed to 
find better alternatives.

As to just dumping curl and doing our own: I just looked and libcurl has 
160K lines. Those include examples and probably a fair amount of header 
chaff etc. that's unneeded in D, but I'd assess there are on the order 
of 100K lines of "pulp" code that does real work. Simply dumping 
std.net.curl would not only break existing code that depends on it in 
unthinkable ways (no replacement and no recourse), but would put us some 
100KLOC behind in terms of a working replacement.

> At this point, I'm inclined to argue that Phobos should not depend on any
> libraries other than the system libraries for each OS so that we can avoid
> further dependencies that may not be available - especially when it comes to
> Windows, since Windows comes with no 3rd party development libraries at all.
> So, I would very much be against std.net.curl's inclusion at this point and
> would argue that it should be in a library separate from Phobos and that if we
> want similar functionality in Phobos, we need to implement it without relying
> on anything other than system libraries. And from previous discussions on the
> topic, I believe that Walter and several other dmd/Phobos devs agree with
> that.

I don't agree with that. Or I don't understand what you're asking. If 
what you're asking is we break all existing code using std.net.curl with 
no suggested replacement, that would be pretty bad, no?

If I understand correctly the complaint is: "people who don't have curl 
can't use std.net.curl". That's to be expected. It's in the name!

A more reasonable complaint is: "people who don't have curl can't use 
high-level HTTP/FTP client networking, which is expected of a modern 
language's standard library". For those we should NOT throw away 
libcurl. We should instead focus on providing e.g. std.net.durl which is 
an in-house library for doing great stuff. Then people can choose. In 
the long term we may even recommend moving away from curl because durl 
is mature and better etc.

> So, I'm all for removing std.net.curl, but if we do that, we'd obviously need
> to deprecate it first rather than simply removing it, and we'd need to have a
> separate library with std.net.curl in it (preferably one which could be pulled
> in via dub) which people could switch to using instead. Whether that library
> would be maintained by us, by Jonas (since he created it), or someone else, I
> don't know, but ideally, it wouldn't be in Phobos any longer. At minimum, I
> think that we should avoid putting any more 3rd party dependencies in Phobos
> in the future.

This is probably going to come again. We can't build everything in house 
in due time. Do we have an openssl layer written in D?


Andrei