etc.curl: Formal review begin

[Johannes Pfau]

Walter Bright wrote:
>On 8/17/2011 4:12 PM, David Nadlinger wrote:
>> the etc.curl module by Jonas Drewsen is at the front of the review
>> queue.
>
>Thanks for doing this. I preface my remarks with saying I have never
>done http: programming and know little about it.
>
>1. The first, most obvious thing I'd like to do is present a URL and
>get a string back that is the contents of that web page:
>
>     string contents = getURL("http://www.digitalmars.com");
>

That's the first example in the API docs:

// Simple GET with connect timeout of 10 seconds
Http.get("http://www.google.com").connectTimeout(dur!"seconds"(10)).toString();

or, if you don't care about timeouts:
string contents = Http.get("http://www.google.com").toString();

>and contents[] should be the web page; what I'd see if I opened it in
>a browser and selected "view source". From perusing the etc.curl doc I
>have absolutely no idea how to do this.
>
>2. I'd like to get the contents as an input range, which I can then
>iterate like any other range.

auto res = Http.getAsync("http://www.digitalmars.com");
auto range = res.byChunk(100);
writeln(range.front);
range.popFront();
writeln(range.front);

There's only a byChunk for synchronous requests, but I think this first
reads the complete Result into memory and then returns slices. (This
is a limitation caused by curl's API, as curl uses callbacks) 

>
>3. A lot of web sites like twitter.com have an API. How do I access
>that API from D?

The twitter API is a REST API, so if you want to get information from
twitter, you use a GET request with a special url:

string xmlTimeLine =
Http.get("http://twitter.com/statuses/user_timeline.xml?id=walterbright").toString();

Posting is a little more complicated, as it requires authentication:

ubyte[] result;
Http twitter =
Http("http://api.twitter.com/1/statuses/update.format?status=Hello%20World");
twitter.setAuthentication("user", "password");
twitter.onReceive = (ubyte[] data) { result ~= data; return
data.length; };
twitter.method = Http.Method.post;
twitter.perform();

(Status should be url encoded, Http.escape can be used for that)

Not sure, if this still works though. Twitter is switching from HTTP
basic authentication to OAuth. Also, this code is not tested ;-)

>
>4. How do I download a file from ftp?
>
>    ubyte[] file =
> downloadFile("ftp://www.digitalmars.com/whatever.zip");

Ftp.get("ftp://www.digitalmars.com/whatever.zip","/tmp/downloaded-file");

or

ubyte[] file = Ftp.get("ftp://www.digitalmars.com/whatever.zip").bytes;

>
>5. I'd like an NNTP interface so I can get and post newsgroup messages
>from D, though I suspect that is beyond the scope of etc.curl.


BTW: There seems to be a small problem with the API docs: Some members
are not properly indented. For example, "statusLine();" which is
a member of Result, is not indented.
-- 
Johannes Pfau
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20110824/f1e21d22/attachment-0001.pgp>