etc.curl: Formal review begin

[Johannes Pfau]

Walter Bright wrote:
>On 8/24/2011 2:18 AM, Johannes Pfau wrote:
>> Walter Bright wrote:
>>> On 8/17/2011 4:12 PM, David Nadlinger wrote:
>>>> the etc.curl module by Jonas Drewsen is at the front of the review
>>>> queue.
>>>
>>> Thanks for doing this. I preface my remarks with saying I have never
>>> done http: programming and know little about it.
>>>
>>> 1. The first, most obvious thing I'd like to do is present a URL and
>>> get a string back that is the contents of that web page:
>>>
>>>      string contents = getURL("http://www.digitalmars.com");
>>>
>>
>> That's the first example in the API docs:
>>
>> // Simple GET with connect timeout of 10 seconds
>> Http.get("http://www.google.com").connectTimeout(dur!"seconds"(10)).toString();
>
>Ok, but 1. there's no indication in the doc that that's what it does
>because 1. it is an incomplete fragment and 2. a simpler interface
>(like what I proposed) would likely be better.
>

I agree, I've included some suggestions on the API docs in my review
comment. I think the big code example should be split into many smaller
examples with better explanations.

>> or, if you don't care about timeouts:
>> string contents = Http.get("http://www.google.com").toString();
>
>This example needs to be first and needs an explanation. Timeouts,
>error recovery, progress indicators, etc., should all be brought up
>later.
>

I completely agree.
>
>>> and contents[] should be the web page; what I'd see if I opened it
>>> in a browser and selected "view source". From perusing the etc.curl
>>> doc I have absolutely no idea how to do this.
>>>
>>> 2. I'd like to get the contents as an input range, which I can then
>>> iterate like any other range.
>>
>> auto res = Http.getAsync("http://www.digitalmars.com");
>> auto range = res.byChunk(100);
>> writeln(range.front);
>> range.popFront();
>> writeln(range.front);
>>
>> There's only a byChunk for synchronous requests, but I think this
>> first reads the complete Result into memory and then returns slices.
>> (This is a limitation caused by curl's API, as curl uses callbacks)
>
>Ok.
(I wanted to say there's _also_ a byChunk for synchronous requests, but
I guess you already figured that out)

>
>>> 3. A lot of web sites like twitter.com have an API. How do I access
>>> that API from D?
>>
>> The twitter API is a REST API, so if you want to get information from
>> twitter, you use a GET request with a special url:
>>
>> string xmlTimeLine =
>> Http.get("http://twitter.com/statuses/user_timeline.xml?id=walterbright").toString();
>>
>> Posting is a little more complicated, as it requires authentication:
>>
>> ubyte[] result;
>> Http twitter =
>> Http("http://api.twitter.com/1/statuses/update.format?status=Hello%20World");
>> twitter.setAuthentication("user", "password");
>> twitter.onReceive = (ubyte[] data) { result ~= data; return
>> data.length; };
>> twitter.method = Http.Method.post;
>> twitter.perform();
>>
>> (Status should be url encoded, Http.escape can be used for that)
>>
>> Not sure, if this still works though. Twitter is switching from HTTP
>> basic authentication to OAuth. Also, this code is not tested ;-)
>
>I'm glad to see this can be done. A complete example in the
>documentation would be very informative and useful.
>

Someone with a twitter account should test it and then it could be added
as an example. BTW: Thinking about it this should also be possible with
the 'simple' API:

auto result =
Http.post(http://api.twitter.com/1/statuses/update.format,
["status" : "Hello%20World"]).authentication("user",
"password").toString();

>>> 4. How do I download a file from ftp?
>>>
>>>     ubyte[] file =
>>> downloadFile("ftp://www.digitalmars.com/whatever.zip");
>>
>> Ftp.get("ftp://www.digitalmars.com/whatever.zip","/tmp/downloaded-file");
>>
>> or
>>
>> ubyte[] file =
>> Ftp.get("ftp://www.digitalmars.com/whatever.zip").bytes;
>
>Great! Please add to the documentation.
>
>
>I also think that etc.curl would benefit greatly from an article
>written about it showing how to use it to accomplish various fun and
>simple tasks.



-- 
Johannes Pfau