Curl wrapper

[jdrewsen]

Please see comments below.

Den 17-05-2011 16:42, Andrei Alexandrescu skrev:
> Thanks for the response! A few more answers and comments within
> (everything deleted counts as "sounds great").
>
> On 5/17/11 3:50 AM, Jonas Drewsen wrote:
>>> 14. Isn't the max redirect configurable via a parameter?
>>
>> Yes. It is called Http.followLocation from libcurls followLocation
>> option. I will rename it to maxRedirections for clearity.
>
> You mention this about many higher-level functions: "Maximum
> redirections are set to 10." Then I'm thinking, if it's an important
> parameter, make it a defaulted parameter for the function.

I guess I should just remove that comment because other defaults has 
been selected as well e.g. timeouts.

>>> 16. Documentation should point to descriptions of the HTTP methods
>>> wrapped (e.g. "post", "del" etc).
>>
>> Do you mean point to the relevant RFC?
>
> Yah, or a good tutorial. (I didn't know DEL existed!)

Well DEL is actually called DELETE in the HTTP RFC. But delete is a 
reserved word i D so I used del() instead. delete() wouldn't work in 
following code in think:

with(auto http = ...) {
	delete(...);
}

>>> 22. byLine/byChunk should not expose a string or generally data that can
>>> be shared safely by the client. That's because it would need to create a
>>> new string with each iteration, which is very inefficient. Instead, they
>>> should expose char[]/ubyte[] just like File.byLine does, and reuse the
>>> buffer with each call. Essentially byLine/byChunk should be
>>> near-zero-overhead means to transfer and inspect arbitrarily large
>>> streams.
>>
>> byLine/byChunk is currenly only available for the async methods which is
>> implemented using message passing. This means that for passing the
>> message a copy has to be made because a message is by-value or immutable
>> data. Is there another way for me to pass the message without doing a
>> copy... some kind of move semantic?
>
> A library element is planned but not on the short list yet. You can work
> around that by internally doing a cast. As long as you observe the
> commitment that buffers are not accessed simultaneously by more than one
> thread you're well within defined behavior.

ok nice to know.

> One more suggestion - you may want to also provide classic boring
> synchronous read/write methods that take a user-provided buffer. I'm
> sure people could use such e.g. when they want to stream data inside
> their own threads, or even in the main thread if they don't mind blocking.

This would mean hooking into libcurl and selecting on its sockets. 
Totally doable. But I would rather stop here feature wise and wrap this 
thing up. I would like to focus my efforts on a candidate for 
std.net.event/reactor/proactor module.

Anyone who wants to implement this read/write in the curl wrapper are 
ofcourse very welcome.


>>> 24. The shutdown mechanism should be handled properly. Shutting down
>>> libcurl would have all pending transfers instantly throw a special
>>> exception. Without a shutdown API, applications won't be able to
>>> implement e.g. a Quit button.
>>
>> If you're only using this API this should be handled. But I see that it
>> is not documented.
>
> Yah, in the future we need to put an onShutdown protocol inside core,
> similar to atexit() in C. Each library that may block would plant a
> hook. That way an application will be able to exit quickly and
> gracefully even if it has blocked threads.

That would be neat.

/Jonas