Curl support RFC

Jonas Drewsen jdrewsen at nospam.com
Sat Mar 12 08:56:24 PST 2011


Thank you.

Regarding scalability: In my experience the fastest network handling for 
multiple concurrent request is done asyncronously using select or epoll. 
The current wrapper would probably use threading and messages to handle 
multiple concurrent requests which is not as efficient.

Usually you only need this kind of scalability for server side 
networking and not client side like libcurl is providing so I do not see 
this as a major issue for an initial version.

I do know how to support epoll/select based curl and by that better 
scalability and that would fortunately just be an extension to the API 
I've shown. Currently I will focus on getting the common things finished 
and rock solid.

/Jonas


On 11/03/11 16.30, dsimcha wrote:
> I don't know much about this kind of stuff except that I use it for very simple
> use cases occasionally.  One thing I'll definitely give your design credit for,
> based on your examples, is making simple things simple.  I don't know how it
> scales to more complex use cases (not saying it doesn't, just that I'm not
> qualified to evaluate that), but I definitely would use this.  Nice work.
>
> BTW, what is the license status of libcurl?  According to Wikipedia it's MIT
> licensed.  Where does that leave us with regard to the binary attribution issue?
>
> == Quote from Jonas Drewsen (jdrewsen at nospam.com)'s article
>> Hi,
>>      So I've spent some time trying to wrap libcurl for D. There is a lot
>> of things that you can do with libcurl which I did not know so I'm
>> starting out small.
>> For now I've created all the declarations for the latest public curl C
>> api. I have put that in the etc.c.curl module.
>> On top of that I've created a more D like api as seen below. This is
>> located in the 'etc.curl' module. What you can see below currently works
>> but before proceeding further down this road I would like to get your
>> comments on it.
>> //
>> // Simple HTTP GET with sane defaults
>> // provides the .content, .headers and .status
>> //
>> writeln( Http.get("http://www.google.com").content );
>> //
>> // GET with custom data receiver delegates
>> //
>> Http http = new Http("http://www.google.dk");
>> http.setReceiveHeaderCallback( (string key, string value) {
>> 	writeln(key ~ ":" ~ value);
>> } );
>> http.setReceiveCallback( (string data) { /* drop */ } );
>> http.perform;
>> //
>> // POST with some timouts
>> //
>> http.setUrl("http://www.testing.com/test.cgi");
>> http.setReceiveCallback( (string data) { writeln(data); } );
>> http.setConnectTimeout(1000);
>> http.setDataTimeout(1000);
>> http.setDnsTimeout(1000);
>> http.setPostData("The quick....");
>> http.perform;
>> //
>> // PUT with data sender delegate
>> //
>> string msg = "Hello world";
>> size_t len = msg.length; /* using chuncked transfer if omitted */
>> http.setSendCallback( delegate size_t(char[] data) {
>>       if (msg.empty) return 0;
>>       auto l = msg.length;
>>       data[0..l] = msg[0..$];
>>       msg.length = 0;
>>       return l;
>>       },
>>       HttpMethod.put, len );
>> http.perform;
>> //
>> // HTTPS
>> //
>> writeln(Http.get("https://mail.google.com").content);
>> //
>> // FTP
>> //
>> writeln(Ftp.get("ftp://ftp.digitalmars.com/sieve.ds",
>>                   "./downloaded-file"));
>> // ... authenication, cookies, interface select, progress callback
>> // etc. is also implemented this way.
>> /Jonas
>



More information about the Digitalmars-d mailing list