uploading with curl

Gleb s4mmael at gmail.com
Tue Apr 17 00:07:05 PDT 2012


Gentlemen,

While working with curl library, I would like to discuss a few 
more issues I faced with, if you don't mind.

Here is the code sample I use:

     auto f = new std.stream.BufferedFile("file.zip", FileMode.In);
     scope (exit) f.close();
     auto client = FTP();
     client.verbose(true);
     client.url = "ftp://192.168.110.58/file.zip";
     client.setAuthentication("user", "pass");
     client.contentLength = cast(size_t)f.size;
     client.handle.set(CurlOption.upload, 1L);
     client.onSend = (void[] data)
     {
         return f.read(cast(ubyte[])data);
     };
     client.perform();

1. TIMEOUTS

The code sample will work great if file.zip is small.
If the file is big enough, we will get the following result:

    * Connecting to 192.168.110.58 (192.168.110.58) port 1468
    > TYPE I
    < 200 Type set to I
    > STOR file.zip
    < 150 Opening BINARY mode data connection for file.zip
    * Operation timed out after 120105 milliseconds with 687751168 
bytes received
    * Closing connection #0
    * Timeout was reached 
std.net.curl.CurlTimeoutException at std\net\curl.d(3348): Timeout 
was reached on handle 16BCB60

So we were disconnected from server because of the timeout. To 
avoid this behavior I use the following:

    client.dataTimeout(dur!"weeks"(10));

I'm not sure if we really need this kind of timeout so I use the 
really big value. Anyway, I believe the default value of 120 
seconds is not enough for practical usage of file transfer 
protocol. Maybe it's suitable for HTTP only?

By the way, for some reason I can't use the value more then 10 
weeks. In this case we can't even connect to server:

    * About to connect() to 192.168.110.58 port 21 (#0)
    *   Trying 192.168.110.58...
    * connected
    * Connected to 192.168.110.58 (192.168.110.58) port 21 (#0)
    * server response timeout
    * Closing connection #0
    * Timeout was reached
    std.net.curl.CurlTimeoutException at std\net\curl.d(3348): 
Timeout was reached on handle 164CB60

Of course, even bigger value is not necessary, but I'm not sure 
this behavior is correct.


2. PROGRESSBAR

I've added the following to the above example:
     client.onProgress = delegate int(size_t dlTotal, size_t 
dlNow, size_t ulTotal, size_t ulNow)
     {
         return 0;
     };

When compiled and run we will see the following:

     > STOR file.zip
     < 150 Opening BINARY mode data connection for file.zip
     * We are completely uploaded and fine
     * Remembering we are in dir ""
     < 226 Transfer complete
     * Connection #0 to host 192.168.110.58 left intact
     > QUIT
     std.net.curl.CurlException at std\net\curl.d(3365): Progress 
callback called on cleaned up Curl instance

It looks like onProgress was called when the file was uploaded 
and the data connection was closed. What am I doing wrong? Should 
I return any other value when ulNow == ulTotal? In code examples 
in the documentation onProgress does not return any value, but it 
must.

Maybe it would be better to use some simple progress bar by 
default when curl.verbose was true? At least something like this:

     static const bStr = replicate("\b", 20);
     static float perc;
     client.onProgress = delegate int(size_t dlTotal, size_t 
dlNow, size_t ulTotal, size_t ulNow)
     {
         perc = ulTotal ? cast(real)ulNow/cast(real)ulTotal*100 : 
0;
         writef("%s* %.2f%c uploaded", bStr, perc, '%');
         return 0;
     };


Thank you in advance for your opinion on this issues.


More information about the Digitalmars-d mailing list