Second Round CURL Wrapper Review

[Jonas Drewsen]

Den 03-12-2011 21:58, Vladimir Panteleev skrev:
> On Sat, 03 Dec 2011 21:17:25 +0200, Jonas Drewsen <jdrewsen at nospam.com>
> wrote:
>
>> Den 03-12-2011 13:10, Vladimir Panteleev skrev:
>>> On Sat, 03 Dec 2011 13:53:16 +0200, Jonas Drewsen <jdrewsen at nospam.com>
>>> wrote:
>>>
>>>> As mentioned the async version performs the request in another thread
>>>> leaving the main thread available for you to do something else. I'll
>>>> clarify in the docs that it is when you actually call empty/front on
>>>> the returned range you will get data from the other thread and may be
>>>> blocking.
>>>
>>> I can't think of a realistic use case for the current asynchronous API.
>>> Basically, all you can do is start a request in the background, but
>>> you're neither notified of the request nor can you poll it to check its
>>> status? So the only thing you CAN do is ask for a request that you will
>>> need in the future, and when that future moment comes, block if
>>> necessary to get the result?
>>
>> The standard example is downloading some content and saving it at the
>> same time.
>>
>> While your main thread saves a chunk to disk (or uploads to another
>> server) the async thread is busy buffering incoming chunks of data.
>> This means that you effectively only wait for the slowest of the two
>> IO operations. If you did it synchronously would worst case have to
>> wait for all everything to be downloaded and the wait for everything
>> to be saved or uploaded.
>>
>> foreach(chunk; byChunkAsync("www.abc.com/hugefile.bin"))
>> {
>> // While writing to file in this thrad
>> // new chunks are downloaded
>> // in the background thread
>> file.write(chunk);
>> }
>>
>> I hope this makes sense.
>
> Well, this makes sense from a theoretical / high-level perspective, but
> OS write buffers greatly reduce the practicality of this. In common use
> cases the speed difference between disk and wire will differ by orders
> of magnitude as well.
>

Read/write buffers does indeed help a lot and there have been quite some 
discussions on this topic earlier in this newsgroups regarding tradeoffs 
etc. Please have a look at these threads (i dont have any links at hand 
unfortunately).

/Jonas