etc.curl: Formal review begin

[Marco Leise]

Am 31.08.2011, 00:07 Uhr, schrieb Andrei Alexandrescu  
<SeeWebsiteForEmail at erdani.org>:

> On 8/30/11 3:34 PM, Marco Leise wrote:
>> Am 30.08.2011, 20:48 Uhr, schrieb Andrei Alexandrescu
>> <SeeWebsiteForEmail at erdani.org>:
>>
>>> On 8/30/11 1:10 PM, Jonathan M Davis wrote:
>>>> std.file.copy is synchronous. Would the suggestion then be to change
>>>> it to be
>>>> asynchronous or to create a second function (e.g. copyAsync) which
>>>> does an
>>>> asynchrous copy?
>>>
>>> I think std.file.copy should do whatever the heck is best to copy a
>>> file. As such, it should be transparently asynchronous. It would be
>>> great if such a task caught your fancy - and don't forget to test
>>> speed under a few circumstances.
>>>
>>> Andrei
>>
>> I expect the performance to degrade if you copy asynchronously on a
>> single HDD, since more seeking is involved.
>
> Why would more seeking be involved?
>
> Andrei

Usually files are laid out on the disk in more or less contiguous chunks.  
The naive algorithm would read a large block first and then write it. The  
disk head only has to move twice for every of these blocks. Once it has to  
seek the source sector and once the destination sector. If on the other  
hand you have both operations in parallel then - unless the OS does some  
heavy heuristic optimization - you have the disk head move constantly as  
the operating system interleaves the long series of reads and writes in  
order to serve data to both threads. I don't know the internals of any  
kernel enough to tell you exactly how this is done, but it makes sense  
that the disk can operate faster if it doesn't waste time moving the head  
around. And that is what would happen in a multi-threaded copy algorithm  
on a single HDD.

If you want to learn more about IO schedulers, take a look here:
http://wlug.org.nz/LinuxIoScheduler
and here
http://www.cs.ccu.edu.tw/~lhr89/linux-kernel/Linux%20IO%20Schedulers.pdf

If you want to test your algorithm without an IO scheduler optimizing your  
reads and writes on Linux you can to the following. First use

     cat /sys/block/<your disk here (hda|sda|other...)>/queue/scheduler

to find out which schedulers are supported and which is active. For me it  
prints:

     noop [cfq]

Then, to select 'noop' the "no operation"-scheduler type:

     echo noop > /sys/block/<your disk here>/queue/scheduler

You can use the first command again to verify that the scheduler is set.

- Marco