etc.curl: Formal review begin

[Andrew Wiley]

On Wed, Aug 31, 2011 at 11:39 AM, Marco Leise <Marco.Leise at gmx.de> wrote:

> Am 31.08.2011, 00:07 Uhr, schrieb Andrei Alexandrescu <
> SeeWebsiteForEmail at erdani.org**>:
>
>
>  On 8/30/11 3:34 PM, Marco Leise wrote:
>>
>>> Am 30.08.2011, 20:48 Uhr, schrieb Andrei Alexandrescu
>>> <SeeWebsiteForEmail at erdani.org**>:
>>>
>>>  On 8/30/11 1:10 PM, Jonathan M Davis wrote:
>>>>
>>>>> std.file.copy is synchronous. Would the suggestion then be to change
>>>>> it to be
>>>>> asynchronous or to create a second function (e.g. copyAsync) which
>>>>> does an
>>>>> asynchrous copy?
>>>>>
>>>>
>>>> I think std.file.copy should do whatever the heck is best to copy a
>>>> file. As such, it should be transparently asynchronous. It would be
>>>> great if such a task caught your fancy - and don't forget to test
>>>> speed under a few circumstances.
>>>>
>>>> Andrei
>>>>
>>>
>>> I expect the performance to degrade if you copy asynchronously on a
>>> single HDD, since more seeking is involved.
>>>
>>
>> Why would more seeking be involved?
>>
>> Andrei
>>
>
> Usually files are laid out on the disk in more or less contiguous chunks.
> The naive algorithm would read a large block first and then write it. The
> disk head only has to move twice for every of these blocks. Once it has to
> seek the source sector and once the destination sector. If on the other hand
> you have both operations in parallel then - unless the OS does some heavy
> heuristic optimization - you have the disk head move constantly as the
> operating system interleaves the long series of reads and writes in order to
> serve data to both threads. I don't know the internals of any kernel enough
> to tell you exactly how this is done, but it makes sense that the disk can
> operate faster if it doesn't waste time moving the head around. And that is
> what would happen in a multi-threaded copy algorithm on a single HDD.
>
> If you want to learn more about IO schedulers, take a look here:
> http://wlug.org.nz/**LinuxIoScheduler<http://wlug.org.nz/LinuxIoScheduler>
> and here
> http://www.cs.ccu.edu.tw/~**lhr89/linux-kernel/Linux%20IO%**
> 20Schedulers.pdf<http://www.cs.ccu.edu.tw/~lhr89/linux-kernel/Linux%20IO%20Schedulers.pdf>
>
> If you want to test your algorithm without an IO scheduler optimizing your
> reads and writes on Linux you can to the following. First use
>
>    cat /sys/block/<your disk here (hda|sda|other...)>/queue/**scheduler
>
> to find out which schedulers are supported and which is active. For me it
> prints:
>
>    noop [cfq]
>
> Then, to select 'noop' the "no operation"-scheduler type:
>
>    echo noop > /sys/block/<your disk here>/queue/scheduler
>
> You can use the first command again to verify that the scheduler is set.
>
>
Yes, but the disk should be able to do out-of-order execution of read and
write requests (which SCSI was designed to allow), which should make this
never be an issue. My understanding is that most SCSI implementations (and
it's in just about every storage protocol out there) allow this, with the
exception of Bulk Only Transport, which is used with USB hard drives and
flash drives. That will soon be replaced by USB Attached SCSI (
http://en.wikipedia.org/wiki/USB_Attached_SCSI), which specifically allows
out of order execution.
The end idea behind much of SCSI is that the disk designers are much better
at knowing what's fast and what's not than the OS developers, and if we need
to worry about it on the application level, something has seriously gone
wrong.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20110831/4608a9ed/attachment.html>