etc.curl: Formal review begin

[Andrei Alexandrescu]

On 8/30/11 1:21 PM, jdrewsen wrote:
> Den 30-08-2011 19:38, Andrei Alexandrescu skrev:
>> On 8/30/11 12:22 PM, jdrewsen wrote:
>>> Walter suggested that I should write an article about using the wrapper.
>>> I've now taken the first steps on writing such an article. I will have
>>> to get the library API rock stable before I can finish it though.
>>
>> I have a suggestion for you - write and test an asynchronous copy
>> program.
>>
>> It is a continuous source of surprise to me that even seasoned
>> programmers don't realize that this is an inefficient copy routine:
>>
>> while (read(source, buffer))
>> write(target, buffer);
>>
>> If the methods are synchronous and the speeds of source and target are
>> independent, the net transfer rate of the routine is R1*R1/(R1+R2),
>> where R1 and R2 are the transfer rates of the source and destination
>> respectively. In the worst case R1=R2 and the net transfer rate is half
>> that.
>>
> [snip]
>
> I guess that e.g. incoming network buffers in the OS often makes the
> shown copy routine faster than you would think i most cases. These
> buffers stores the incoming network data asynchronously by the OS and
> makes the next read() instantanous. The same can be said about writing
> to disk. Calling sync() is the real enemy here. This is only true as
> long as the buffers are not full of course.

Unless the OS issues speculative reads (which I don't think it does for 
either files or sockets), any time spent in write() is a net loss for 
reading speed. Now, if write is buffered and the buffers are flushed 
asynchronously, calls to write() would be instantaneous. I'm not sure to 
what extent the major OSs do that, and for what types of files.


Andrei