reddit discussion on replacing Python in 0install

Steven Schveighoffer schveiguy at yahoo.com
Wed Jun 12 11:36:02 PDT 2013


On Wed, 12 Jun 2013 12:40:53 -0400, Walter Bright  
<newshound2 at digitalmars.com> wrote:

> On 6/12/2013 9:02 AM, Steven Schveighoffer wrote:
>> On Wed, 12 Jun 2013 11:55:39 -0400, Walter Bright  
>> <newshound2 at digitalmars.com>
>> wrote:
>>
>>> On 6/12/2013 8:18 AM, Steven Schveighoffer wrote:
>>>> No, it does perform well.  You are still not understanding the  
>>>> proposal.
>>>
>>> Yes, I understand your proposal quite well.
>>>
>>> Your benchmark is seriously flawed. Most modern systems cache writes in
>>> memory, and have a delayed write to the media. This hides the problem.
>>
>> Sorry, but this is just bunk.  One uneven flush does not mess up all  
>> future
>> aligned writes.  I/O is not that fragile.
>>
>> If you can demonstrate your theory, I will concede.  Until you do,  
>> don't expect
>> any more responses, it's no use arguing when you don't understand the  
>> problem.
>
> May I present source code?
> -----------------------------
> int fflush(FILE *fp)
> {       int length;
>          int result= 0;
>
>          /* if fflush(NULL) flush all buffers */
>          if (fp == NULL)
>          {
>                  if (flushall() >= 0)
>                          result = 0;
>          }
>          else
>          {
>            /* don't flush buffer if we are not writing   */
>          __fp_lock(fp);
>          if ((fp->_flag & (_IOWRT | _IONBF | _IOERR)) == _IOWRT &&
>              (fp->_base
> #ifdef BIGBUF
>                  || fp->_seg
> #endif
>                          ))
>          {       length = fp->_ptr - fp->_base;  /* # of bytes in buffer  
> */
> #ifdef BIGBUF
>                  if (length &&  
> _writex(fp->_file,fp->_base,length,fp->_seg)
>                          != length)
>                          fp->_flag |= _IOERR;
> #else
>                  if (length)
>                  {   int nwritten = write(fp->_file,fp->_base,length);
>                      /* The following check for isatty() is because:
>                       *  #define WIN32_LEAN_AND_MEAN
>                       *  #include <windows.h>
>                       *  #include <stdio.h>
>                       *  void main()
>                       *  {
>                       *      // Set console output to UTF-8 (one can use  
> 'chcp 65001' instead)
>                       *      SetConsoleOutputCP( 65001 );
>                       *      // Latin small letter e with acute
>                       *      fputs( "Output utf-8 accented char  
> \xc3\xa9\n... and the rest is cut of
>                       *  }
>                       * fails because WriteFile() apparently treats UTF-8
>                       * sequences as 1 byte when writing to the console.
>                       */
>                      if (!nwritten || (nwritten != length &&  
> !isatty(fp->_file)))
>                          fp->_flag |= _IOERR;
>                  }
> #endif
>                  fp->_cnt = 0;
>                  fp->_ptr = fp->_base;
>          }
>          else
>                  fp->_cnt = 0;
>          result = (ferror(fp)) ? EOF : 0;
>          __fp_unlock(fp);
>          }
>          return result;
> }
> -----------------------------------
> This does exactly what I said it does. It's from the Digital Mars C  
> runtime library. Now you could argue that this code sux, and you might  
> even be right. But there are a lot of implementations of C fflush out  
> there - are you sure that all of them will realign after a misaligned  
> write?

If I understand correctly, the buffer is flushed, and ptr is reset to the  
base, the count is reset to 0.  I don't see any code in there that does  
any kind of realignment.  Are you suggesting that other fflush code  
doesn't do this?

> And even if they do realign, you cannot write 10 bytes to a disk. You  
> can only write a block or a sector. Which means the next write, even if  
> aligned, has to write that block or sector again.

That is one block or sector.  The write cache of the OS or the drive will  
probably absorb this hit anyway.

The performance hit is extremely negligible.  Not only that, but the  
hardware can differ from file system to file system.  You are going  
through the file system driver, through the disk driver, through the  
disk.  All of those pieces are written to optimize writes that SPECIFIC  
hardware.  There isn't much you can do to make this perform poorly.

The only performance hit you can really affect is the system call  
penalty.  And that is done by allowing the normal flushing routine to  
continue for subsequent writes.

> You are proposing that this repeated write of the first sector be done  
> for all file output. You probably won't notice a speed difference if  
> write caching is done, but that doesn't make it a good idea.

No, this is incorrect.  fflush is not called on subsequent writeln.

Not only that, but we only have to do this on FILE * that were initialized  
with unknown file descriptors (such as stdin/stdout/stderr).  If we use  
fopen, you don't have to do this.

-Steve


More information about the Digitalmars-d mailing list