read() performance - Linux.too?

[Derek Parnell]

On Mon, 24 Jul 2006 04:55:17 +0200, Bob W wrote:

> /*
> The std.file.read() function in dmd causes a performance
> issue after reading large files from 100MB upwards.
> Reading the file seems to be no problem, but cleanup
> afterwards takes forever.

Its a GC effect. The GC is scanning through the buffer looking for
addresses to clean up.

A simple delete of the buffer will prevent the GC from trying so hard.

 // Try reading a 100MB+ file with the following
 // program (some patience required):
 
 import std.stdio, std.file;
 
 alias writefln wrl;
 
 void main(char[][] av) {
   wrl();
   if (av.length<2)  {
     wrl("Need file name to test read() !");
     return;
   }
   char[] fn=av[1];
   wrl("Reading '%s' ...", fn);
   char[] bf=cast(char[])read(fn);
   wrl("%d bytes read.",bf.length);
   wrl("Doing something ...");
   int n=0;
   foreach(c;bf)  n+=c;
   wrl("Result: %s, done.",n);

   delete bf;

   wrl("No delay here now after reading a huge file ...");
   wrl();
 }

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
"Down with mediocrity!"
24/07/2006 2:11:34 PM