Read text file fast, how?

[Johan Holmberg via Digitalmars-d]

Hi!

I am trying to port a program I have written earlier to D. My previous
versions are in C++ and Python. I was hoping that a D version would be
similar in speed to the C++ version, rather than similar to the Python
version. But currently it isn't.

Part of the problem may be that I haven't learned the idiomatic way to do
things in D. One such thing is perhaps: how do I read large text files in
an efficient manner in D?

Currently I have created a little test-program that does the same job as
the UNIX-command "wc -lc", i.e. counting the number of lines and characters
in a file. The timings I get in different languages are:

D:           15s
C++:       1.1s
Python:   3.7s
Perl:        2.9s

The central loop in my D program looks like:

        foreach (line; f.byLine) {

            nlines += 1;

            nchars += line.length + 1;

        }

I have also tried another variant with this inner loop:

        char[] line;

        while(f.readln(line)) {

            nlines += 1;

            nchars += line.length;

        }

but in both cases this D program is much slower than any of the others in
C++/Python/Perl. I don't understand what can cause this dramatic difference
to C++, and a factor 4 to Python. My D programs are built with DMD 2.067.1
on MacOS Yosemite, using the flags "-O -release".

Is there something I can do to make the program run faster, and still be
"idiomatic D"?

(I append the whole program for reference)

Regards,
/Johan Holmberg

=======================================
import std.stdio;
import std.file;

void main(string[] argv) {
    foreach (fname; argv[1..$]) {
        auto f = File(fname);
        int nlines = 0;
        int nchars = 0;
        foreach (line; f.byLine) {
            nlines += 1;
            nchars += line.length + 1;
        }
        writeln(nlines, "\t", nchars, "\t", fname);
    }
}
=======================================
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20150725/26dba3c4/attachment.html>