Read text file fast, how?

[Johan Holmberg via Digitalmars-d]

On Sat, Jul 25, 2015 at 10:12 PM, Andrei Alexandrescu via Digitalmars-d <
digitalmars-d at puremagic.com> wrote:

> On 7/25/15 1:53 PM, Johan Holmberg via Digitalmars-d wrote:
>
>> Thanks, my question seems like a carbon copy of the Stack Overflow
>> article :) Somehow I had missed it when googling.
>>
>> I download a dmd 2.068 beta, and re-tried with my input file: now the D
>> program takes 1.6s (a 10x improvement).
>>
>
> Great, though it still seems to be behind the C++ version, which is a
> bummer. -- Andrei
>
>
My C++ program was actually doing C-style IO via <stdio.h>. I didn't think
about the distinction C/C++ when reporting the earlier numbers.

If I switch to full C++ style: <fstream> + <string> + C++ version of
getline(), then the C++-solution is even slower than Python: 5.2s. I think
it is the C++ libraries of Clang on MacOS Yosemite that are slow.

This prompted me to re-run the tests on a Linux machine (Ubuntu 14.04),
still with the same input file, a text file with 7M lines and total size of
466MB:

C++ with <stdio.h> style IO:    0.40s
C++ with <fstream> style IO:   0.31s
D 2.067                                    1.75s
D 2.068 beta 2:                        0.69s
Perl:                                         1.49s
Python:                                    1.86s

So on Ubuntu, the C++ <fstream> version was clearly best. And the
improvement in DMD 2.068 beta "only" a factor of 2.5 from 2.067.

/johan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20150726/bc71d769/attachment.html>