Read text file fast, how?

[Johan Holmberg via Digitalmars-d]

On Sat, Jul 25, 2015 at 7:14 PM, Andrei Alexandrescu via Digitalmars-d <
digitalmars-d at puremagic.com> wrote:

> On 7/25/15 8:19 AM, Johan Holmberg via Digitalmars-d wrote:
>
>> Hi!
>>
>> I am trying to port a program I have written earlier to D. My previous
>> versions are in C++ and Python. I was hoping that a D version would be
>> similar in speed to the C++ version, rather than similar to the Python
>> version. But currently it isn't.
>>
>> Part of the problem may be that I haven't learned the idiomatic way to
>> do things in D. One such thing is perhaps: how do I read large text
>> files in an efficient manner in D?
>>
>> Currently I have created a little test-program that does the same job as
>> the UNIX-command "wc -lc", i.e. counting the number of lines and
>> characters in a file. The timings I get in different languages are:
>>
>> D:           15s
>> C++:       1.1s
>> Python:   3.7s
>> Perl:        2.9s
>>
>
> I think this harkens back to the problem discussed here:
>
>
> http://stackoverflow.com/questions/28922323/improving-line-wise-i-o-operations-in-d/29153508
>
> As I discuss there, the performance bug has been fixed for 2.068. With
> your code:
>
> $ time wc -l <(repeat 1000000 echo hello)
>  1000000 /dev/fd/11
> wc -l <(repeat 1000000 echo hello)  0.11s user 2.35s system 54% cpu 4.529
> total
> $ time ./test.d <(repeat 1000000 echo hello)
>  1000000 6000000 /dev/fd/11
> ./test.d <(repeat 1000000 echo hello)  0.73s user 1.76s system 64% cpu
> 3.870 total
>
> The compilation was flag free (no -O -inline -release etc).
>
>
> Andrei
>
>

Thanks, my question seems like a carbon copy of the Stack Overflow article
:) Somehow I had missed it when googling.

I download a dmd 2.068 beta, and re-tried with my input file: now the D
program takes 1.6s (a 10x improvement).

/johan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20150725/e861c44b/attachment.html>