howto count lines - fast

[H. S. Teoh via Digitalmars-d-learn]

On Tue, May 30, 2017 at 05:13:46PM -0700, Ali Çehreli via Digitalmars-d-learn wrote:
[...]
> I could not make the D program come close to wc's performance when the
> data was piped from stdin.
[...]

Hmm. This is a particularly interesting case, because I adapted some of
my algorithms to handle reading from stdin (i.e., std.mmfile is not an
option), and I could not get it to reach wc's performance!  I even
installed ldc just to see if that made a difference... it was somewhat
faster than gdc, but still, the timings were about twice as slow as wc.

I did some digging around, and it seems that wc is using glibc's memchr,
which is highly-optimized, whereas std.algorithm.count just uses a
simplistic loop. Which is strange, because I'm pretty sure somebody
optimized std.algorithm some time ago to use memchr() instead of a loop
when searching for a byte value in an array.  Whatever happened to
that??


T

-- 
Sometimes the best solution to morale problems is just to fire all of the unhappy people. -- despair.com