howto count lines - fast

[Stanislav Blinov via Digitalmars-d-learn]

On Tuesday, 30 May 2017 at 23:41:01 UTC, H. S. Teoh wrote:

> This little challenge piqued my interest.  So I decided to take 
> a shot at seeing if I could beat my system's /usr/bin/wc -l.
>
> First order of business: whenever it comes to performance, 
> always choose the right compiler for the job...


>
> Woohoo!!! Finally, we beat the system's wc -l!! And by a pretty 
> fair margin, too.  Eat your heart out, wc!!!  (The large user 
> time is because we're using all 6 cores at once. But the actual 
> elapsed time is shorter.)


Hm... I cheated a little bit: took your program and compiled with 
`ldc2 -release -O3 -mcpu=skylake`. For data I took std.datetime 
concatenated 1000 times (35446000 lines). Results (minimum of 10 
runs each):

wc -l
real 0.50
user 0.40
sys 0.09

lineCount1
real 0.23
user 0.19
sys 0.04

lineCount2
real 0.29
user 0.17
sys 0.12

lineCount3
real 0.23
user 0.18
sys 0.04

lineCount4
real 0.22
user 1.52
sys 0.04

Seems like all of them beat wc, so machine and compiler matter a 
great deal in this comparison. Thing is, on small files wc is 
going to win pretty much always, due to D's clunky startup. But 
with larger ones - wc falls far behind.
Interestingly, both lineCount1 and lineCount3 on this machine are 
nearly even with lineCount4 :)