Faster Command Line Tools in D

[John Colvin via Digitalmars-d-announce]

On Wednesday, 24 May 2017 at 13:39:57 UTC, Mike Parker wrote:
> Some of you may remember Jon Degenhardt's talk from one of the 
> Silicon Valley D meetups, where he described the performance 
> improvements he saw when he rewrote some of eBay's command line 
> tools in D. He has now put the effort into crafting a blog post 
> on the same topic, where he takes D version of a command-line 
> tool written in Python and incrementally improves its 
> performance.
>
> The blog:
> https://dlang.org/blog/2017/05/24/faster-command-line-tools-in-d/
>
> Reddit:
> https://www.reddit.com/r/programming/comments/6d25mg/faster_command_line_tools_in_d/

I spent some time fiddling with my own manual approaches to 
making this as fast, wasn't satisfied and so decided to try using 
Steven's iopipe (https://github.com/schveiguy/iopipe) instead. 
Results were excellent.

https://gist.github.com/John-Colvin/980b11f2b7a7e23faf8dfb44bd9f1242

On my machine:
python takes a little over 20s, pypy wobbles around 3.5s, v1 from 
the blog takes about 3.9s, v4b took 1.45s, a version of my own 
that is hideous* manages 0.78s at best, the above version with 
iopipe hits below 0.67s most runs.

Not bad for a process that most people would call "IO-bound" 
(code for "I don't want to have to write fast code & it's all the 
disk's fault").

Obviously this version is a bit more code than is ideal, iopipe 
is currently quite "barebones", but I don't see why with some 
clever abstractions and wrappers it couldn't be the default thing 
that one does even for small scripts.

*using byChunk and manually managing linesplits over chunks, very 
nasty.