Faster Command Line Tools in D

[Steven Schveighoffer via Digitalmars-d-announce]

On 5/26/17 10:41 AM, John Colvin wrote:
> On Wednesday, 24 May 2017 at 13:39:57 UTC, Mike Parker wrote:
>> Some of you may remember Jon Degenhardt's talk from one of the Silicon
>> Valley D meetups, where he described the performance improvements he
>> saw when he rewrote some of eBay's command line tools in D. He has now
>> put the effort into crafting a blog post on the same topic, where he
>> takes D version of a command-line tool written in Python and
>> incrementally improves its performance.
>>
>> The blog:
>> https://dlang.org/blog/2017/05/24/faster-command-line-tools-in-d/
>>
>> Reddit:
>> https://www.reddit.com/r/programming/comments/6d25mg/faster_command_line_tools_in_d/
>>
>
> I spent some time fiddling with my own manual approaches to making this
> as fast, wasn't satisfied and so decided to try using Steven's iopipe
> (https://github.com/schveiguy/iopipe) instead. Results were excellent.
>
> https://gist.github.com/John-Colvin/980b11f2b7a7e23faf8dfb44bd9f1242

nice! hm....

/** something vaguely like this should be in iopipe, users shouldn't 
need to write it */
auto ref runWithEncoding(alias process, FileT, Args...)(FileT file, auto 
ref Args args)

stealing for iopipe, thanks :) I'll need to dedicate another slide to you...

>
> On my machine:
> python takes a little over 20s, pypy wobbles around 3.5s, v1 from the
> blog takes about 3.9s, v4b took 1.45s, a version of my own that is
> hideous* manages 0.78s at best, the above version with iopipe hits below
> 0.67s most runs.
>
> Not bad for a process that most people would call "IO-bound" (code for
> "I don't want to have to write fast code & it's all the disk's fault").
>
> Obviously this version is a bit more code than is ideal, iopipe is
> currently quite "barebones", but I don't see why with some clever
> abstractions and wrappers it couldn't be the default thing that one does
> even for small scripts.

The idea behind iopipe is to give you the building blocks to create 
exactly the pipeline you need, without a lot of effort. Once you have 
those blocks, then you make higher level functions out of it. Like you 
have above :)

BTW, there is a byLineRange function that handles slicing off the 
newline character inside iopipe.textpipe.

-Steve