iopipe v0.1.0 - now with Windows support!

[DigitalDesigns]

On Tuesday, 12 June 2018 at 13:51:56 UTC, Steven Schveighoffer 
wrote:
> On 6/12/18 1:51 AM, DigitalDesigns wrote:
>> 
>> Could you explain some benefits specific to this 
>> implementation and a bit of the functional aspects for a 
>> proper overview of it's capabilities and why I should chose 
>> this method over others?
>
> The things that I think make this approach better are:
>
> 1. Direct buffer access
>
> Direct buffer access I have found is one of those ideas that 
> doesn't seem like it's that impressive until you start using 
> it. Many times, buffering is used in a generic library for 
> optimization (basically amortizing reads) and is an 
> implementation detail hidden from your view. Think of how FILE 
> * keeps a large buffer of data inside itself, but only gives 
> you access to one character at a time.
>
> This forces you to create your *own* buffering scheme on top of 
> that. What a waste! Iopipe allows you to use buffering for your 
> purposes on top of the benefits of amortization. It's my belief 
> that this is why iopipe's byline feature is 2x faster than 
> Phobos'.
>
> 2. Using templates to their fullest
>
> Iopipes are all templated on the buffer or iopipe underneath 
> it. This makes tings easily swappable. It's really cool to be 
> able to take your JSON or XML parser, and hook it onto an 
> in-memory string in one line, and then hook it onto a socket, 
> and everything is optimized for that situation. It takes the 
> fun and flexibility of range programming and brings it to i/o.
>
> This is why iopipe's byline handles all forms of UTF, compared 
> to Phobos which only handles UTF8.
>
> For example, I handle all forms of UTF with iopipe, with a 
> decent set of utilities. Here is a complete program using 
> iopipe that converts any type of UTF into another type, 
> optimized for the specific situation:
>
> https://github.com/schveiguy/iopipe/blob/master/examples/convert/convert.d
>
> 3. Compiler optimization for everything
>
> All parts of iopipe, except for the low-level reads and writes 
> (which ironically are not really part of iopipe) are visible to 
> the compiler for inlining and optimization. I'm leveraging the 
> power of the decades of optimization experience that the 
> compiler can provide. This makes it easy to write code that 
> performs well.
>
> An anecdote: For my talk on iopipe in 2017 
> (http://dconf.org/2017/talks/schveighoffer.html) I wanted to 
> have a live demo showing the performance power. I literally was 
> still working on it 2 or 3 days before, while at dconf. I had 
> already written a JSON parser, which was part of my 
> presentation, but when I was showing it to another D user 
> (Daniel Murphy), I couldn't really answer the question "how 
> does it perform?". So he gave me a challenge -- do pretty 
> printing on a JSON file. Simple enough, with the parser I had 
> already written, took me about 1 hour to write it. It performed 
> poorly compared to what we would have expected, but tweaking a 
> few things (almost all were due to using some algorithms 
> incorrectly), I got it to go faster than RapidJson in certain 
> use cases, and reasonably close in others. And I did nothing in 
> terms of lookup tables or using any special instructions. All 
> in all, it was probably 2 hours of work, and the code is 
> beautiful IMO! 
> https://github.com/schveiguy/jsoniopipe/blob/master/examples/formatjson/formatjson.d
>
> I think anyone who is doing parsing should have a look at 
> iopipe, it not only may make your code much simpler and easier 
> to read and write, but it would help me tune iopipe to cater to 
> parsers, which I think is its wheelhouse.
>
> I plan to eventually finish the JSON parser for a releasable 
> state, and eventually tackle XML and a few other things.
>
> -Steve

Thanks!