iopipe v0.0.4 - RingBuffers!

Steven Schveighoffer schveiguy at yahoo.com
Thu May 10 23:22:02 UTC 2018


OK, so at dconf I spoke with a few very smart guys about how I can use 
mmap to make a zero-copy buffer. And I implemented this on the plane 
ride home.

However, I am struggling to find a use case for this that showcases why 
you would want to use it. While it does work, and works beautifully, it 
doesn't show any measurable difference vs. the array allocated buffer 
that copies data when it needs to extend.

If anyone has any good use cases for it, I'm open to suggestions. 
Something that is going to potentially increase performance is an 
application that needs to keep the buffer mostly full when extending 
(i.e. something like 75% full or more).

The buffer is selected by using `rbufd` instead of just `bufd`. 
Everything should be a drop-in replacement except for that.

Note: I have ONLY tested on Macos, so if you find bugs in other OSes let 
me know. This is still a Posix-only library for now, but more on that 
later...

As a test for Ring buffers, I implemented a simple "grep-like" search 
program that doesn't use regex, but phobos' canFind to look for lines 
that match. It also prints some lines of context, configurable on the 
command line. The lines of context I thought would show better 
performance with the RingBuffer than the standard buffer since it has to 
keep a bunch of lines in the buffer. But alas, it's roughly the same, 
even with large number of lines for context (like 200).

However, this example *does* show the power of iopipe -- it handles all 
flavors of unicode with one template function, is quite straightforward 
(though I want to abstract the line tracking code, that stuff is really 
tricky to get right). Oh, and it's roughly 10x faster than grep, and a 
bunch faster than fgrep, at least on my machine ;) I'm tempted to add 
regex processing to see if it still beats grep.

Next up (when my bug fix for dmd is merged, see 
https://issues.dlang.org/show_bug.cgi?id=17968) I will be migrating 
iopipe to depend on https://github.com/MartinNowak/io, which should 
unlock Windows support (and I will add RingBuffer Windows support at 
that point).

Enjoy!

https://github.com/schveiguy/iopipe
https://code.dlang.org/packages/iopipe
http://schveiguy.github.io/iopipe/

-Steve


More information about the Digitalmars-d-announce mailing list