iopipe v0.0.4 - RingBuffers!

[Steven Schveighoffer]

On 5/11/18 1:30 AM, Dmitry Olshansky wrote:
> On Thursday, 10 May 2018 at 23:22:02 UTC, Steven Schveighoffer wrote:
>> OK, so at dconf I spoke with a few very smart guys about how I can use 
>> mmap to make a zero-copy buffer. And I implemented this on the plane 
>> ride home.
>>
>> However, I am struggling to find a use case for this that showcases 
>> why you would want to use it. While it does work, and works 
>> beautifully, it doesn't show any measurable difference vs. the array 
>> allocated buffer that copies data when it needs to extend.
> 
> I’d start with something clinicaly synthetic.
> Say your record size is exactly half of buffer + 1 byte. If you were to 
> extend the size of buffer, it would amortize.

Hm.. this wouldn't work, because the idea is to keep some of the buffer 
full. What will happen here is that the buffer will extend to be able to 
accomodate the extra byte, and then you are back to having less of the 
buffer full at once. Iopipe is not afraid to increase the buffer :)

> 
> Basically:
> 16 Mb buffer fixed
> vs
> 16 Mb mmap-ed ring
> 
> Where you read pieces in 8M+1 blocks.Yes, we are aiming to blow the CPU 
> cache there. Otherwise CPU cache is so fast that ocasional copy is 
> zilch, once we hit primary memory it’s not. Adjust sizes for your CPU.

This isn't how it will work. The system looks at the buffer and says 
"oh, I can just read 8MB - 1 byte," which gives you 2 bytes less than 
you need. Then you need the extra 2 bytes, so it will increase the 
buffer to hold at least 2 records.

I do get the point of having to go outside the cache. I'll look and see 
if maybe specifying a 1000 line context helps ;)

Update: nope, still pretty much the same.

> The amount of work done per byte though has to be minimal to actually 
> see anything.

Right, this is another part of the problem -- if copying is so rare 
compared to the other operations, then the difference is going to be 
lost in the noise.

What I have learned here is:

1. Ring buffers are really cool (I still love how it works) and perform 
as well as normal buffers
2. The use cases are much smaller than I thought
3. In most real-world applications, they are a wash, and not worth the 
OS tricks needed to use it.
4. iopipe makes testing with a different kind of buffer really easy, 
which was one of my original goals. So I'm glad that works!

I'm going to (obviously) leave them there, hoping that someone finds a 
good use case, but I can say that my extreme excitement at getting it to 
work was depressed quite a bit when I found it didn't really gain much 
in terms of performance for the use cases I have been doing.

>> in the buffer. But alas, it's roughly the same, even with large number 
>> of lines for context (like 200).
>>
>> However, this example *does* show the power of iopipe -- it handles 
>> all flavors of unicode with one template function, is quite 
>> straightforward (though I want to abstract the line tracking code, 
>> that stuff is really tricky to get right). Oh, and it's roughly 10x 
>> faster than grep, and a bunch faster than fgrep, at least on my 
>> machine ;) I'm tempted to add regex processing to see if it still 
>> beats grep.
> 
> Should be mostly trivial in fact. I mean our first designs for IOpipe is 
> where I wanted regex to work with it.
> 
> Basically - if we started a match, extend window until we get it or lose 
> it. Then release up to the next point of potential start.

I'm thinking it's even simpler than that. All matches are dead on a line 
break (it's how grep normally works), so you simply have to parse the 
lines and run each one via regex. What I don't know is how much it costs 
regex to startup and run on an individual line.

One thing I could do to amortize is keep 2N lines in the buffer, and run 
the regex on a whole context's worth of lines, then dump them all.

I don't get why grep is so bad at this, since it is supposedly doing the 
matching without line boundaries. I was actually quite shocked when 
iopipe was that much faster -- even when I'm not asking grep to print 
out line numbers (so it doesn't actually ever really have to keep track 
of lines).

-Steve