iopipe v0.0.4 - RingBuffers!

[Uknown]

On Friday, 11 May 2018 at 13:28:58 UTC, Steven Schveighoffer 
wrote:
> [...]
> I do get the point of having to go outside the cache. I'll look 
> and see if maybe specifying a 1000 line context helps ;)
>
> Update: nope, still pretty much the same.
>

I'm sure someone will find some good show off program.

>> The amount of work done per byte though has to be minimal to 
>> actually see anything.
>
> Right, this is another part of the problem -- if copying is so 
> rare compared to the other operations, then the difference is 
> going to be lost in the noise.
>
> What I have learned here is:
>
> 1. Ring buffers are really cool (I still love how it works) and 
> perform as well as normal buffers
> 2. The use cases are much smaller than I thought
> 3. In most real-world applications, they are a wash, and not 
> worth the OS tricks needed to use it.

Now I need to learn all about ring-buffers. Do you have any good 
starting points?

> 4. iopipe makes testing with a different kind of buffer really 
> easy, which was one of my original goals. So I'm glad that 
> works!

That satisfying feeling when the code works exactly the way you 
wanted it to!

> I'm going to (obviously) leave them there, hoping that someone 
> finds a good use case, but I can say that my extreme excitement 
> at getting it to work was depressed quite a bit when I found it 
> didn't really gain much in terms of performance for the use 
> cases I have been doing.

I'm sure someone will find a place where its useful.


>>> However, this example *does* show the power of iopipe -- it 
>>> handles all flavors of unicode with one template function, is 
>>> quite straightforward (though I want to abstract the line 
>>> tracking code, that stuff is really tricky to get right). Oh, 
>>> and it's roughly 10x faster than grep, and a bunch faster 
>>> than fgrep, at least on my machine ;) I'm tempted to add 
>>> regex processing to see if it still beats grep.
>> 
>> Should be mostly trivial in fact. I mean our first designs for 
>> IOpipe is where I wanted regex to work with it.
>> 
>> Basically - if we started a match, extend window until we get 
>> it or lose it. Then release up to the next point of potential 
>> start.
>
> I'm thinking it's even simpler than that. All matches are dead 
> on a line break (it's how grep normally works), so you simply 
> have to parse the lines and run each one via regex. What I 
> don't know is how much it costs regex to startup and run on an 
> individual line.
>
> One thing I could do to amortize is keep 2N lines in the 
> buffer, and run the regex on a whole context's worth of lines, 
> then dump them all.

iopipe is looking like a great library!

> I don't get why grep is so bad at this, since it is supposedly 
> doing the matching without line boundaries. I was actually 
> quite shocked when iopipe was that much faster -- even when I'm 
> not asking grep to print out line numbers (so it doesn't 
> actually ever really have to keep track of lines).
>
> -Steve

That reminds me of this great blog post detailing grep's 
performance:
http://ridiculousfish.com/blog/posts/old-age-and-treachery.html

Also, one of the original authors of grep wrote about its 
performance optimizations, for anyone interested:
https://lists.freebsd.org/pipermail/freebsd-current/2010-August/019310.html