Looking for a Code Review of a Bioinformatics POC

[duck_tape]

On Thursday, 11 June 2020 at 23:45:31 UTC, H. S. Teoh wrote:
>
> Hmm, looks like it's not so much input that's slow, but 
> *output*. In fact, it looks pretty bad, taking almost as much 
> time as overlap() does in total!
>
> This makes me think that writing your own output buffer could 
> be worthwhile.  Here's a quick-n-dirty way of doing that:
>
> 	import std.array : appender;
> 	auto filebuf = appender!string;
> 	...
> 	// Replace every call to writeln with this:
> 	put(filebuf, text(... /* arguments go here */ ..., "\n"));
>
> 	...
>
> 	// At the end of the main loop:
> 	enum bufLimit = 0x1000; // whatever the limit you want
> 	if (filebuf.length > someLimit) {
> 		write(filebuf.data); // flush output data
> 		stdout.flush;
> 		filebuf.clear;
> 	}
>
> This is just a rough sketch for an initial test, of course.  
> For a truly optimized output buffer I'd write a container 
> struct with methods for managing the appending and flushing of 
> output. But this is just to get an idea of whether it actually 
> improves performance before investing more effort into going in 
> this direction.
>
>
> T

I'll play with that a bit tomorrow! I saw a nice implementation 
on eBay's tsvutils that I may need to look closer at.

Someone else suggested that stdout flushes per line by default. I 
dug around the stdlib but could confirm that. I also played 
around with setvbuf but it didn't seem to change anything.

Have you run into that before / know if stdout is flushing every 
newline? I'm not above opening '/dev/stdout' as a file of that 
writes faster.