std.algorithm for images

[Adam D. Ruppe]

I'm pretty happy with how simpledisplay.d is turning out, and want to
spend some time thinking about the client side images now.

I think a nice way to do it would be to go with ranges and templates,
like std.algorithm, so things will work across many different image
formats with the best efficiency and minimal coupling. But, we need to
think about some specifics before going too far.

Here's a first draft module to show the idea while doing some useful
work. It's dependency free so you can download and play with ease.

http://arsdnet.net/dcode/imagedraft.d


If you also grab simpledisplay.d
http://arsdnet.net/dcode/simpledisplay.d

You can compile this little png display program:

=======
import imagedraft;
import std.stdio;

import simpledisplay;

void main(string[] args) {
	auto png = pngFromBytes(File(args[1]).byChunk(4096));
	auto image = new Image(png.header.width, png.header.height);

	int y;
	foreach(line; png.byRgbaScanline) {
		foreach(x, color; line.pixels)
			image.putPixel(cast(int) x, cast(int) y,
				simpledisplay.Color(color.r, color.g, color.b, color.a));
		y++;
	}

	displayImage(image);
}
====

And use it as a base to play around and see your algorithm's results
right in the window. (btw, that displayImage() function is what used
to be image.display() in simpledisplay. A standalone function will
let me separate dependencies better. It works the same way - just
show it and wait for the user to hit any key to continue.)

The foreach there is ugly. I was going to write up a demo
of some actual algorithms for images, but I ended up spending
most the day getting my BufferedInputRange and LazyPngFile to
work correctly...

But, you can see one of the big ideas: providing ranges on images
of various formats that present the data to you in a requested
standard format (any PNG image is lazily converted to an rgba list
of lines here).

It also shows how I plan to write other image format loaders - they
take arbitrary input ranges, and lazily load and decode the file.
(While I had a png loader before, it was in C style.... I'm hoping
that it will be easier to port my bmp code now that I have a working
foundation...)

I'll write an image saver too. It will take an rgbaScanLine range
and save it to the correct format - png, bmp, whatever.


Now, back to the main idea here. I wish I had more time. If I fleshed
out this existing range a little more, we could do fun things
like call std.algorithm.levenshteinDistance on it.. but right now,
it's just an input range, which is fairly limited. I figure the
best thing to do is offer whatever facilities the source range offers,
so if you give it an array, you can save(), back(), etc. too.

Still, we can run a few functions on it.

 foreach(line; filter!"a.pixels[0].r > 200"(png.byRgbaScanline)) {
 }

Only keeps lines with a lot of red in the first pixel.

So that's pretty cool.



Aaaanyway, I want to have a bunch of algorithms written in here that
can work on these ranges, capable of doing both in-place and lazy
changes to your image.

Since I messed up the time, I only wrote one (and it sucks, hard),
but it's the idea I have in mind: the convertToGreyscale function.
Any image that provides a range it can use will work for it.



What kind of ranges would be useful? A by line surely is, which
is why I started with it. A byPixel perhaps? Maybe a byBlock? Though,
it for byPixel, it seems like opApply is the way to do it. Maybe it
can even shove it off to the GPU to do the work. That's beyond my
knowledge, but David Simcha has done some cool stuff with foreach
and parallelism, and this is basically the same concept, so I'm
sure it can be done.

Then, of course, you'll surely still want opIndex into the pixels
too.

I was hoping to have time to write a gaussian blur function to put
the idea to the test...


The other thing is drawing. I don't think ranges will help much there.
Drawing algorithms will probably be ok with opIndexAssign into the
pixels.


But, it's time for me to get to the work I was supposed to do today.
If nothing else, hopefully that PNG loader in there will be useful,
but I really do think there's a lot of potential in the idea of
a std.algorithm for images and would like any input you have.

Thanks for all the feedback this weekend, everyone!