Feasible Idea?: Range Tester

[Jonathan M Davis]

On Thursday, March 21, 2013 13:08:58 Nick Sabalausky wrote:
> Suppose you create a range: struct MyRange{...}
> 
> Now you have to unittest it. And there's a lot of ways a range
> implementation can go wrong. There's a fair amount of things to test.
> And they all, ideally, need to be done for *every* range type created.
> 
> But, ranges are all supposed to conform to a standard interface with
> standardized semantics. So can't most of this testing be generalized
> into a generic InputRangeTester, RandomAccessRageTester, etc? All you'd
> do is feed it an array of elements to test with, and, in theory, it
> should be able to unittest the range's semantics for conformance.
> 
> Obviously it couldn't guarantee 100% conformance, because no unittest
> can do that, but it seems reasonable that it could be just as good as a
> well-designed set of hand-written tests.
> 
> I think this would be an enormously useful thing to have in Phobos. Can
> any Phobos/range gurus say fur sure how feasible this is? Does Phobos
> perhaps already do something similar internally that could just be
> generalized?
> 
> (This message has been brought to you by three random access ranges I
> just created but really, really don't feel like unittesting ;) )

Hmmm. I've been working on stuff which makes creating ranges to test range-
based functions easier, but I've never thought about creating something to 
test conformance. If I need something like that I just use static asserts to 
test isInputRange, isForwardRange, etc. and make sure that each of the ones 
that are supposed to pass for that range do and that those that aren't 
supposed to pass don't. All that that's going to test though is the API, not 
behavior. But I don't know how you'd automate testing behavior when the exact 
behavior of the range is very much dependent on the range itself. In most 
cases though, you end up with a new range type, because it's being returned 
from a range-based function, in which case, simply unit testing the function 
with a variety of range types generally takes care of it, and those aren't 
particularly automatable, because what the tests need to test depends entirely 
on what the function is supposed to do.

- Jonathan M Davis