RTest, a random testing framework

[Fawzi Mohamed]

On 2008-07-22 19:22:19 +0200, BCS <ao at pathlink.com> said:

> Reply to Fawzi,
> 
>> On 2008-07-22 00:12:51 +0200, BCS <ao at pathlink.com> said:
>> 
>>> Reply to Fawzi,
>>> 
>>> [...]
>>> I had an idea a while ago to might make for a better interface and
>>> might even allow better application of constraints:
>>> 
>>> double x,y;
>>> Assert(1/x=pow(x,-1)).
>>> Where(x).NotZero.
>>> TestRandom();
>>> Assert(x<y).
>>> Where(x).LessThan(y).
>>> Where(x).InRange(-10,10).
>>> TestEdges(1000); //< 1000 test points
>>> Assert's arg would be lazy bool and the delegate would be storeed,
>>> the Where would use ref so it can pick up a pointer to x. Some sort
>>> of internal magic would keep track of what constraints apply to what
>>> and let the Test* functions inelegantly search the envelope.
>>> 
>> I like your proposal but I fail to see how it can scale to more
>> complex cases.
>> [...]
> 
> The way the magic above would work is that the Where clauses are not 
> just tested, they are used to define the test envelope. e.g. the clause 
> Where(x).InRange(-10,10) would define a radome number generator that 
> sets x to a radome number in the given range, then the 
> Where(x).LessThan(y) clause would be evaluated and a radome number 
> generated for y that is always in the specified range (in this cases 
> [-inf, x]). Thus in the given cases, ALL test cases would be valid.
> 
> Other 'Where' predicates could be defended that don't drive the 
> generator but are only checked to see if the test cases is valid (like 
> the det()!=0 example you gave) or even only used to determine the 
> expected result so that correct failure cases could be checked as well.
> 
> Other variants of the where clause could be used like:
> 
>  With(arg).InRange(0,8).Eval(matrix.SquareArrray(arg)). // set arg in 
> [0,10] then eval the expression
> 
> The overloading needed to mix ref and lazy parameters with constant 
> arguments would take some work but I think it is doable. There would 
> also need to be quite a bit of magic in the final test function to find 
> test cases within the provided constraints but for many cases this 
> could be made a lot better than guess and check.
> 
> I would love to take a crack at the problem but I have about 4 
> different project in the queue already.

ok I see how this could work, you have one where clause for each 
generator, and that is used to pick up the type and the address of the 
variable and store them is some structure (like a Variant).
Then subsequent messages would set the generator, and maybe constraints.
It could be done, and would be an interesting and challenging project...

Would it be easier to use or simpler to implement than string mixins? I 
don't know, probably not, still an interesting approach.

thanks for the feedback BCS!

Fawzi

> p.s. typo: Assert(1/x=pow(x,-1)).  -> Assert(1/x==pow(x,-1)).

and the second where should be Where(y) :)