Exhaustive unit tests

[Chris via Digitalmars-d]

I'm referring to this bug/enhancement request:

https://issues.dlang.org/show_bug.cgi?id=14615

I checked the unit test(s) for it (see below), and found that 
they only check for the positive case, i.e. that a substitution 
takes place, not for the case that no substitution takes place. 
The bug referred to above would have been spotted by testing for 
a mismatch.

Exactly this kind of optimistic unit testing has happened to me 
too. The problem is the "gold standard phenomenon". If you test 
your program against a gold standard, you tend to focus only on 
this standard and think that everything is all right, once you 
meet all the criteria of the standard. However, you forget to 
test for deviating, non-standard behavior. I wonder how much of 
this is still in Phobos. I'm sure someone has opened a ticket / 
PR or something on this, haven't they?

The problem here is that you probably wouldn't have spotted the 
flaw in the unit test by running "-unittest -cov". For 
replaceFirst yes (although not obvious)

        |public R replaceFirst(R, C, RegEx)(R input, RegEx re, 
const(C)[] format)
        |    if(isSomeString!R && is(C : dchar) && 
isRegexFor!(RegEx, R))
        |{
      36|    return replaceFirstWith!((m, sink) => 
replaceFmt(format, m, sink))(input, re);
        |}

        |//  a general skeleton of replaceFirst
        |private R replaceFirstWith(alias output, R, RegEx)(R 
input, RegEx re)
        |    if(isSomeString!R && isRegexFor!(RegEx, R))
        |{
      25|    auto data = matchFirst(input, re);
      25|    if(data.empty)
0000000|        return input;  // <== Not tested for
      25|    auto app = appender!(R)();
      25|    replaceCapturesInto!output(app, input, data);
      25|    return app.data;
        |}

but not for replaceFirstInto

public @trusted void replaceFirstInto(Sink, R, C, RegEx)
        |        (ref Sink sink, R input, RegEx re, const(C)[] 
format)
        |    if(isOutputRange!(Sink, dchar) && isSomeString!R
        |        && is(C : dchar) && isRegexFor!(RegEx, R))
        |    {
      39|    replaceCapturesInto!((m, sink) => replaceFmt(format, 
m, sink))
        |        (sink, input, matchFirst(input, re));
        |    }
        |
        |///ditto
        |public @trusted void replaceFirstInto(alias fun, Sink, R, 
RegEx)
        |    (Sink sink, R input, RegEx re)
        |    if(isOutputRange!(Sink, dchar) && isSomeString!R && 
isRegexFor!(RegEx, R))
        |{
       1|    replaceCapturesInto!fun(sink, input, 
matchFirst(input, re));
        |}

        |// the lowest level - just stuff replacements into the 
sink
        |private @trusted void replaceCapturesInto(alias output, 
Sink, R, T)
        |        (ref Sink sink, R input, T captures)
        |    if(isOutputRange!(Sink, dchar) && isSomeString!R)
        |{
      39|    sink.put(captures.pre);
        |    // a hack to get around bogus errors, should be 
simply output(captures, sink)
        |    // "is a nested function and cannot be accessed from"
        |    static if(isReplaceFunctor!(output, R))
       1|        sink.put(output(captures)); //"mutator" type of 
function
        |    else
      38|        output(captures, sink); //"output" type of 
function
      39|    sink.put(captures.post);
        |}

package.d is 88% covered

Unit test in std/regex/package.d

https://github.com/D-Programming-Language/phobos/blob/master/std/regex/package.d#L1127

See also:
https://github.com/D-Programming-Language/phobos/blob/master/std/regex/package.d#L1266