dub test

[Andrei Alexandrescu via Digitalmars-d]

On 08/24/2016 09:25 AM, Seb wrote:
> On Wednesday, 24 August 2016 at 12:00:32 UTC, Andrei Alexandrescu wrote:
>> On 08/24/2016 04:14 AM, Edwin van Leeuwen wrote:
>>> I might be dense, but the only other thing than integration tests that I
>>> can think of is if you use random data for testing, but that would be
>>> more correctly solved by using more random data during the unittests.
>>> Nothing is worse than tests that only sometimes fail.
>>
>> Randomized unit testing is a respected approach to testing:
>> https://hackage.haskell.org/package/QuickCheck,
>> https://github.com/rickynils/scalacheck,
>> https://blogs.msdn.microsoft.com/dsyme/2008/08/08/fscheck-0-2/, etc.
>> Some of the Phobos tests I wrote use it (and virtually all of my
>> current code e.g. on median computation), and when they fail I just
>> print the seed of the RNG and then hardcode it to reproduce the test.
>> -- Andrei
>
> Do you remember that there is a pending PR for this at Phobos that is
> blocked because of missing feedback / acceptance?
>
> https://github.com/dlang/phobos/pull/2995

I know. What I need to write:

unittest
{
     void theFunctionToTest(Gen!(int, 1, 5) a, Gen!(float, 0.0, 10.0) b)
     {
         // This will always be true
         assert(a >= 1 && a <= 5);
         assert(a >= 0.0 && a <= 10.0);

         // super expensive operation
         auto rslt = (a + b);
         doNotOptimizeAway(rslt);

         debug
         {
             assert(rslt > 1.0);
         }
     }

     benchmark!theFunctionToTest();
}

What I want to write:

@Benchmark(ParamDomain!1(1, 5), ParamDomain!2(0, 10), Aggregate.median)
void theFunctionToTest(int a, float b)
{
     // This will always be true
     assert(a >= 1 && a <= 5);
     assert(a >= 0.0 && a <= 10.0);

     // super expensive operation
     auto rslt = (a + b);
     doNotOptimizeAway(rslt);
     debug
     {
         assert(rslt > 1.0);
     }
}

Attributes are a very attractive (if not the best) way to set up 
benchmarks. There are a variety of Java benchmarking frameworks we could 
draw inspiration from, see list at 
http://stackoverflow.com/questions/7146207/what-is-the-best-macro-benchmarking-tool-framework-to-measure-a-single-threade. 
Take a look e.g. at 
https://github.com/google/caliper/blob/master/examples/src/main/java/examples/BitSetBenchmark.java, 
which is so clear and simple.


Andrei