Suggestion (ping Walter): Improve unit testing.

[Ary Manzana]

I also very much agree.

If a summary is going to be printed for the unittests (i.e. TESTS: 1 
PASSED: 0  FAILED: 1), it could be great also to label unittests with a 
name. This is more informative that just the file and line where the 
assertion failed. Compare "foo.bar.Exception.AssertException on 
test.d(7)" to "heap sort makes list sorted".

To to this, you can optionaly pass a string to the unittest:

unittest("heap sort makes list sorted") {
    // ...
}

It's backward compatible, and allows future nicer integration with IDEs: 
they could show a list of the tests passed and failed, and names are 
mandatory in this case.

Gregor Richards escribió:
> There are a number of improvements I'd like to see in D's unit testing 
> framework. All of them are standard library changes, and I've made them 
> work in Tango (as an external patch). If they were added to both Phobos 
> and Tango, DSSS could easily run unit tests, which would improve its 
> test suite.
> 
> Here are two major problems I see with unit tests right now:
> 1) You can't run only unit tests, you are forced to run main() as well.
> 2) Unit testing doesn't actually output anything useful, it just works 
> under the no-news-is-good-news principle.
> 
> Each problem has a fairly easy solution in the core library:
> 
> 
> 
> 1) Adding a special option, handled by the core library's main function, 
> which will cause only unit tests to run. The changes I made in Tango:
> Index: lib/compiler/gdc/dgccmain2.d
> ===================================================================
> --- lib/compiler/gdc/dgccmain2.d    (revision 2100)
> +++ lib/compiler/gdc/dgccmain2.d    (working copy)
> @@ -35,7 +35,7 @@
>  extern (C) void _minit();
>  extern (C) void _moduleCtor();
>  extern (C) void _moduleDtor();
> -extern (C) void _moduleUnitTests();
> +extern (C) void _moduleUnitTests(int);
> 
>  /***********************************
>   * These functions must be defined for any D program linked
> @@ -108,6 +108,7 @@
>  {
>      char[][] args;
>      int result;
> +    int testOnly = 0;
> 
>      version (GC_Use_Stack_Guess)
>      {
> @@ -137,11 +138,17 @@
>          }
>          args = am[0 .. argc];
>      }
> +
> +    // check for --d-unittests-only
> +    foreach (arg; args) {
> +        if (arg == "--d-unittests-only")
> +            testOnly = 1;
> +    }
> 
>      void run()
>      {
>          _moduleCtor();
> -        _moduleUnitTests();
> +        _moduleUnitTests(testOnly);
>          result = main_func(args);
>          isHalting = true;
>          _moduleDtor();
> 
> 
> As you can see, if --d-unittests-only is supplied, it passes a 1 into 
> the (now modified) _moduleUnitTests function, which will cause it to 
> exit when finished testing.
> 
> 
> 
> 2) The _moduleUnitTests function can be improved to provide useful 
> output. The output I've implemented for Tango is:
> Failure in test for module 'test':
>   tango.core.Exception.AssertException on test.d(7): Assertion failure
> TESTS: 1  PASSED: 0  FAILED: 1
> 
> The patch to make this work was fairly simple as well:
> Index: lib/compiler/gdc/genobj.d
> ===================================================================
> --- lib/compiler/gdc/genobj.d    (revision 2100)
> +++ lib/compiler/gdc/genobj.d    (working copy)
> @@ -42,7 +42,8 @@
>      import tango.stdc.string; // : memcmp, memcpy;
>      import tango.stdc.stdlib; // : calloc, realloc, free;
>      import util.string;
> -    debug(PRINTF) import tango.stdc.stdio; // : printf;
> +    //debug(PRINTF) import tango.stdc.stdio; // : printf;
> +    extern (C) int printf(char*, ...);
> 
>      extern (C) void onOutOfMemoryError();
>      extern (C) Object _d_newclass(ClassInfo ci);
> @@ -1054,11 +1055,13 @@
>  }
> 
>  /**
> - * Run unit tests.
> + * Run unit tests. If testOnly is true, output results and quit.
>   */
> 
> -extern (C) void _moduleUnitTests()
> +extern (C) void _moduleUnitTests(int testOnly)
>  {
> +    int testCount, testFail;
> +    testCount = testFail = 0;
>      debug(PRINTF) printf("_moduleUnitTests()\n");
>      for (uint i = 0; i < _moduleinfo_array.length; i++)
>      {
> @@ -1070,9 +1073,26 @@
>          debug(PRINTF) printf("\tmodule[%d] = '%.*s'\n", i, m.name);
>          if (m.unitTest)
>          {
> -            (*m.unitTest)();
> +            testCount++;
> +            try {
> +                (*m.unitTest)();
> +            } catch (Exception e) {
> +                printf("Failure in test for module '%.*s':\n", m.name);
> +                printf("  %.*s on %.*s(%d): %.*s\n",
> +                       e.classinfo.name, e.file, e.line, e.msg);
> +                testFail++;
> +            }
>          }
>      }
> +    if (testOnly != 0 || testFail > 0)
> +    {
> +        printf("TESTS: %d  PASSED: %d  FAILED: %d\n",
> +               testCount, testCount - testFail, testFail);
> +        if (testFail == 0)
> +            exit(0);
> +        else
> +            exit(1);
> +    }
>  }
> 
> 
> 
> These additions are fairly minor, and would make unit testing 
> immeasurably better than it is right now. The sections I modified in 
> Tango are mostly the same as the Phobos versions, so the patch should be 
> almost exactly the same as mine above.
> 
> Comments?
> 
>  - Gregor Richards