DUnit: Advanced unit testing toolkit.

[Russel Winder]

On Mon, 2013-09-23 at 18:40 +0200, jostly wrote:
[…]
> I think it's great to see the D unit testing ecosystem growing. 
> Since it's still relatively small, I think we have a good chance 
> here to create interoperability between the different frameworks.

There is also integration and system testing of course, not just unit
testing. The same testing framework can generally be used for all forms.

In the Java sphere, JUnit gave way to TestNG exactly because TestNG
supports all forms of testing not just unit testing. Now though TestNG
is giving way to Spock enabling elements of BDD as well as TDD.

D has some unit testing capability built in, which i good, but it is
also good to have an external testing framework that can do unit,
integration and system testing supporting TDD and BDD. 

> As I see it, we have:
> 
> 1. Running unit tests
> 
> This is where D shines with the builting facility for unit tests. 
> However, it suffers a bit from the fact that, if we use assert, 
> it will stop on the first assertion failure, and there is (as far 
> as I've been able to tell) no reliable way to run specific code 
> before or after all the unit tests. If I'm wrong on that 
> assumption, please correct me, that would simplify the spec 
> running for specd.

So the built-in is not entirely up to the task of real unit testing?

> In specd, the actual code inside the unittest { } sections only 
> collect results, and the reporting is called from a main() 
> supplied by compiling with version "specrunner" set. I haven't 
> checked to see if your dunit do something similar.
> 
> 2. Asserting results
> 
> Varies from the builtin assert() to xUnit-like assertEquals() to 
> the more verbose x.must.equal(y) used in specd.

In the Scala variant of the JVM arena, ScalaTest mingles really nicely
all the classic TDD asserts styles, along with Hamcrest matchers, but
also supports the more BDD style test specifications. Spock also does
this. Corollary, D must do this.

NB Go is going through all this just now. The built in unit test
capability is minimalist and for testing the Go implementation. GoCheck
is classic TDD assert style, and there are some candidate BDD styles on
the horizon. Will Go beat D to having the capability that the JVM
languages already enjoy?

Note that Groovy and the py.test Python test framework dispense with the
need for assertEquals and that family of JUnit thingies, in favour of
using the built-in assert and catching the AssertionError exception,
doing detailed stack analysis and provided very detailed information
about the evaluated expression: power asserts. Why should D follow 1990s
thinking when there is 2010s thinking that is much better?

> This could easily be standardized by letting all custom asserts 
> throw an AssertError, though I would prefer to use another 
> exception that encapsulates the expected and actual result, to 
> help with bridging to reporting.

See above :-)

> 3. Reporting results
> 
> If we have moved beyond basic assert() and use some kind of unit 
> test runner, then we have the ability to report a summary of run 
> tests, and which (and how many) failed.
> 
> This is one area where IDE integration would be very nice, and I 
> would very much prefer it if the different unit test frameworks 
> agreed on one standard unit test runner interface, so that the 
> IDE integration problem becomes one of adapting each IDE to one 
> runner interface, instead of adapting each framework to each IDE.
> 
> In my experience from the Java and Scala world, the last point is 
> the biggest. Users expect to be able to run unit tests and see 
> the report in whatever standard way their IDE has. In practice 
> this most often means that various libraries pretend to be JUnit 
> when it comes to running tests, because JUnit is supported by all 
> IDEs.
> 
> Let's not end up in that situation, but rather work out a common 
> API to run unit tests, and the D unit test community can be the 
> envy of every other unit tester. :)

Currently, and very sadly, this generally means writing an XML file
using the JUnit schema. On the other hand if D did this Eclipse, IDEA,
NetBeans, etc. would immediately render excellent data displays.

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder at ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel at winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://lists.puremagic.com/pipermail/digitalmars-d-announce/attachments/20130923/3e3755da/attachment.pgp>