Parallel execution of unittests

[Dicebot via Digitalmars-d]

On Tuesday, 6 May 2014 at 15:54:30 UTC, Bruno Medeiros wrote:
> But before we continue the discussion, we are missing am more 
> basic assumption here: Do we want D to have a Unit-Testing 
> facility, or a Testing facility?? In other words, do we want to 
> be able to write automated tests that are Integration tests or 
> just Unit tests? Because if we go with this option of making D 
> unittest blocks run in parallel, we kill the option of them 
> supporting Integration Tests. I don't think this is good.

These days I often find myself leaning towards writing mostly 
integration tests with only limited amount of unit tests. But 
writing good integration test is very different from writing good 
unit test and usually implies quite lot of boilerplate. Truth is 
D does not currently have any higher-level facility at all. It 
has an _awesome_ unit test facility which gets often misused for 
writing sloppy integration tests.

I'd love to keep existing facility as-is and think about 
providing good library augmentation for any sort of higher level 
approach.

Key property of D unit tests is how easy it is to add those 
inline to existing project, unconstrained simplicity. In perfect 
world those are closer to contracts than other types of tests. 
This provides good basic sanity check for all modules you 
recursively import when run via `rdmd -unittest`.

Good integration test is very different. It has certain 
assumptions about initial system state and notifies user if those 
are not met. It can take ages to run and can test real-world 
situations. It is not supposed to be run implicitly and 
frequently. You don't want to keep your integration tests inline 
because of amount of boilerplate code those usually need.

I see no good in trying to unite those very different beasts and 
my experience with existing test libraries has been very 
unpleasant in that regard.