std.serialization: pre-voting review / discussion

[David Nadlinger]

On Sunday, 18 August 2013 at 14:52:04 UTC, Andrej Mitrovic wrote:
> Normally the project maintainers would do this
> themselves, but it's easy to run out of time or just to forget 
> to test
> things, and then it's too late (well we have fixup DMD releases 
> now so it's not too bad).

The big problem with this right now is that quite frequently, you 
run the tests and discover one regression in the beta, file it, 
fix it (or wait for it to get fixed), then run the tests again, 
discover that they still don't pass, etc.

This is not only an annoying and time-intensive job for the 
maintainer of the project (as during beta you have to pretty much 
always be on your toes for a new version to test lest Walter 
decide to make the final release), but this also increases beta 
duration.

One obvious reaction to this (as a project maintainer) would be 
to continuously track Git master and report regressions as they 
arise. However, this is also not always practical, as quite 
often, there is a regression/backwards-incompatible change early 
on in the development process that is not fixed until much later, 
so that multiple issues can still pile up unnoticed.

Having a system that regularly, automatically runs the test 
suites of several larger, well-known D projects with the results 
being readily available to the DMD/druntime/Phobos teams would 
certainly help. But it's also not ideal, since if a project 
starts to fail, the exact nature of the issue (regression in DMD 
or bug in the project, and if the former, a minimal test case) 
can often be hard to track down for somebody not already familiar 
with the code base.

David