6-weeks release cycle

[Rikki Cattermole via Digitalmars-d]

On 5/06/2015 8:28 p.m., Jonathan M Davis wrote:
> On Friday, 5 June 2015 at 06:42:06 UTC, Rikki Cattermole wrote:
>> And shouldn't the CI be doing regression testing already?
>
> The autotester runs the unit tests that are in druntime, Phobos, and
> dmd. It catches a lot of stuff and generally prevents us from merging
> bad code. But it can't possibly catch everything. It catches the stuff
> that we thought to specifically test for and bugs that were fixed (since
> unit tests are usually added for bug fixes). But it's not that
> infrequent for something _new_ to break that's never broken before and
> is subtle enough that it doesn't get past all of the tests - especially
> when you're dealing with changes to the compiler.
>
> To catch all of that stuff before it goes out the door in a release, we
> need to test a _much_ larger code base than just the standard stuff -
> which is part of why we have betas. We want folks to try out their
> projects with the betas so that we can catch the stuff that we missed
> before it gets released in an official release. Simply grabbing an
> arbitrary commit and declaring it a release just because it's at about
> the time that we want to do a release would be a disaster. Too much gets
> through as it is simply because not enough folks test the betas and
> report what they find. _All_ of that would get through if we just picked
> a random commit and declared it to be a release.
>
> _Maybe_ someday our test suites will catch such a large portion of the
> regressions that we won't actually end up with any regressions getting
> out, but I doubt it. Even large, heavily used projects like gcc or KDE
> end up with regressions getting out, much as they try to avoid it. But
> like them, we need to do our best to have releases which have been
> tested well enough via betas and whatnot rather than just releasing
> stuff simply because it's a certain date.
>
> - Jonathan M Davis

I wonder if we could integrate a bit more with travis and have a big 
list of projects compiling + testing against HEAD.
Would be an excellent indicator.

Maybe even have this as part of dub repo?