[dmd-internals] regressions, criticals, and blockers

[Jonathan M Davis]

On Friday, April 27, 2012 21:38:06 Brad Roberts wrote:
> On 4/27/2012 7:17 PM, Jonathan M Davis wrote:
> > On Friday, April 27, 2012 17:38:49 Walter Bright wrote:
> >> The severity level is set by the submitter. I think that we can either
> >> spend time arguing over what severity level it should be, or we can
> >> spend the time fixing them.
> > 
> > True, but there's something to be said for either fixing all blockers or
> > changing them to critical with any given release, since it can't really be
> > a blocker if it doesn't block the release.
> > 
> > Regardless, I thought that I should point out the fact that blockers
> > really
> > aren't treated that way at all, making that severity level a bit pointless
> > as it stands.
> > 
> > - Jonathan M Davis
> 
> There's at least two definitions for blocker here:
> 
>  1) should block the release
>  2) blocks the submitters project -- ie, no work around known
> 
> Bug submitters are almost certainly treating the severity as the latter.

True, but that's a pretty useless definition for blocker, since it says nothing 
about the severity of the bug as far as the community at large is concerned. 
It just means that it's causing major problems for the person reporting it. 
The actual  bug could be quite minor.

> For DMD and related releases, we haven't even achieved the presumably easier
> goal of no regressions.  We've gotten better at no new regressions.  Until
> we hit that goal, I'm not overly stressed about hitting the goal of zero
> critical bugs and zero blocker bugs.  Making progress on all three of those
> is a very good plan, but I very much want to see a zero regression release
> policy.  We're so close to it already.
> 
> For what it's worth, 4 of the 10 regressions are tagged with the pull
> keyword.  0 of 11 blockers have pull requests. That assumes that the pull
> keyword is accurate, which isn't a great assumption.

I don't really expect us to be postponing releases due to blockers or critical 
bugs at this point for the reasons that you state, but I do think that the 
term blocker is a bit odd when it doesn't actually block the release.

I just thought that I should bring it up, since as it stands, the fact that 
something is marked as blocker is pretty meaningless beyond the fact that it's 
_probably_ one of the nastier bugs. But we don't really have the manpower, 
control, or organization required to organize severity levels very accurately 
(i.e. having very exact definitions for them and making sure all bugs are 
marked at the appropriate level according to those definitions).

- Jonathan M Davis