Bugzilla Reward System

[Mike Parker]

On Thursday, 16 September 2021 at 14:35:08 UTC, Paul Backus wrote:

>
> In my experience, the only severity settings most people 
> actually use when filing issues on Bugzilla are "enhancement", 
> "normal", and "regression". And when people do use the other 
> settings, there's no consistency to how they get applied. For 
> example, the first two search results for priority "blocker", 
> issues [22283][] and [22148][], have no indication of what (if 
> anything) they block. Meanwhile, issues [14196][] and [13983][] 
> are both enhancement requests but have their priority set to 
> "major", and issue [22136][] is listed as "critical" even 
> though it is actually a regression!

Yes of course. Rule #1:

> Severity levels are not always accurately set when issues are 
> first reported and may not have been updated since. The 
> reviewer of a pull request that closes a Bugzilla issue will 
> evaluate the issue’s severity level and may change it if he or 
> she determines it is inaccurate. I will moderate any 
> disagreements that may arise about severity levels.

Obviously, this isn't perfect, but it's the best we've got for 
the moment. I expect there will be quite a few kinks to work out 
as time goes by. The initial trial run will surely provide us 
with lessons we can apply going forward, and we will continue to 
refine the process as needed.

> I don't blame anyone who files reports like these. The fact is, 
> there is no official guidance anywhere about what distinguishes 
> a "minor" issue from a "normal" one, or a "normal" issue from a 
> "major" one, so people just guess. But treating the output of 
> this guessing process as though it were meaningful data is 
> probably a mistake.

They aren't meaningful because there has been no organized 
process to make them meaningful. Part of Razvan's job description 
is to oversee the Bugzilla issues, and this initiative is a 
product of that. But he's one man, and the issues are legion :-) 
I predict we'll see the publication of reviewer guidelines down 
the road, and will eventually ask for a handful of people to 
assist Razvan in making sure Bugzilla issues have roughly 
accurate severity levels before a PR is submitted. Those are the 
most obvious steps I see to improving accuracy.