Some Observations on the D Development Process

[Mike Franklin]

This is in response to some of the frustrations offered in the 
thread staring with 
http://forum.dlang.org/post/oeigdutfphxhenexcqsn@forum.dlang.org

I share some of those frustrations, and I've been vocal about 
such things myself.  However, in the last 2 months, at least in 
terms of the DMD repository, things may be starting to change for 
the better.

Recent Promising Progress
-------------------------
A few of us are working hard to revive old PRs and either see 
them across the finish line, or give them a quick path to 
closure. 2 months ago the DMD PR queue stood at about 190.  Now 
it's hovering between 140 and 150.  So, we're getting better; 
take a look at the graphs here for evidence:  
https://auto-tester.puremagic.com/chart.ghtml?projectid=1. Notice 
the spike in PR submissions in December, and we're still 40 PRs 
less than we were 2 months ago.

2 months ago, the oldest PR was dated 2012, believe it or not:  
https://github.com/dlang/dmd/pull/1145.  Our oldest is now 2013, 
which is still pretty bad, but it was partially revived recently 
at https://github.com/dlang/dmd/pull/7503.  Unfortunately, it's 
probably the most difficult PR we have.

I'll take the risk of arguing that we've widened the review 
bottleneck recently for straight-forward PRs.  However, for 
difficult, complex, or controversial PRs, it still looks like we 
lack contributors with the right skills that are willing 
volunteer their time, and tolerate some inconveniences.

There has also been a lot of work done on improving 
infrastructure for testing (e.g. Adding the Appveyor CI for 
testing Windows) and automating some of the mundane tasks (e.g. 
https://github.com/dlang/dmd/pull/7507).

A Few Observations About Our Talent
-----------------------------------
IMO, one of the most worrisome issues we have is a talent 
retention problem (at least for DMD).  It appears many of those 
with the talent to review and fix difficult compiler bugs are no 
longer active, or just pop in from time-to-time.  I'm not sure 
why that is. I know a few have left or don't participate much due 
to dissatisfaction in the way D is managed.  Others, I think, 
were working on D as students and now have blossoming careers 
outside of D to maintain.  I also know of at least one that found 
Rust to be a better alternative.  But, I can also understand how 
someone would just grow sick and tired of working on the same 
thing for months or years.

Walter seems to pop in daily, and occasionally reviews PRs, and 
his PRs of late are mostly just refactorings rather than fixing 
difficult bugs.  We desperately need his help on the more 
difficult issues (e.g. https://github.com/dlang/dmd/pull/7568 and 
https://github.com/dlang/dmd/pull/7566), but I don't know if he's 
well-aware of those items and just busy with other things, or if 
those items are simply not on his radar.  If it's the latter, and 
it would help, I'll publish a digest periodically.

Andrei pops in unpredictably, and sometimes goes on a PR review 
rampage.  The infrequency, however, makes it difficult to 
follow-up and maintain momentum on individual items that need his 
judgement (e.g. https://github.com/dlang/dmd/pull/7099)

That being said, there are some very talented software engineers 
active on this forum that are only seldom active in the PR queue. 
I don't know why that is; perhaps it's simply a matter of time 
and energy, or perhaps it's frustration.  As others have said, 
you don't need to have commit access to be a helpful reviewer. 
And, after showing a willingness to participate, demonstrating 
your abilities, and earning the confidence of the leadership, you 
might be asked to join the team, and then you'll really be able 
to make things happen.

Final Words
-----------
Due to recent changes, if you have a straight-forward, 
non-controversial PR, it will likely be merged in a week or two; 
usually less, and sometimes even within a few hours.  If the PR 
is difficult, complex, or controversial, then it may languish in 
PR purgatory for some time, unfortunately.  I don't know what can 
be done about that at this time without skilled, 
frequently-active talent.

My two-cents,

Mike