The DIP Process

[Joseph Rushton Wakeling]

On Tuesday, 26 February 2019 at 17:46:32 UTC, Jonathan Marler 
wrote:
> I think most points of the DIP process make sense, but it has 
> one very large problem.  Feedback from language maintainers 
> (Walter and Andrei) is not done until the end of the process. 
> You're asking someone to go through a process that can take a 
> year before the people who have the power to accept or reject 
> the proposal look at it or leave any feedback.  This is 
> extremely wasteful of the author's time, the reviewer's time 
> and causes extreme pressure for everyone involved.  A years 
> worth of waiting, debate and revision that are now wasted and 
> could have easily been avoided if language maintainers left 
> their feedback early on.  Walter and Andrei should be involved 
> in the process throughout, not just render judgement at the end.

It's worth making a comparison to how new features are 
incorporated into other languages, such as C++, or Rust, or 
Fortran, or Haskell.  In every case, it's hard -- years of work 
trying out different designs, different implementation attempts, 
balancing costs against benefits.  D is if anything far more 
flexible than many of its counterparts when it comes to accepting 
new ideas.

Obviously it's not great if something goes through multiple 
rounds of feedback and revision only to be rejected at the very 
end, but sometimes it's only _because_ of those multiple rounds 
of feedback and revision that a proposal is clear enough to take 
that kind of decision on it.  Early feedback doesn't help with 
that.

On the other hand, one can flip the problem on its head: perhaps 
the problem is less that Walter and Andrei can come in at the end 
with a rejection, than that things they are going to reject get 
put in front of them in the first place.  That suggests that 
either the review process may be ineffective at weeding out 
problematic proposals, or that DIP authors may be too strongly 
biased towards revising and resubmitting their proposals rather 
than withdrawing them when problems are identified.

> In the years I've been here I have found that feedback from 
> anyone other than Walter and Andrei has very little bearing on 
> what Walter and Andrei will think.  The entire community can 
> think an idea is great, but then Walter or Andrei completely 
> reject it.  And the opposite is true as well, I've seen W&A 
> champion an idea that the community generally rejects. 
> Designing a process to ask many people to create and perfect a 
> proposal for a year catered to 2 specific people without any 
> feedback from them is mind-boggling to me.

I'm going to say something comically rude here, but with the 
serious intent of provoking everyone to reflect a bit.

Which is: have you considered the possibility that Walter and 
Andrei are the only people who really know what they're talking 
about, and everyone else is a [expletive] idiot? :-)

Clearly whenever a tiny number of people wield absolute veto 
power, there is a risk of a certain amount of arbitrariness or 
preference-of-the-moment-based decision making (or, perhaps more 
problematically, the perception of such, even when it isn't the 
case).  But there are very few people in the D community with a 
breadth and depth of experience to match Walter and Andrei either 
as individuals or together, so it's hardly surprising that they 
might regularly perceive the advantages and disadvantages of a 
course of action differently from the majority.

After all, where subtle technical decisions are concerned, it's 
highly likely that a small number of domain experts are going to 
be able to make a better decision than the majority of users.

I'd also question how one establishes that "the entire community 
can think an idea is great".  We make a big mistake if we assume 
that the opinion of the self-selecting group of people who post 
on forums, or are active on GitHub, is representative of the 
wider D community of often silent users.

In fact, that's the kind of thing it's worth asking both Walter 
and Andrei: what kinds of demonstrable consensus (and of whom?) 
do you think might cause you to change your minds about something 
you would otherwise have vetoed?

> Based on my observations, my guess is that the DIP process was 
> designed to alleviate the amount of work needed from Walter and 
> Andrei, but look what it's produced.

Returning to the point earlier -- perhaps what it has produced is 
something that is inadequate at reducing the amount of work they 
have to do.

It's surely nice if we can find ways to reduce the amount of time 
and effort DIP authors need to put in before getting a clear 
decision on their work.  But part of the problem here is seeing 
that time and effort as wasted if a DIP is rejected.

If instead we were to focus on seeing a DIP as a learning 
experience through which everyone (authors, reviewers, and 
language maintainers) get a better understanding of what things 
are possible, what things are practical or useful, and why, then 
a rejected DIP isn't an adversarial or problematic outcome, or a 
waste of anyone's time: it's a collaborative effort that 
established the lack of viability of a particular course of 
action.  Just as in science or medicine, clearly identifying 
_negative_ results is not a bad thing -- it's a valuable (and too 
often undervalued) contribution!