Generality creep

[Andrei Alexandrescu]

On 3/31/19 8:01 PM, Mike Franklin wrote:
> On Sunday, 31 March 2019 at 16:18:35 UTC, Rubn wrote:
> 
>> Look at this comment as someone pointed out before:
>>
>> https://github.com/dlang/dmd/pull/8557#pullrequestreview-149952733
> 
> Yes, Andrei's review was a bit too strong but he did change course later 
> in that thread.  Also recognize that the original author did not provide 
> much motivation or justification (though, I'm guilty of that too).

Those were unnecessarily strong words.

Allow me to give a bit more context whilst clarifying I am not defending 
or condoning that attitude.

That comment came after a few phone conversations whereby Walter 
mentioned he has a deluge of pull requests to review. He said if he's to 
spend due time on all of those, he'd be unable to do any work of his 
own. Those pull requests that have these things in common (note that 
some may not apply to the pragma pull request, I'd just wrongly put it 
in the same bin):

* they are large
* the improvement they bring needs arguing
* their quality could be improved so they need careful review and a 
couple of passes of changes

This is what I call Good Work.

Good Work is often sizable in quantity, and is visibly the result of 
concerted effort by a competent person. Those unfamiliar with a codebase 
and with the subtleties of the task at hand cannot produce Good Work.

(Contrast that with Bad Work: by definition, it is easy to inspect and 
reject. It does not create a major slowdown in a project.)

Good Work is more often than not complex. Complexity is misrecognized as 
evidence of the complexity of the problem; the task was hard, the 
reasoning goes, because the solution is difficult in proportion.

Good Work begets more Good Work. Typically Good Work produces context, 
opportunity, and precedent for more of the same. The same reviewer who 
rubber stamped a piece of Good Work will have an idea how to produce 
more Good Work derived from it. The kind of environment where Good Work 
is revered encourages its creation, in a cycle that creates the illusion 
of progress. Because Good Work is complex, it produces "bug ripples" 
whereby increasingly complex Good Work fixes one bug but is liable to 
introduce others.

Good Work is difficult to argue against. The main argument in favor, 
which is very difficult to counter, is that Good Work is better than Bad 
Work and better than No Work. It is easy to create an opinion trend in 
favor of this or that Good Work.

However, there are problems with Good Work. The first one is scale: an 
accumulation of Good Work does not add up to Great Work. More often, the 
increasing entropy leads to the thermal death of the project.

An accumulation of Good Work is what leads to six nested calls in an 
advanced library to convert a string to an integer. Even if they are 
inlined away in execution, their smell persists.

An accumulation of Good Work is what leads to a multitude of "is this 
kinda sorta almost like-a string hey I really mean it this time quite 
like a string" tests, or "we must to add @trusted every five lines of code".

The other problem of Good Work is that it takes the air out of the room, 
causing Great Work to suffocate. Good Work takes great effort to author, 
debate, review, debug, and maintain. All that takes away time and mental 
share that should go into Great Work instead.

Which brings us to Great Work.

Great Work solves difficult problems with a paucity of means; it is 
quintessentially and surprisingly /not/ proportional response. Because 
of that, it is often deceptively simple, but always in a way that is 
impossible to anticipate yet obvious in afterthought. Scala's implicits 
and D's static if are great work.

Great Work often reformulates an entire challenge to reveal false 
choices or artificial constraints. Alexander the Great figured it 
doesn't matter how to get rid of the Gordian knot, so he cut it. While 
philosophers were still puzzling over The Ship of Theseus, Richard 
Feynman pointed out that the very notion of identity is ill-defined 
because a philosopher wouldn't even be able to tell which atoms belong 
to a chair, and which don't. Replacing complex type-based 
metaprogramming with trivial compile-time evaluation is great work.

Great Work is often recognizable to out-of-domain people, the person on 
the street. In contrast, Good Work has high-brow expertise as a 
prerequisite. Beethoven's Fourth Symphony is a respectable repertoire 
piece for any Philharmonic, one that music aficionados would listen to 
with respect. I once saw a janitor, a little old lady with no previous 
exposure to classical music, crying when she heard Bach's Air for the 
first time. Using the same programming language on the client and 
server, or at compile-time and run-time, or for computation and its 
constraints, is great work clear as rain. You don't need to be an expert 
to appreciate that.

Great Work is what we should all aspire to. Great Work is the cure for 
Good Work.

This, however, brings up a question: it seems that Great Work is not 
really easy to do on a regular basis. What to do, therefore, on a 
"regular" day when inspiration doesn't strike?

This brings us to Right Work.

Right Work is work that is undeniably, pound-on-the-table good, however 
unexciting or trivial.

Right Work will be silently appreciated by one's peers as the 
incontestably right thing to do. Paying your rent, debts, and other 
bills is Right Work.

Right Work is not necessarily simple. Becoming a better spouse or 
teaching your kids that lying is not a good policy - that's also Right Work.

In a software project, reducing the number of global variables is Right 
Work.

Making public data private is Right Work. (Careful, perversions are 
always possible. This is not Right work:

public int percent;

==>

private int _percent;
int percent() { return _percent; }
void percent(int p) { _percent = p; }

The obscure word "phronesis" - thanks Laeeth - should be a buzzword in 
software engineering circles.)

Reducing state and mutation is Right Work. That means adding the 
"immutable" qualifier to variables and the "pure" attribute to functions.

Making unsafe code safe is Right Work. That means inserting "@safe" 
wherever possible, and identifying the smallest "@trusted" primitives.

Replacing legacy pointer-and-length ad-hoc pairs with slices is Right 
Work. So is replacing a complicated unstructured loop with a structured 
foreach loop.

Most refactorings that meaningfully reduce the number of lines of code 
are Right Work. Of course, that could be perverted, too. I'm not talking 
more statements per line of code. I'm talking more /work/ per line of code.

Much Right work paradoxically requires less virtuosity than complicated 
artifacts of Good Work.

A successful software system is a construction of Great Work on a 
foundation of Right Work, with the inevitable Good Work here and there.

That's where we want to be.