DIP1028 - Rationale for accepting as is

[Walter Bright]

I have made these points before, but I'll summarize them here
for convenient referral.

----

The proposed amendment to Safe by Default is:

"Extern C and C++ functions without bodies should be @system
by default."

The rationale is that since the compiler cannot check them for @safe,
they should be assumed unsafe.

This is Obviously A Good Idea. Why would I oppose it?

1. I've been hittin' the crack pipe again.
2. I was secretly convinced, but wanted to save face.
3. I make decisions based on consultation with my astrologer.
4. I am evil.

I'm going to play the experience card. Many Obviously Good Ideas
appear again and again. Why they aren't so hot only becomes
clear after (sometimes considerable) experience. Some are:

1. Implicit declaration of variables.
2. AST macros (debated in the n.g. multiple times).
3. Exception specifications.

I'll expound on Exception Specifications, as they are more similar
to this case than the others.

Exception Specifications appeared with Java back in the 90's. With it,
each function must list which exceptions it throws, such as (in D):

     T myFunction() throws Alpha, Gamma;

If myFunction() throws any exception that is not derived from Alpha
or Gamma, the compiler issues a compile-time error.

This is Obviously A Good Idea. Smart, experienced, well-meaning programmers
endorsed it wholeheartedly. (Even C++98 adopted the idea.) One such person,
Bruce Eckel (who incidentally authored the early Zortech C++ manual and
is a friend of mine) authored the well-regarded "Thinking In ..." series of
books, one of which was "Thinking in Java". He bought the idea, and preached
and promoted it heavily.

Then came the hangover. Bruce wrote a brilliant mea culpa (that has since
sadly vanished from the internet) about what went wrong. The troubles were:

1. It was tedious to go through every function called by myFunction() and
figure out what all the possible Exceptions thrown might be.

2. If a low level function started throwing a new type, every single caller
all the way up the call tree had to add it to its exception specification.

This, of course, was incredibly tedious and annoying. So what happened?

1. Functions would just do a try-catch(Throwable) and ignore the exceptions.
Oops.

2. The Exception Specifications became simply `throws Throwable`. Oops.

What was most damning was the Bruce Eckel himself was horrified to find that
this was his behavior when writing his own code. Other advocates of it also
admitted they did that too. Exception specifications became a farce, and
Java was quietly softened to not require them. Exception specifications were
just ignored by the C++ community.

How does this relate to safe by default?

Consider the common (because that's how D started out) case of:

----- clibrary.d --------

     T massage_data(... many parameters ...);
     ... 200 more such declarations ...

----- app.d ----------

     import clibrary;

     void useClibrary( ... parameters ...) {
         massage_data(parameters);
     }

---------------------

This code, today, does not use annotations and it works. It's been working
for a long time. Now, we implement the amendment, and it stops compiling
because useClibrary is @safe and massage_data is @system. The user is faced
with the following alternatives:

1. Go through 200 functions in clibrary.d and determine which are @safe
and which are @system. This is what we want them to do. We try to motivate
this with compiler error messages. Unfortunately, this is both tedious and
thoroughly impractical, as our poor user Will Not Know which are safe and
which are system. We can correctly annotate core.stdc.stdio because I know
those functions intimately. This is not true for other system C APIs, and
even less true for some third party C library we're trying to interface to.

2. Annotate useClibrary() as @trusted or @system. While easier, this causes
all benefits to @safe by default to be lost.

3. Wrap the call to massage_data() with:

     () @trusted { massage_data(parameters); } ();

If there are a lot of calls to clibrary, this is going to look pretty awful.
Nobody likes writing or reading such ugly code. It's ok here and there, but
not as a general thing.

4. Edit clibrary.d and make the first line:

     @safe:

I submit that, just like with Java, Option 4 is what people will reach for,
nearly every time. I've had some private conversations where people admitted
this was what they'd do. People who knew it was wrong to do that.

If it's @safe by default, and then someone chooses to annotate it with @system
here and there, I'd feel a lot more confident about the accuracy of the code
annotations than if it just had @safe: at the top. At least they tried.

What is actually accomplished with this amendment if it was implemented?

1. Adds a funky, special case rule. It's better to have simple, easily
understood rules than ones with special cases offering little improvement.

2. Existing, working code breaks.

3. The most likely code fixes are to just make it compile, absolutely
nothing safety-wise is improved. The added annotations will be a fraud.

D should not encourage "greenwashing" practices like the Java
exception specification engendered. The compiler cannot vet the accuracy
of bodyless C functions, and we'll just have to live with that. The proposed
amendment does not fix that.

And so, I did not incorporate the proposed amendment to the Safe by Default
DIP.