Discussion Thread: DIP 1028--Make @safe the Default--Final Review

[Jonathan M Davis]

On Saturday, April 4, 2020 12:53:57 AM MDT Walter Bright via Digitalmars-d 
wrote:
> On 4/3/2020 2:06 PM, Steven Schveighoffer wrote:
> > I want to make sure you understand that we are not talking about
> > extern(C) functions that are written in D.
>
> I understand totally what you are talking about.
>
> > But what should absolutely not compile is:
> >
> > extern(C) int free(void *);
> >
> > void foo(int *ptr) // now inferred @safe
> > {
> >
> >     free(ptr);
> >
> > }
>
> I understand your proposal. You want C functions without bodies to be
> @system.

Anything else would go against what @safe is supposed to be promising. @safe
means that the compiler verified the function for memory safety, which it
obviously didn't do in the case of extern(C) declarations. You're needlessly
putting a hole in the @safety system as part of making @safe the default. We
already have enough problems with @trusted being used incorrectly without
the compiler blindly applying it to declarations by assuming that they're
@safe with no verification whatsoever.

> > The fact that we cannot control where/how people define their prototypes
> > means we have to be firm on this. They need to opt-in to @safe with
> > extern(C), it cannot be default.
>
> On the other hand, special cases like this tend to cause unexpected
> problems in the future. Experience pretty much guarantees it. It's likely
> to be tricky to implement as well.
>
> People remember simple rules. They don't remember rules with odd
> exceptions to them, that always winds up with trouble and bug reports.
> Simple rules applied evenly lead to a compiler that works and is
> reliable. I'm afraid the weight of all the special rules will crush D.

I would have thought that a rule that function declarations which are not
extern(D) would be @system by default would be very clear and simple. But if
it's not, would it be more straightforward to just make all function
declarations be treated as @system by default? Or would it be simpler to
just say that anything that isn't extern(D) is @system by default? Or would
it be simpler to just outright require that all non-extern(D) functions be
explicitly marked by the programmer instead of assuming anything?

There has got to be a straightforward rule here that doesn't involve the
compiler assuming that code is @safe when it hasn't verified it, and the
programmer hasn't told the compiler that it's @trusted. If this DIP is
accepted as-is, it will no longer be the case that you can find all memory
safety problems problems by searching for @trusted code. All non-extern(D)
functions will be a potential problem and will have to be checked for
whether they've been explicitly marked with any attributes by the programmer
or were simply assumed to be @safe by the compiler, whereas right now, they
have to be marked with @trusted in order to treated as @safe, just like any
function body where the compiler can't verify that it's memory safe.

> Now, as to what to do. I spent a few minutes and added `@system:` in to
> the C headers in druntime for windows, posix, and linux. Done. I hope
> someone else will do it for freebsd, etc., if not I'll go do that to.

If the programmer has to explicitly mark extern(C) declarations as @system
just so that they're not @safe, then you're basically turning @trusted on
its head. Instead of the compiler guaranteeing that something is memory safe
and only treating something that it can't guarantee is memory safe as memory
safe if the programmer marks it as @trusted, the compiler is then assuming
that these functions are @safe, because it can't verify that they're not,
forcing the programming to mark them with @system when they're not. I really
don't understand how that can be viewed as acceptable given than @safe is
supposed to be about compiler-verified memory safety.

And remember that not all extern(C) declarations are in druntime. Plenty of
programmers interface with other C libraries. So, going through druntime and
making sure that all of the extern(C) declarations in it are marked
appropriately doesn't solve the problem. It just makes sure that those
particular extern(C) declarations aren't being incorrectly treated as @safe
by the compiler.

It very much sounds like you're putting a hole in @safe, because you think
that it's simpler to have a hole than it is to have the compiler tell the
programmer that they need to actually verify extern(C) declarations for
memory safety. This situation reminds me of how you tried to make it so that
array bounds checking went away even in @safe code. If you remove the
checks, you remove the guarantees of memory safety.

>  > is going to cause huge issues.
>
> I doubt that for the simple reason that @system by default has not caused
> huge issues.
>
> The rule is simple:
>
> "For a D module with a bunch of C declarations in it, start it with
> `@system:`."
>
> It's not a hard rule to check. It's one line. D's C interface has always
> relied on the user to get right. Recall that C doesn't do any name
> mangling at all:
>
>    ----- mylib.di
>      extern (C) int addOne(int i);
>
>    ----- mylib.c
>      double addOne(double d) { return d + 1; }
>
>
> That'll link fine and yet fail at runtime. Oops! Calling it @system will
> not help at all. If the C implementation is bad, there's not a damn thing
> D can do about it, with or without @system. It's always going to be this
> way.

The key difference is that right now, the programmer has to actually tell
the compiler that a particular C declaration is @trusted for it to be
treated as @safe. If they fail to take the time to mark an extern(C)
declaration approriately or they miss it for whatever reason, the code using
it will have to deal with the fact that it wasn't marked properly. There
will be errors when @safe code tries to use it, and the programmers working
on that code will know that they have to make sure that what they're doing
can be @trusted. With this DIP, as it currently stands, those mistakes will
then be invisible, and it's much less likely that they will be caught. It
goes completely against the idea that the compiler only treats code as @safe
if it can actually verify it that is or if the programmer has told the
compiler that it is.

Of course, the programmer can use @trusted incorrectly and screw themselves
over, but then at least when the program has problems due to memory safety
bugs, you only have to track down the @trusted code to see what code you
have to examine for memory safety bugs. None of it is invisible, and the
compiler is not claiming that anything is @safe when it's not.

Personally, I would much rather see @safe not be the default than to have
holes like this put in it. Yes, as things stand, too much code is left as
@system, and too often, programmers don't take the time to make sure that
their code is @safe where it can be. But at least the compiler isn't
claiming that something is @safe when it hasn't actually verified that it
is. And I really don't see why it would be a big problem to either treat all
non-extern(D) function declarations as @system by default (or even all
function declarations as @system by default if that's simpler) or to just
simply require that the programmer explicitly mark them. It would avoid
introducing holes into @safe while still allowing us to have @safe be the
default, and it seems really simple to explain and easy to understand.
Instead, with this DIP, we'll have to be explaining to people why they have
to be extremely wary of any extern(C) declarations in their code leading to
@system code being treated as @safe without the compiler saying anything
about it.

- Jonathan M Davis