[Not really OT] Crowdstrike Analysis: It was a NULL pointer from the memory unsafe C++ language.

[Quirin Schroll]

On Saturday, 27 July 2024 at 01:12:09 UTC, Walter Bright wrote:
> You're right that ref's can be null, too. C++ says they can't 
> be null, but it's trivial to make one in C++, so a fat lot of 
> good that does.
>
> Let's say we have a linked list. It will have a `next` pointer:
>
> ```d
> struct List {
>     int payload;
>     List* next;
> }
> ```
>
> A null value is perfectly valid for `next`, as that is how the 
> end is found. At least in my coding, I use null values all the 
> time to signify there is nothing there.

Yes, in your example, by virtue of `List` being a linked list, 
that `null` is a valid value for `next` can be readily guessed.

My suggestion would be that such a pointer would be defined as 
`List*?` to indicate *to the type system* that null is a valid 
value for it. This *forces* any function operating on lists to at 
least consider the possibility of a `null` pointer for `next`.

> It it wasn't null, some other value would have to be there to 
> signify "not a valid item". If there must be something there, 
> then the value would have to be checked against the "not a 
> valid item" value. What should I then do if it unexpectedly is 
> the "not a valid item"? The only sensible thing is to abort the 
> program, as it's a program bug.
>
> But with null, I don't have to check, because the hardware does 
> it for free.

I understand the hardware check. I really do. Most of us do.

TL;DR: Let me rephrase it in terms of implicit conversions: If 
you have a reference-type object (e.g. a pointer) that may be 
null according to the type system (which, currently, is every 
reference type except slices), using it in a way that requires it 
not to be null is an implicit conversion to the non-null version 
of its type. (The fact that it has zero run-time cost is 
irrelevant.) We, the many on this thread, want this implicit 
conversion to be an error. That’s because it’s a wrong implicit 
conversion the same way calling a mutable member function on a 
const object would be a wrong implicit conversion from const to 
mutable. This does not bar anyone from using an explicit cast to 
assert the programmer’s wit over the rules of the type system. 
The other direction, non-null to nullable, is of course valid and 
should not require an explicit cast, same as mutable to const 
does not require an explicit cast.

(End of TL;DR.)

What we want is the type system reminding us to mind the null 
value where it could be a bug to ignore it. To take an analogy 
from the London Underground, you want to mind the gap, right? But 
what if you had to mind the gap on every possible step? Would you 
really do it? Of course you wouldn’t; it’s exhausting and wasted 
attention on almost every step. But sometimes, you’d trip because 
you actually should have minded the gap.

What I’m saying is, if programmers can specify which pointers are 
expected to be null and which are expected not to be null, and 
the type system keeps track that a non-nullable pointer isn’t 
assigned a possibly null value without some explicit cast (which 
may even have zero-runtime cost with the right compiler switches, 
giving you core dumps or – on WebAssembly – UB), we can mind the 
null where it is to be minded, and rest assured that there won’t 
be nulls where we don’t expect them.

> The only time a null pointer dereference is an actual problem 
> is when running on a machine that does not have memory 
> protection, which are decades in obsolescence.

I don’t know myself, but people consistently point out that it’s 
not true. As far as I’m told, we’re here:
- D can target WebAssembly and adds nullable/non-nullable 
annotations.
- D’s null is `@safe`.

Choose one.

> The real issue with null pointer seg faults is the screen dump 
> of mysterious numbers and letters.

What you’re saying is that null dereferences, which are bugs, can 
rather easily debugged. First, that depends on the experience of 
the programmer; I wouldn’t bet my life on being able to 
understand a core dump, let a lone that of a link-time optimized 
program. Second, I’d rather have a compile-error that tells me 
I’m risking a null dereference bug here than having to test and 
hopefully discover the bug before code goes to production. What I 
do with the error depends on circumstance. I have the following 
options, probably more:
- Mark the left-hand side nullable.
- Mark the origin of the right-hand side as non-nullable.
- Handle the null case.
- Insert a `cast(!null)`, risking a bug if I’m wrong.

Only in the “handle the null case” does it incur a run-time cost, 
which I decided was actually needed and had merely forgotten to 
do.

My suggestion is, adding two type suffixes: `T?` for indicating 
that `null` is a valid value and `T!` for indicating that `null` 
is not a valid value. Together with module defaults, that makes 
for a rather seamless transition.

In the current state, the language default is `?`, i.e. every 
reference type is as if suffixed by `?`. A module default of 
`!null` changes that to `!`, so explicit `?` are needed. A module 
default only affects what is lexically in the module, not e.g. 
imported stuff.
```d
// D tomorrow:
module m;
int*  f(); // as if: `int*? f();`
int*? g(); // same type as `f`
int*! h(); // explicitly non-nullable result
```
```d
// D tomorrow:
default(!null)
module m;
int*  f(); // as if: `int*! f();`: explicitly non-nullable result
int*? g(); // explicitly nullable result
int*! h(); // same type as `f`
```
```d
// Possible future D edition where non-null as the language 
default:
module m;
int*  f(); // as if: `int*! f();`
int*? g(); // explicitly nullable result
int*! h(); // same type as `f`
```
```d
// Possible future D edition where non-null as the language 
default:
default(null)
module m;
int*  f(); // as if: `int*? f();` explicitly non-nullable result
int*? g(); // same type as `f()`
int*! h(); // explicitly non-nullable result
```

This is how D could introduce non-nullable reference types 
(pointers, delegates, class handles, …) to the language. Only for 
`ref`, I’d immediately go with `ref?` is allowed to be `null`, 
but ordinary `ref` isn’t because practically, almost all `ref` 
function parameters are expected to be non-null and are bound to 
arguments that are obviously not `*null`.

As pointed out, if D targets WebAssembly, the `cast(!null)` can’t 
be `@safe`. This isn’t even controversial. D can only target the 
WebAssembly that exists with all its flaws.

> The real biggest mistake in C is the eager decay of arrays to 
> pointers, and the tragedy of C is nobody has any interest in 
> fixing it.

I don’t disagree, but it’s unrelated.