Lints, Condate and bugs

[bearophile]

Sorry for the answering delay.

Don:

> With the bugs I've fixed in the DMD source, I've seen very many cases of
> 7, several cases of 2 and 6, and only one case of 8.
> Many bugs are also caused by dangerous casts (where a pointer is cast
> from one type to another).
> But almost everything else been caused by a logic error.
> 
> I am certain that there are still many null pointer bugs in DMD.

Thank you for your numbers. We may think about ways to reduce similar bugs in D code.

------------------

Walter Bright:

> None of the null pointer bugs dmd has had would have been prevented by using
> non-nullable types. I.e. they were not "I forgot to initialize this pointer",
> but were instead the result of logic errors, like running off the end of a list.

Non-null references/pointers are meant to protect against more than just forgotten initalializations.

There are ways to help against some of those that you call "logic errors".


> NULL in dmd represents "this datum has not been computed yet" or "this datum has
> an invalid value" or "this datum does not exist". With non-nullable types,
> they'd have to be set to a datum that asserts whenever it is accessed, leading
> to the same behavior.

If not-null pointers are supported, then probably a language has to ask you to manage the null case explicitly (in D this is less easy to do).

---------------------------

Walter:

> It still causes the same slowdown. If the CPU can speculatively execute 5
> instructions ahead, then you're reducing it to 4.

I don't think the situation is as linear as you say :-)

Anyway, I have experimentally seen both in few years of Delphi usage and some experiments in C# that the situation is not as bad as you think. In practice the slow down they cause is comparable or less than the slowdown caused by array bound tests.


> If you were right I'd be reading all the time about integer overflow bugs,
> not buffer overflow bugs.

Good C compilers (and lints) warn against mixing signed-unsigned values :-) So maybe the programmers are able to avoid those bugs. But I agree that buffer overflow bugs are more common in C code.


> Languages which have these have failed to gain traction. That might be for other
> reasons, but it's not ausipcious.

:-)


> Delphi has also failed in the marketplace.

Delphi (and TurboPascal) has being used for many years, in the whole world, probably it will eventually die because people like C-family languages more, but so far it has had 100 times the success of D. Delphi devs also have designed C#, that too has overflow bounds (but not bounded values). Also, people use Ada in high safety software, and people use Ada still for such purposes.


> Or perhaps because people really aren't having a problem with integer overflows.

I don't know.


> Such a switch is completely impractical, because such a language would then have
> two quite incompatible variants.

I agree, that's why I prefer integer overflows.


> Also, *you* care about performance,

I care more for correct programs and for a language that helps me spot bugs very quickly.

---------------------------

dennis luehring:

> and there are million developers out there who likes/and use null-able
> values for flow-control - if the null-able "feature" is removed without
> something that keeps the style working, you will loose them, or much
> more evil, they will try to code around the non-null-able-style getting
> back to there well known null-able behavior, by using bools, ints,
> strings whatever -> that will not help in library growth around D
> 
> try comming up with an pattern that keeps both pro/cons...

Now it's too much late to introduce not-nullables on default, so this isn't a problem.

---------------------------

Denis Koroskin:

> No one is talking about removing nullable references but rather adding  
> non-nullable types and making them default. You could still achieve old  
> behavior if it is needed (most proposed proposed syntax):
> 
> Foo? foo = stuff.find(predicate);
> if (foo is null) {
>      // not found
> }

See my answer to dennis luehring. Now we can hope to do the opposite, to introduce some syntax to denote what pointers/references can't be null.

---------------------------

Now references/pointers in D are nullable on default. But a light syntax may be added to denote pointers that can't be null.

I have suggested to add a trailing @ (or +) to denote not-nullable reference or pointer:


class T {}
T nullable_reference;
T@ nonnullable_reference = new T@();

struct S {}
S nullable_pointer;
S@ nonnullable_pointer = new S@();


class T {}
T nullable_reference;
T+ nonnullable_reference = new T+();

struct S {}
S nullable_pointer;
S+ nonnullable_pointer = new S+();


With OOP there is a problem in using this kind of references, see this for more info, this is an unfinished proposal:

http://d.puremagic.com/issues/show_bug.cgi?id=4571

I think this first part may be implemented well and it will avoid some null bugs. The 


Walter:
> Having the program abort due to an assert failure rather than a segment
> violation is not the great advance it's sold as.

The compiler enforces the not-null nature at compile-time, so some run-time errors may be avoided.

You can't give a nullable pointer to a function with this annotation, so there is no risk of forgetting to test for null and it can't cause a null reference error:

void foo(SomeClass@ c) {


In other situations when the not-nullable reference is created it may generate an assert error, but this is better than a segment violation later because the bug is spotted when the reference is built, and not when it's used, so the line number of the assert error is closer to the true location of the bug. This speeds up debugging.

But you are right that not-null references/pointers can't catch all "logic errors", because in many/some situations you need nullable references/pointers. So there's a optional second half of the proposal.

To have a more null-safe language you need to ask for tests every time a nullable pointer is about to be dereferenced, unless you already are inside a branch of a "if" where the reference/pointer is statically known to be non-nullable (or below an assert(some_pointer)). In presence of gotos and exceptions this becomes harder to do, so the compiler may need to act conservatively and ask for a test (or that assert(some_pointer) where it can't infer.

Bye,
bearophile