Bad array indexing is considered deadly

[John Colvin via Digitalmars-d]

On Thursday, 1 June 2017 at 10:26:24 UTC, Steven Schveighoffer 
wrote:
> On 5/31/17 9:05 PM, Walter Bright wrote:
>> On 5/31/2017 6:04 AM, Steven Schveighoffer wrote:
>>> Technically this is a programming error, and a bug. But 
>>> memory hasn't
>>> actually been corrupted.
>>
>> Since you don't know where the bad index came from, such a 
>> conclusion
>> cannot be drawn.
>
> You could say that about any error. You could say that about 
> malformed unicode strings, malformed JSON data, file not found. 
> In this mindset, everything should be an Error, and nothing 
> should be recoverable.
>
>>
>>
>>> This seems like a large penalty for "almost" corrupting 
>>> memory. No
>>> other web framework I've used crashes the entire web server 
>>> for such a
>>> simple programming error.
>>
>> Hence the endless vectors for malware insertion in those other 
>> frameworks.
>
> No, those are due to the implementation of the interpreter. If 
> the interpreter is implemented in @safe D, then you don't have 
> those problems.
>
>>> Compare this to, let's say, a malformed unicode string 
>>> (exception),
>> malformed JSON data (exception), file not found (exception), 
>> etc.
>>
>> That's because those are input and environmental errors, not 
>> programming
>> bugs.
>
> Not necessarily. A file name could be sourced from the program, 
> but have a typo. An index could come from the environment. The 
> library can't know, but makes assumptions one way or the other. 
> Just like we assume you want to use the GC, these assumptions 
> are harmful for those who need it to be the other way.
>
>> There can be grey areas in classifying problems as input 
>> errors or
>> programming bugs, and those will need some careful thought by 
>> the
>> programmer as to which bin they fall into, and then code 
>> accordingly.
>>
>> Array overflows are not a grey area, however. They are always
>> programming bugs.
>
> Of course, programming bugs cause all kinds of Errors and 
> Exceptions alike. Environmental bugs can cause Array overflows.

I think the idea is that no, array overflows can never be caused 
by the environment in a correct program. If you don't adequately 
screen the environmental input, your program is incorrect.


This is how I think about it:

There are 3 categories of bugs: known safe to survive, known 
unsafe to survive, unknown safety.

Range Errors are an example of errors that can be considered 
"unknown safety", so by default we assume it is unsafe to 
continue.

If you - as the human developer - decide that the specific 
RangeError bug from this place in the code is actually known safe 
to survive, you should add screening for the bad value and throw 
an Exception instead, or if that's difficult to do then catch the 
Error and then throw an Exception*. Note that these aren't fixes 
for the bug, these are explicit recognition of the continued 
existence of the bug while promising (@trusted style) that 
everything will still be OK.

If you decide it is truly an "unsafe to continue" bug, then let 
it carry on crashing there.

Ultimately of course you screen the environment at the 
appropriate level or fix the bug, do the "right thing" whatever 
that may be.

*note that you could abstract this away into an array type that 
throws Exceptions, but where would you know it was safe to use? 
Perhaps not so many places.

Tldr; if you know that a bug is safe to continue/recover from, 
put in the necessary code to do so.


I would be interested to see ideas of how to implement some sort 
of logical sandboxing in D. Perhaps if one calls a strongly pure 
@safe function, there is no way it can mess up shared state, so 
you know that as long as you disregard the result it will always 
be safe to continue... Effectively it's a "process within a 
process" or something like that. Of course you'd need to be able 
to guarantee you can catch Errors, plus even though the function 
you've called can't have *caused* the problem, it might be the 
only place where you *find* the problem and that might be bad 
enough to not want to continue from...