Fantastic exchange from DConf

[Patrick Schluter via Digitalmars-d]

On Tuesday, 9 May 2017 at 14:13:31 UTC, Walter Bright wrote:
> On 5/8/2017 1:55 PM, John Carter wrote:
>> On Saturday, 6 May 2017 at 06:26:29 UTC, Joakim wrote:
>>
>>> Walter: I believe memory safety will kill C.
>>
>> C/C++ has been granted an extension of life by the likes of 
>> valgrind and purify
>> and *-sanitizer.
>
> I agree. But one inevitably runs into problems relying on 
> valgrind and other third party tools:
>
> 1. it isn't part of the language
>
> 2. it may not be available on your platform
>
> 3. somebody has to find it, install it, and integrate it into 
> the dev/test process
>
> 4. it's incredibly slow to run valgrind, so there are powerful 
> tendencies to skip it
>
> valgrind is a stopgap measure, and has saved me much grief over 
> the years, but it is NOT the solution.

And it doesn't catch everything. I had the case yesterday at work 
where one of the file converters that had been written 15 years 
ago, suddenly crashed in production*. It came from an upstream 
bug in a script that filled one attribute with garbage. I tried 
to reproduce the bug in the development environment and funily it 
didn't crash with newest version of the base library. The 
production library is one version behind. The garbage in the 
attribute triggered a buffer overflow in a fixed size array (496 
UTF-16 characters in a buffer of 200 character size). This 
converter is one of the last one with fixed sized arrays. The 
interesting observation was that valgrind catches the buffer 
overflow when linked with version 2.31 of the main library but is 
completely silent when using version 2.32. The changes in that 
library are minimal and in parts that have nothing to do with 
this app. It is solely the placement of variables in data iand 
the bss that change. It is surprizing to see such a big buffer 
overflow completely missed by valgrind.

TL;DR valgrind does not always catch buffer overflows especially 
if the memory overwritten is not in the head but in the data or 
the bss segment. There it cannot add guard pages as it does on 
the heap.

* To give a little context. I work at the European Commission on 
the central translation memory system called Euramis (probably 
the biggest in the world with more than a billion segments). The 
system is used intensively by all translators of all European 
institutions and without it, nothing would be possible. The issue 
with it is that the back end is written in C and the code goes 
back to 1990. Me and my colleagues managed to modernize the 
system and catch most of the code issues with intensive use of 
C99 idioms, newest gcc and clang diagnostics and also valgrind 
and such things.