Everyone who writes safety critical software should read this

[Chris]

On Thursday, 31 October 2013 at 06:32:41 UTC, Adam Wilson wrote:
>
> Having experience with a 737 flight deck and Cessna 172/G1000 
> flight deck. I can personally say that if even one of the devs 
> on both of those (very different) flight information systems 
> had a clue about HCI he was physically beaten for bringing it 
> up. Yes, the absolute fundamentals might be intuitive (AI, DG, 
> etc,). But if you need anything advanced ... God Help You. I 
> did eventually figure it out (and started helping the 
> instructors at my FBO), but intuitive is NOT the word I would 
> use...
>
> There is also a story floating around about the boys (I'll not 
> deign to call the programmers...) at Honeywell FINALLY called 
> in a group of pilots for HCI analysis/critique of the 787 
> flight management systems months after they had shipped the 
> code to the FAA for certification...
>
> And lastly, although it got buried because France needs to 
> protect EADS, there was a "By Design" bug that caused the Angle 
> of Attack indicator to NOT show when AF447 was in deep stall, 
> overridden by the faulty airspeed indication, never mind that 
> this is the ONLY indicator a pilot needs to recover from a 
> stall... If the pilots had seen this when the plane went into 
> it's unusual attitude, the pilots could've seen it and 
> corrected immediately.

> Sorry Airbus, but the computer does NOT always know best, it's 
> only as good as the [non-pilot] programmers feeding it code... 
> :-)

I'm still waiting for the day when people will realize this! I 
always hear users say "Ah, it's been calculated by a computer! It 
must be correct.", assuming that machines are perfect. I always 
ask the question "But who builds and programs machines?" Humans, 
of course. And we are not perfect, far from it.

Another story I've heard is that the German revenue had a clever 
program that could find out, if a shop or pub owner was cheating. 
The program assumed that if  a certain threshold of round numbers 
(in the colloquial sense of the word) in his/her balances was 
exceeded the business owner was cheating. Now, there was one pub 
owner who only had prices with round numbers (and I know others 
too), simply because he couldn't be bothered to deal with stupid 
prices like €4.27 and to always have the right change. This is 
not uncommon.

The programmers of the revenue had based their stats on all 
retailers of the country, including supermarkets, department 
stores etc.