Everyone who writes safety critical software should read this

[Adam Wilson]

On Wed, 30 Oct 2013 11:12:48 -0700, Walter Bright  
<newshound2 at digitalmars.com> wrote:

> On 10/30/2013 3:01 AM, Chris wrote:
>> On Wednesday, 30 October 2013 at 03:24:54 UTC, Walter Bright wrote:
>>> Take a look at the reddit thread on this:
>>>
>>> http://www.reddit.com/r/programming/comments/1pgyaa/toyotas_killer_firmware_bad_design_and_its/
>>>
>>>
>>> Do a search for "failsafe". Sigh.
>>
>> One of the comments under the original article you posted says
>>
>> "Poorly designed firmware caused unintended operation, lack of driver  
>> training
>> made it fatal."
>>
>> So it's the driver's fault, who couldn't possibly know what was going  
>> on in that
>> car-gone-mad? To put the blame on the driver is cynicism of the worst  
>> kind.
>
> Much effort in cockpit design goes into trying to figure out what the  
> pilot would do "intuitively" and ensuring that that is the right thing  
> to do.
>
> Of course, we try to do that with programming language design, too, with  
> varying degrees of success.
>
>> Unfortunately, that's a common (and dangerous) attitude I've come  
>> across among
>> programmers and engineers. The user has to adapt to anything they fail  
>> to
>> implement or didn't think of. However, machines have to adapt to humans  
>> not the
>> other way around (realizing this was part of Apple's success in UI  
>> design,
>> Ubuntu is very good now too).
>>
>> I warmly recommend the book "Architect or Bee":
>>
>> http://www.amazon.com/Architect-Bee-Human-Technology-Relationship/dp/0896081311/ref=sr_1_1?ie=UTF8&qid=1383127030&sr=8-1&keywords=architect+or+bee
>>
>

Having experience with a 737 flight deck and Cessna 172/G1000 flight deck.  
I can personally say that if even one of the devs on both of those (very  
different) flight information systems had a clue about HCI he was  
physically beaten for bringing it up. Yes, the absolute fundamentals might  
be intuitive (AI, DG, etc,). But if you need anything advanced ... God  
Help You. I did eventually figure it out (and started helping the  
instructors at my FBO), but intuitive is NOT the word I would use...

There is also a story floating around about the boys (I'll not deign to  
call the programmers...) at Honeywell FINALLY called in a group of pilots  
for HCI analysis/critique of the 787 flight management systems months  
after they had shipped the code to the FAA for certification...

And lastly, although it got buried because France needs to protect EADS,  
there was a "By Design" bug that caused the Angle of Attack indicator to  
NOT show when AF447 was in deep stall, overridden by the faulty airspeed  
indication, never mind that this is the ONLY indicator a pilot needs to  
recover from a stall... If the pilots had seen this when the plane went  
into it's unusual attitude, the pilots could've seen it and corrected  
immediately. Sorry Airbus, but the computer does NOT always know best,  
it's only as good as the [non-pilot] programmers feeding it code... :-)

-- 
Adam Wilson
IRC: LightBender
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/