Counter Rollover Brings Down Rail Service

In October 2018 Hong Kong had "six hours of turmoil" in their rail service due to as signalling outage. The culprit has now been identified as counter roll-over.

https://www.scmp.com/news/hong-kong/transport/article/2178723/unknown-signalling-system-incompatibility-caused-october

South China Morning Post
https://www.scmp.com/news/hong-kong/transport/article/2178723/unknown-signalling-system-incompatibility-caused-october

Summary version: a system synchronization counter had been counting away since 1996 and required a system reset when it saturated.  (At least it didn't just roll over without anything noticing.)  But over the years two different systems with slightly different counter roll-over procedures were installed.  When rollover time came, they disagreed with each other on count value, paralyzing the system during the window until the second system shut down due to counter saturation.  Details below quoted from the official report. (https://www.mtr.com.hk/archive/corporate/en/press_release/PR-18-108-E.pdf)

The Detailed version:

"5.1.3. Data transmission between sector computers is always synchronized through an internal software counter in each sector computer. If any individual sector computer is individually rebooted, its counter will be re-initialized and will immediately synchronize to the higher counter figure for the whole synchronized network. Therefore, when the Siemens sector computers were commissioned and put into service in 2001/2002, the relevant counters were synchronized to those of the Alstom sector computers which were installed in 1996. If the counter reaches its ceiling figure, the associated sector computer will halt and need to be re-initialized. However the counter re-initialization arrangements for the two suppliers’ sector computers are different. The Alstom sector computers will be re-initialized automatically once their counters reach an inbuilt re-initialization triggering point approximately 5 hours before reaching the ceiling figure. However, this internal software function was not made known to the operators and maintainers. The Siemens sector computers do not have an automatic reinitialization function and therefore need to be manually reinitialized through rebooting in SER by maintenance staff.  

5.1.4 At around 05:26 hours on the incident day, the Alstom software counters reached the triggering point for automatic re- initialization while the Siemens sector computers continued counting up, creating an inconsistent re-initialization situation between the two interconnected sector computers at KWT (Alstom) and LAT (Siemens). This resulted in repeated execution of re-initialization followed by re-synchronization with the higher counter figure from LAT, in the KWT sector computer in an endless loop causing corresponding instability in all 25 Alstom sector computers in the system.  

5.1.5 When all the Siemens software counters reached the ceiling figure at around 10:22 hours, some 5 hours after the Alstom sector computers had passed their automatic re-initialization triggering point, the 8 Siemens sector computers halted as designed. Moreover, trains on the TKL had already encountered trainborne signalling failure earlier at 10:02 hours due to the around 20 minutes counter look ahead validity requirements. 

5.1.6 After the interconnections between the signalling systems of the relevant lines and the Alstom and Siemens sector computers between KWT and LAT were isolated, all sector computers were effectively rebooted to complete the entire re-initialization process and the signalling system for the four incident lines resumed normal. "

With credit for calling my attention to the report to:
Date: Sun, 30 Dec 2018 15:39:37 +0800

From: Richard Stein 
Subject: Re: MTR East Rail disruption caused by failure of both primary 

 and backup (Stein, RISKS-30.89)

Sometimes Bug Severity Isn't the Most Important Thing

Generally you need to take into account both the consequence of a software defect as well as how often it occurs when doing bug triage.  (See: Using a Risk Analysis Table to Categorize Bug Priority)

But an important special case is one in which the consequence is a business consequence such as brand tarnish rather than a spectacular software crash.   I used to use a hypothetical example of the audience's company name being misspelled on the system display to illustrate the point.  Well, it's not hypothetical any more!

Lamborghini sells a quarter-million dollar SUV with numerous software defects, including spelling the company name as "Lanborghini"   Guess which defect gets the press?

https://carbuzz.com/news/lamborghini-urus-experiencing-major-software-bugs

And it turns out that a software update not only didn't solve the typo, but also broke a bunch more functionality.  

Cost of highly safety critical software

It's always interesting to see data on industry software costs. I recently came across a report on software costs for the aviation industry. The context was flight-critical radio communications, but the safety standards discussed were DO-178B and DO-254, which apply to flight controls as well.

Here's the most interesting picture from the report for my purposes:

(Source: Page 28 https://www.eurocontrol.int/sites/default/files/content/documents/communications/29012009-certification-cost-estimation-for-fci-platform.pdf.pdf )

Translating from DO-178B terminology, this means:

• DAL A  (failure would be "catastrophic"):  3 - 12 SLOC/day
• DAL B  (failure would be "hazardous"): 8 - 20 SLOC/day
• DAL C (failure would be "major"): 15 - 40 SLOC/day
• DAL D (failure would be "minor"): 25 - 64 SLOC/day

Worth noting is that, in my experience, really solid mission critical but NOT life-critical embedded software can be done at up to 16 SLOC per day for well-run experienced teams, so it tends to line up with DAL B costs.

For interpretation, "DAL" expresses a criticality level (a "Development Assurance Level"), with more critical software requiring more rigorous processes.  The document has quite a lot to say about how the engineering process works, and is worth a read if you want to see how the aviation folks do business.  (I'm aware that DO-178C is out, but this paper talks about the older "B" version.)    Note that there are other cost models in the paper that are less pessimistic in that report, but this is the one that says "industry experience."

Have you found other cost of software data for embedded or mission critical systems?

Potentially deadly automotive software defects

Page moved here:

https://betterembsw.blogspot.com/p/potentially-deadly-automotive-software.html

Different types of risk analysis: ALARP, GAMAB, MEMS and more

When we talk about how much risk is enough, it is common to do things like compare the risk to current systems, or argue about whether something is more (or less) likely than events such as being killed by lightning. There are established ways to think about this topic, each with tradeoffs.

The next time you need to think about how much risk is appropriate in a safety-critical system, try these existing approaches on for size instead of making up something on your own:

ALARP: "As Low As Reasonably Practicable"  Some risks are acceptable. Some are unacceptable. Some are worth taking in exchange for benefit, but if that is done the risk must be reduced to be ALARP.

GAMAB: "Globalement Au Moins Aussi Bon"  Offer a level of risk at least as good as the risk offered by an equivalent existing system. (i.e., no more dangerous than what we have already for a similar function)

MEM: "Minimum Endogenous Mortality"  The technical system must not create a significant risk compared to globally existing risks. For example, this should cause a minimal increase in overall death rates compared to the existing population death rates.

MGS: "Mindestens Gleiche Sicherheit"   (At least the same level of safety) Deviations from accepted practices must be supported by an explicit safety argument showing at least the same level of safety. This is more about waivers than whole-system evaluation.

NMAU: "Nicht Mehr Als Unvermeidbar"  (Not more than unavoidable)  Assuming there is a public benefit to the operation of the system, hazards should be avoided by reasonable safety measures implemented with reasonable cost.

Each of these approaches has pros and cons.  The above terms were paraphrased from this nice discussion:
Kron, On the evaluation of risk acceptance principles,
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.455.4506&rep=rep1&type=pdf

There is an interesting set of slides that covers similar ground here, and works some examples. In particular the graphs involving whether risks are taken voluntarily for different scenarios is thought provoking:
http://agse3.informatik.uni-kl.de/teaching/suze/ws2014/material/folien/SRES_03_Risk_Acceptance.pdf

In general, if you want to dig deeper into this area, a search on
    gamab mem alarp 
will bring up a number of hits

Also note that legal and other types of considerations exist, especially regarding product liability.

Activity on my Safe Autonomy Blog

For those who might have missed it, most of my recent blogging has been on my Safe Autonomy Blog: https://safeautonomy.blogspot.com

Recent post topics include slides and paper preprints such as:

• Robustness Testing of Autonomy Software (ICSE 2018)
• Safety Validation and Edge Case Testing for Autonomous Vehicles 
• Heavy Tail Ceiling Problem for AV Testing
• Toward a framework for Highly Automated Vehicle Safety Validation

Embedded Code Quality and Best Practices Training Videos full length

I've posted the full series of my available embedded system code quality and related best practices videos on YouTube.  These are full-length narrated slides of the core set of safety topics from my new course.  They concentrate on getting the big picture about code quality and good programming practices.

• Code Quality, Safety, Security overview
• Global Variables
• Spaghetti Code
• Coding Style for Humans 
• Coding Style for Compilers/Language Use
• Stack Overflow 
• Peer Reviews 
• Key Software Process metrics 

Each of the videos is posted to YouTube as a playlist, with each video covering a slide or two. The full lecture consists of playing the entire play list, with most lectures being 5-7 videos in sequence. (The slide download has been updated for my CMU grad course, so in general has a little more material than the original video. They'll get synchronized eventually, but for now this is what I have.)

Obviously there is more to code quality and safety than just these topics. Additional topics are available slides-only.  You can see the full set of course slides including for those lectures and others here:
  https://users.ece.cmu.edu/~koopman/lectures/index.html#642