NASA projects go through the full life cycle, from concept to implementation to closeout, with a number of reviews along the way. These reviews ensure that the developed project can meet mission objectives and do so safely. The project must make a case for itself: show and explain why the system will work as intended. Enter, the safety assurance case, or simply safety case as it may be more commonly known. Researchers at Ames Research Center have been fine tuning their notion of the safety case and developing a tool to make its development easier.

Safety assurance cases are based on the concept of assurance cases: a comprehensive, auditable safety Risk Management artifact that acts as a record of identified safety risks and the processes and mechanisms in place to reduce those risks. More simply, it’s evidence to back up claims. Assurance cases capture a variety of rationale, including how claims follow from the evidence supplied, why the verification is appropriate, and counterarguments and how they were addressed.

Assurance cases are not a new concept; there’s a decades-long history of their growing use in industry and government. In 1990, the report on the Piper Alpha disaster in the North Sea recommended the use of safety cases for offshore installations, leading to the U.K Ministry of Defense adopting them into its safety management requirements standard. In the years since, safety cases have become more common in safety-critical industries, including offshore oil and gas companies, defense and civil aviation organizations, the automotive industry, the Food and Drug Administration, the Federal Aviation Administration (FAA), and the Nuclear Regulatory Commission (NRC), to name a few.

In response to the growing use of assurance cases, researchers Ewen Denney and Ganesh Pai, both with KBR Wyle Services in the Robust Software Engineering technical area at Ames, refined the popular notion of a safety case (compatible with NASA’s risk-informed safety case concept) and developed the Assurance Case Automation Toolset (AdvoCATE) to assist in developing assurance cases.

“It’s a recognition on our part that safety cases are becoming more accepted and, second, we ourselves have developed safety cases and found that they are a lot of work,” said Denney. “It [AdvoCATE] makes our own jobs easier.”

“Developing our own toolset was another way that we could leverage our experience to build a better assurance framework that can bring to bear some of the automation technologies that we’re using,” added Pai.

Both Denney and Pai believe it’s important that the safety case be a living document. The case should drive the development of the mission and serve as a record of identified safety risks throughout the life cycle, proving they are well understood and mitigated.

“Traditionally, a safety case is a static thing,” said Denney. “But really, what it should be is an active document you use to govern your activities, so you update it when you learn more about your hazards and the effectiveness of your mitigations and so on. Traditionally, people have unfairly criticized safety cases because there is a perception that you get it approved and then forget about it.”

In addition to viewing it as a dynamic document, Denney and Pai’s notion of a safety case includes

• An explicit statement of safety assurance objectives
• A structured argument with rationale for why the evidence supports the claims made and a framework that incorporates various forms of heterogeneous evidence and analysis
• A safety architecture that provides a risk basis
• A hazard and risk analysis, along with assurance requirements
• An evidence model

The structured arguments are given in a graphical notation called Goal Structuring Notation (GSN), which has elements for capturing claims, reasoning strategies, evidence and contextual information. GSN-based arguments have close connections to the objective hierarchies approach promulgated by NASA’s Office of Safety and Mission Assurance.

 “The idea of the assurance case is it’s the single overarching artifact that shows how everything fits together,” said Denney. “Rather than having these independent analyses that may have some bearing on each other, you want to show how, at some level, these things may fit together.”

AdvoCATE connects all the pieces. It provides hazard analysis and risk assessment, safety and assurance requirements, structured argument development, safety architecture and development, traceability and consistency, and evidence management and analytics through dashboards.

“In terms of research, our goal is to understand what is assurance,” explained Denney. “The tool is a good test bed for our ideas.”

Another benefit of the tool is that it provides an open platform to expand capabilities and allows for linkage to other data systems, a capability that fits well into NASA’s Digital Transformation efforts.

Now that the tool exists, users across NASA, as well as other government organizations and industry, are finding use for it.

“We are under a mandate to develop technology that can be transitioned to U.S. industry within our group,” explained Denney. “And so one of our primary motivators is to develop advanced technology we can transition and that’s something we’re doing with this tool. It’s been requested by quite a range of different companies.”

Both the FAA and NRC have been given briefings on the tool, and it has been requested by a number of companies from the aerospace, self-driving car, medical device and cybersecurity domains. Use is also growing within NASA, but Denney and Pai want to continue improving the tool to broaden its use, especially at the agency.

 “A lot of our application thus far has been in aviation systems, and we’re very keen to apply it to the space domain,” said Pai. “So, we’ve talked to other colleagues within the spaceflight divisions of NASA and we’d like to continue to push that.”

Specifically, they want to broaden the use of the safety case during reviews and approval processes. They also hope to continue their push to shift safety cases from static to dynamic.

“We want it [AdvoCATE] to have some means for updating operational information and updating the safety case as you learn more in the mission.”

“Another thing we’re working on is the use of ontologies,” said Pai. “Ontologies are a means for developing a vocabulary in a domain and how they relate to each other. It’s a nice, user-friendly way to add domain-specific notions that build upon the core safety concepts implemented in the tool.”

Denney and Pai note that it’s a balance: Originally, the tool was developed for their own research, but as it gets distributed to other stakeholders, there are other needs and purposes to address so the tool integrates into their specific practices. Increasingly, they are applying it to autonomy, specifically with regard to aviation and maritime domains. More specific capabilities will be introduced into the tool as they expand the dynamic safety case capabilities.

“One challenge is that we obviously are drawing on our own personal experiences to develop this tool,” said Denney. “But a key stakeholder for something like this is an evaluator of a safety case, and we don’t have personal experience with that. It’s something we lack insight into compared to actually developing safety cases as a safety engineer, so we would rely on people we’ve worked with in the past here at Ames. We need insight from decision makers.”

Those interested in learning more about safety cases or AdvoCATE, or those interested in using the tool, can contact Denney or Pai.

The following is a small excerpt of the Software Objectives Hierarchy, its representation in Goal Structuring Notation and its subsequent tailoring to a specific project (BioSentinel).