Researchers wrapped up nine NASA Software Assurance Research Program (SARP)-sponsored research projects aimed to benefit Software Assurance (SA) processes across the agency and presented the results to the Software Assurance Working Group (SAWG). After select proposed initiatives were chosen in 2016, researchers were given a year to develop, analyze, test, record findings and present results to the SAWG. Two of the projects were multi-year studies that continued from the prior year.
SARP, a Headquarters SA program that is delegated to the Independent Verification and Validation (IV&V) Program’s Safety and Mission Assurance (SMA) Support Office, was designed to address fundamental SA problems in the field of software engineering and stay current with new practices, methods and tools needed to produce safe and reliable software. Each year, the SAWG identifies initiatives based on current needs in the SA community, collects research proposals, evaluates their intent and awards a set of them that would best serve the SARP objectives.
“The researchers are encouraged to propose initiatives based on the needs that we have identified,” said SMA Support Office Lead for the IV&V Program Don Ohi. “However, we also welcome them to submit proposals on other topics that may identify needs we haven’t expressed.”
The following list contains an overview and the results of each Fiscal Year 2017 SARP project.
As Model-Based System Engineering (MBSE) became a more promising direction for developing systems and software, the SA community recognized that it came with substantial MBSE-specific costs. The goal for this project was to create a model that tied together the benefits and costs of MBSE, so that SA practitioners could define the conditions under which it was worth using.
Results: Researchers reviewed Model-Based Mission Assurance (MBMA) practices and surveyed missions using MBSE development. They gathered data to capture the benefits and costs that go along with adopting MBSE. Researchers developed best practices and guidelines for SA practitioners to understand the costs to build systems and assure them in a model and ensure they make the right decision to achieve the best assurance for their systems.
The SA community started looking at tools for auto-generating code to save time and help assure software in MBSE. The goal for this project was to recommend adaptations and provide guidance to SA practitioners needed to address opportunities and challenges.
Results: Researchers investigated how auto-generated programming would impact SA. One challenge they found was that the auto-generated code was difficult to read, as the names of variables were not connotative and elements of the code were not as human readable. The study suggested that auto-generated code was not as efficient as hand-generated code, since the nature of the technology tends to solve model constructs with complicated algorithms that a human developer would not use. This could lead to unforeseen issues in the process or utilization. The project compared benefits and detractors and created guidelines for SA practitioners to understand the types of risks associated with auto-generated code and how it could lead to positive or negative results.
Research on the SAPE tool carried over from Fiscal Year 2016. One of the challenges SA practitioners face is that government funding tends to go towards projects that put more emphasis on engineering, rather than assurance resources. SAPE is a tool that would provide SA practitioners a template for work plans that would efficiently break down cost estimates and capture statuses and results from executing a SA task. Using SAPE, SA practitioners hope to provide consistent estimates to project managers, so they will understand the true cost of doing appropriate SA on missions. The goal for this project was to expand SAPE to include Software Safety requirements standards and implement flexibility to import different versions of the standards and validate the tool through NASA-wide deployment.
Results: Researchers pulled metrics from the tool to develop a clear understanding of what is accomplished on missions in SA. They collected data to better determine if there is a gap in what needs to be accomplished and how it may impact missions. Researchers also updated tool features, simplified architecture and eliminated dependencies. The project will continue in Fiscal Year 2018, when researchers focus on finding how to create a centralized version to streamline maintenance and deployment.
Carried through from Fiscal Year 2016, this project aimed to create a database to capture all types of adverse conditions the SA community sees in NASA missions. The goal of this project was to improve SA strategies by reusing adverse condition data across projects to improve fault management and ultimately the probability of mission success.
Results: Researchers created a database tool to capture fault data from mission projects. Then, SA practitioners would be able to query for elements that apply to their missions and get a head start on planning how to build in fault tolerant features or eliminate those types of faults. Researchers investigated ways in which the adverse condition database could be designed to interoperate with SAPE and other SA tools. The role of an adverse conditions repository in assurance of fault management systems was published and presented at the 33rd Space Symposium.
NASA’s mission systems are often built on Real-Time Operating Systems (RTOS) that gather data and run in loops, sending commands to subsystem devices. Software that runs in control loops, such as attitude control, is vital for mission success. Since RTOS are off-the-shelf components that are used on multiple missions, the researchers wanted to develop a set of tests and checks to pre-qualify RTOS against a common set of criteria, alleviating the need for this work to be done on each mission project. The goal of this project was to compare and contrast RTOS, and identify a core set of artifacts for RTOS qualification.
Results: Researchers developed documentation sets and test cases and ran several operating systems through them. They looked at most commonly used features, costs and security aspects of the operating systems and created guidelines based on the findings. This project also presented a case study conducted on RTOS qualification of the RTEMS operating system at the Flight Software Workshop.
Researchers wanted to improve application of SA on projects using Model-Based Software Development (MBSD) and auto-generated code. The goal of this project was to produce guidance to help SA practitioners to sufficiently identify project risks and plan appropriate assurance activities.
Results: Researchers gathered data-documented MBSD best practices for modeling effectively. They created a model of the development process for using model-based tools. Rules can be built into auto-checkers in the model to evaluate and find problems in the model being developed.
The SA community follows a System Safety process that reviews hazards and how they occur. Software features are built into systems to flag requirements that are safety critical to prevent hazards from occurring. This is a key element in SA, as requirements specification flaws are the biggest contributing factor to most of the accidents related to software. Many of these requirements are ambiguous, incomplete or missing. The goal for this project was to develop a method to identify Safety-Critical Requirements (SCR) and to effectively address problems early in development.
Results: Researchers developed techniques to auto-scan requirements to identify those that potentially should be labeled safety-critical based on the requirement sets they studied. They created rules for SA practitioners to use when performing analysis. This project will continue in Fiscal Year 2018, where researchers will try to make advances in the tool.
SA practitioners are able to tailor software assurance requirements using a risk-based approach if the practitioner determines the requirements are not applicable or the project under consideration does not need the level of adherence to meeting particular SA requirements (e.g. a small, 6 month project would not need biannual audits). The goal of this project was to research and characterize the tailoring that has occurred on projects and create a methodology for SA practitioners to assess the risks of tailored requirements and enable projects to understand the individual and cumulative risk of tailoring.
Results: Researchers reviewed how software requirements tailoring is implemented across NASA, then performed risk assessments of past projects to understand decision patterns and the risks assumed by tailoring. SAPE will potentially have features built into the tool that support methodologies found in this project.
Researchers looked into threat modeling, a tool that would identify cyber-security threats that could affect software components. The tool, developed by Microsoft, contains a library of cyber-security threats. When implemented into a model, the threat modeling tool can use the library to pinpoint particular exposures to cyber-attacks in the mission architecture. The goal of this project was to review the Microsoft threat modeling tool and determine how it could be adapted for use in NASA missions.
Results: Researchers applied the tool to reusable software packages developed in Open Source. They looked to see what vulnerabilities they could find, then applied the tool to compare results. Researchers were able to show that using this tool could help with cyber-security assurance in missions and developed a tutorial on Threat Modeling. The project is being considered for use in other initiatives.
SARP researchers are currently working on 10 initiatives through Fiscal Year 2018. To find out more about current and past SARP initiatives visit SARP’s NASA Engineering Network or contact Ohi or Martha Wetherholt, the SA Program Manager and Technical Fellow.