NASA software developers and engineers are using agile methods to enhance timeliness and efficiency as they develop critical applications for the Space Launch System (SLS) and other major projects. This poses new challenges for NASA’s Software Assurance (SA) professionals who strive to ensure safety and mission success.

Agile Versus Traditional Methods

Several methods fall within agile development, including Scrum, Crystal and Extreme Programming. The biggest difference between agile methods and traditional, plan-driven methods is that agile methods allow more flexibility for project teams to incrementally deliver planned functionality earlier in the development lifecycle.

Capability Maturity Model Integration, the NPR 7150.2A-required method for critical NASA Class A and Class B software projects, and is used heavily for defense and aerospace projects, is a rigorous process improvement model. Its best practices necessitate both documented processes and evidence that the processes are being followed.

On the other hand, most “pure” agile development methods cycle through sprints — quick rounds of production followed by review. There is typically less documentation and the team learns as it goes.

“It’s an ideal approach in private industry, where you want to be first to market. You get your product out there and then you release updates,” said NASA SA Technical Fellow Martha Wetherholt. “But when you’re heading for one launch of one vehicle, where safety needs to be proved and documented, a straight agile approach may not be the best option.”

Wetherholt envisions NASA using an approach that combines elements of agile development with more traditional, plan-based development. Rigorous documentation is still necessary for NASA’s safety critical applications and functions. Software needs to be analyzed to see how it may be contributing to system hazards. Then, needed controls and mitigations need to be designed in and tested to make sure that the controls work as required. Also, software is often used to detect and mitigate hardware hazards; these software requirements and design changes also must be documented, designed in, and verified and validated to work. SA personnel need to provide evidence and proof that appropriate safety measures were taken.

Use in NASA Critical Applications

Marshall Space Flight Center’s (MSFC) SLS flight software team has been doing just that for the last two years, using tailored agile methods in combination with more traditional methods.

Software Engineering Process Group Lead Helen Housch (Cepeda Systems) described a tailored process where portions of the development lifecycle are performed within sprints (agile methods), while others — such as overall planning, black-box requirements development, and final product integration — are done outside of the sprints.

The SLS team has had success with the agile development process, and has seen several improvements. Housch stated communication and coordination among requirements, design/implementation, and test teams have improved significantly, as teams work together to incrementally develop software and other work products.

Communication with external stakeholders (including SA) and senior management has also improved, as incremental development progress is more quantifiable and evident through post-sprint review meetings.

“Incremental development allows the customer to see progress much earlier in the lifecycle,” said Housch. “The traditional software development methods do not always allow customer and manager visibility into the progress until the end of the implementation phase. With agile, stakeholders are able to see progress at regular intervals (every six months or so) as software is developed and planned functionality is released.”

A customized agile approach allows flexibility to tailor the project’s processes to effectively and efficiently meet mission objectives. “No two organizations do agile exactly the same,” said Housch. “The agile approach should be tailored to the goals of the organization that’s performing the agile activities.”

Challenges for Assurance

Agile development’s flexibility, along with the ever-evolving processes, can pose a challenge for SA personnel.

“Some agile development processes like sticky notes and task boards aren’t appropriately documented, which means you can’t see problem areas, issues and how they are being resolved,” explained Wetherholt. “Some people also misunderstand what agile is; real agile development should have quality assurance built in.”

A secondary challenge has been the integration of SA into the development teams. Agile development is typically performed by small, close-knit groups of developers. It can be difficult for SA personnel to bring oversight to the team, especially if they are brought in for only portions of the process. If quality, safety and reliability are truly built into the process, then SA should be performing hazard and reliability analyses during the sprints, and have inputs to daily stand-ups as well as the sprint planning, reviews and retrospectives. This takes dedicated SA personnel that are trained in the agile process.

Wetherholt noted, "It's better if SA is brought into the team [from the beginning] so that they can be trained as part of the team."

The SLS flight software team’s SA personnel participate in sprint review meetings and are involved in the sprints as they elect. Housch agreed that SA participation in the sprint activities is beneficial to the process.

In addition to bringing them into the sprints, Wetherholt has proposed educating SA personnel through Scrum Master training and certification to combat these challenges and to keep up with new technology. A “Scrum Master” is the team member responsible for the agile process working and improving, resolving impediments, and communicating to external stakeholders. Scrum Master is a good possible role for SA to support as an agile project member.

Wetherholt is conducting a pilot by training five or six people this year, and plans to begin benchmarking agile development in the auto, aero and other similar industries next fiscal year.

Despite its challenges, agile development brings creativity and opportunity to NASA’s next-generation missions. In addition, many of NASA’s commercial partners use or will use agile development in their projects. To provide effective oversight, SA personnel must continue to adapt, learn and keep up-to-date with the latest software development processes, and software developers need to embrace SA as part of the agile process.