“Automating Software Complexity Monitoring and Security Analysis,” a Software Assurance Research Program (SARP) project, intends to identify a set of complexity-related metrics and automatically generate them for use by Software Assurance (SA) personnel and developers throughout the entire software development process. SARP — hosted by NASA’s Independent Verification and Validation (IV&V) Program — is sponsoring five research projects in Fiscal Year 2020 aimed to benefit SA processes across the agency.

Research Goals

The complexity metrics will be able to help to identify areas of focus where software defects may be more likely to occur, including security-related concerns. This research, lead by Chris Williams and Jerry Williams, NASA IV&V SA Tools Team, is driven by the need for more practical and objective monitoring methods for areas of possible security concern earlier in the development process, as security tends to be focused on vulnerability scanning and remediation of problems in operational systems. Automated metrics monitoring can encourage changes in developer habits and better design choices up front. Metrics can support adherence to established coding standards or help inform updates to coding standards.

The team is configuring the automatic generation of the identified complexity metrics to operate within a modern Continuous Integration build approach within a Development and Operations (DevOps) workflow where static code analysis can also be performed against the same target source code. Both static code analysis results and generated complexity metrics are captured in a backend database and web application for viewing. The team is looking at candidate complexity-related metrics to determine if any correlation exists between the reported metrics for a given set of software functions/methods and the reported security defects (e.g., Top 25 Common Weakness Enumerations (CWE)) from performing automated static code analysis. They are identifying and using open source projects in the primary mission-critical languages of interest (C, C++ and Java) to automate the running of static code analysis to determine what, if any, correlation there is between the reported complexity metrics and instances of reported CWE defects.

The team is documenting particular software architecture components/areas which can support security risk identification that can be focused on for complexity-related monitoring from a security standpoint (e.g., authentication, access control and input validation).

Project Benefits 

This research will result in the following value and benefits to SA, which will be immediately useful in practice because the team plans to deliver a working prototype of operational capability at the end of the project:

• New capability to automatically generate a useful set of complexity-related metrics for consideration when performing software risk identification relative to security concerns or other concerns with critical software.
• Flexible and low-cost deployment approach, as the resulting prototype implements an industry standard Docker container and automated build server and DevOps pipeline using open source tools to generate metrics.
• Open source approach to implementing metrics generation, which can be further customized by adding additional metrics of interest or modifying the implementation algorithms.
• Database capture of generated metrics that supports reporting, trending and historical analysis.
• Integrated web results viewing application provides ability to review both static code analysis results and generated metrics in the context of the source code.
• Recommendations and guidance for which complexity-related metrics to include in updated software coding standards and for monitoring throughout software development.
• Documentation of other software architecture areas that can support security risk identification that can be focused on for complexity-related monitoring to review key software from a security standpoint (e.g., authentication, access control and input validation).
• Resulting platform to generate metrics that provides a foundation for possible follow on research into other assurance areas with future potential for automation such as software architecture analysis with the use of Design Structure Matrix (DSM), as combining code metrics with DSM-modeling techniques may provide a useful tool to help identify areas of high technological risk within a software system.

For more information on this project, contact Chris Williams and Jerry Williams.

SARP Background

SARP is aligned to support discipline goals to improve how NASA performs SA activities. The research program is designed to provide NASA with greater knowledge about the SA practices, methods and tools needed to produce safe and reliable software.

SARP is designed to address fundamental SA problems in the field of Software Engineering, primarily as it relates to software safety, quality, IV&V, testability and reliability. It is intended to develop and transfer into practice SA technologies, methods and tools to support and improve the quality of the software produced by and for NASA, and to assist the agency in continuing its leadership in the development of safe, reliable and cost-effective software. Thus, by sponsoring forward-thinking research as well as addressing current needs, SARP helps assure that sufficient and appropriate software risk mitigation is applied to the software that controls and monitors NASA’s systems.