SME-Defined Scenarios for Autonomy (SDSA): A method for exploring complex aviation system safety and performance

Abstract

Trends in aviation systems continue a natural progression towards certain characteristics: autonomy, complexity, safety-criticality. These trends are largely inevitable, as new technology offers new capabilities (e.g. advanced sensors, fast processing, and ubiquitous connectivity at low cost and high availability). However, these trends drastically intensify certain verification and validation (V&V) challenges. Conventional design and test processes, which focus on the primary and intended usages, could miss vitally important “corner cases.” It is our view that increasing levels of autonomy and complexity - along with the need to maintain or improve safety - call for new methods of analysis, verification, and validation to assure system performance and safety. Here we introduce a new method, SME-Defined Scenarios for Autonomy (SDSA), which maximizes the utilization of scenarios and Subject Matter Experts (SMEs). In our view, both of these elements (scenarios and SMEs) warrant more effective, more efficient, and more systematic utilization in complex system V&V. SDSA synthesizes and extends numerous prior methods including scenario/use case development, storyboards, cognitive walkthroughs, and risk assessment. To further tune our method to concerns specific to authority and autonomy (A&A), SDSA incorporates custom “probes” to highlight certain patterns and contributing factors that have led to past failures, with the effect of stimulating SMEs to identify similar patterns in new contexts. Our “structured scenario” format facilitates the exploration and management of complex scenario trees. SDSA can be used early in the design of a system (or system of systems), including the conceptual stage, and can also be used to identify new safety or efficiency concerns in both prototype and fielded systems. This paper introduces the details of SDSA as a nascent method. Following that introduction, we present an initial example recently completed (SDSA applied to landing automation) to illustrate how we can successfully identify detailed, relevant scenario paths of concern.