Contents
1.    INTRODUCTION    12
1.1    Scope    12
1.2    Purpose    13
1.3    Acknowledgments    14
1.4    Associated Documents    14
1.5    Roadmap of this Guidebook    14
2.    SOFTWARE SAFETY IN A  SYSTEM SAFETY CONTEXT    17
2.1    What is a Hazard?    17
2.2    What Makes Software Hazardous?    18
2.2.1    What is Safety Critical Software?    19
2.2.2    How Does Software Control Hazards?    19
2.2.3    What About Hardware Controls?    19
2.2.4    Caveats with Software Controls    20
2.2.5    What is Fault Tolerance?    21
2.3    The System Safety Program    21
2.3.1    Safety Requirements Determination    22
2.4    Preliminary Hazard Analysis (PHA)    23
2.4.1    PHA Approach    24
2.4.1.1    Identifying Hazards    25
2.4.1.2    Risk Levels    26
2.4.1.3    NASA Policy for Hazard Elimination/Control    28
2.4.2    Preliminary Hazard Analysis Process    28
2.4.3    Tools and Methods for PHA    30
2.4.4    PHA is a Living Document    32
2.5    Software Subsystem Hazard Analysis    32
3.    SOFTWARE SAFETY PLANNING    33
3.1    Software Development Life-cycle Approach    34
3.2    Scope of Software Subsystem Safety Effort    36
3.2.1    Identify Safety Critical Software    37
3.2.2    Categorize Safety Critical Software Subsystems    38
3.2.2.1    Software Control Categories    39
3.2.2.2    Software Hazard Criticality Matrix    40
3.2.3.1    Determine Extent of  Effort    42
3.2.3.2    Oversight Required    43
3.2.3.3    Tailoring the Effort    44
3.2.3.3.1    “Full” Software Safety Effort    45
3.2.3.3.2    “Moderate” Software Safety Effort    45
3.2.3.3.3    “Minimum” Software Safety Effort    46
3.2.3.3.4    Match the Safety Activities to Meet the Development Effort    46
3.3    Incorporating Software Safety into Software Development    47
4.    SAFETY CRITICAL SOFTWARE DEVELOPMENT    55
4.1    Software Concept and Initiation Phase    55
4.2    Software Requirements Phase    56
4.2.1    Development of Software Safety Requirements    57
4.2.1.1    Safety Requirements Flow-down    57
4.2.2     Generic Software Safety Requirements    57
4.2.2.1     Fault and Failure Tolerance/Independence    58
4.2.2.2     Hazardous Commands    60
4.2.2.3     Timing, Sizing and Throughput Considerations    61
4.2.3    Formal Methods - Specification Development    63
4.2.3.1     Why Is Formal Methods Necessary?    64
4.2.3.2     What Is Formal Methods?    65
4.2.4     Model Checking    66
4.2.4.1     How Model Checking Works    66
4.2.4.2     Tools    67
4.2.4.3     Challenges    68
4.2.5    Formal Inspections of Specifications    68
4.2.6    Test Planning    69
4.3    Architectural Design Phase    70
4.3.1    Safety Objectives of Architectural Design    70
4.3.1.1    Fault Containment Regions    71
4.3.1.2    N-Version Programming    72
4.3.1.3  Redundant Architecture    73
4.3.2    Structured Design Techniques    73
4.3.2.1    Object Oriented Analysis and Design    75
4.3.2.2     Unified Modeling Language (UML)    77
4.3.3    Selection of COTS and Reuse    78
4.3.4    Selection of development tools and operating systems    78
4.3.5    Coding Standards    78
4.3.6    Test Plan Update    79
4.4    Detailed Design Phase    79
4.5    Software Implementation    81
4.5.1    Coding Checklists    81
4.5.2    Defensive Programming    82
4.5.3    Refactoring    82
4.5.4    Unit Level Testing    83
4.6     Software Integration and Test    84
4.6.1    Testing Techniques    86
4.6.2    Test Setups and Documentation    91
4.6.3    Integration Testing    92
4.6.4    Object Oriented Testing    92
4.6.5    System Testing    93
4.6.6    Regression Testing    94
4.6.7    Software Safety Testing    95
4.6.8    Test Witnessing    96
4.7    Software Acceptance and Delivery Phase    97
4.8    Software Operations & Maintenance    97
5.    SOFTWARE SAFETY ANALYSIS    99
5.1    Software Safety Requirements Analysis    100
5.1.1    Software Safety Requirements Flow-down Analysis    100
5.1.1.1    Checklists and cross references    101
5.1.2    Requirements Criticality Analysis    101
5.1.2.1    Critical Software Characteristics    103
5.1.3    Specification Analysis    105
5.1.3.1    Control-flow analysis    106
5.1.3.2    Information-flow analysis    106
5.1.3.3    Functional simulation models    106
5.1.4    Formal Inspections    107
5.1.5    Timing, Throughput And Sizing Analysis    107
5.1.6    Software Fault Tree Analysis    109
5.1.7    Conclusion    109
5.2    Architectural Design Analysis    110
5.2.1    Update Criticality Analysis    110
5.2.2    Conduct Hazard Risk Assessment    111
5.2.3    Analyze Architectural Design    111
5.2.3.1    Design Reviews    112
5.2.3.2    Prototype/Animation/Simulation    112
5.2.4    Interface Analysis    113
5.2.4.1    Interdependence Analysis    113
5.2.4.2    Independence Analysis    113
5.2.5    Update Timing, Throughput, and Sizing Analysis    113
5.2.6    Update Software Fault Tree Analysis    113
5.2.7    Formal Inspections of Architectural Design Products    114
5.2.8     Formal Methods and Model Checking    114
5.3    Detailed Design Analysis    114
5.3.1    Design Logic Analysis (DLA)    115
5.3.2    Design Data Analysis    115
5.3.3    Design Interface Analysis    116
5.3.4    Design Constraint Analysis    117
5.3.5    Design Functional Analysis    117
5.3.6    Software Element Analysis    118
5.3.7    Rate Monotonic Analysis    118
5.3.8    Dynamic Flowgraph Analysis    118
5.3.9    Markov Modeling    119
5.3.10    Measurement of Complexity    119
5.3.10.1    Function Points    120
5.3.10.2    Function Point extensions    121
5.3.11    Selection of  Programming Languages    122
5.3.12    Formal Methods and Model Checking    123
5.3.13    Requirements State Machines    123
5.3.14    Formal Inspections of Detailed Design Products    123
5.3.15    Software Failure Modes and Effects Analysis    123
5.3.16    Updates to Previous Analyses    124
5.4    Code Analysis    124
5.4.1    Code Logic Analysis    125
5.4.2    Code Data Analysis    126
5.4.3    Code Interface Analysis    126
5.4.4    Update Measurement of Complexity    126
5.4.5    Update Design Constraint Analysis    126
5.4.6    Formal Code Inspections,  Checklists, and Coding Standards    127
5.4.7    Applying Formal Methods to Code    127
5.4.8    Unused Code Analysis    128
5.4.9    Interrupt Analysis    128
5.4.10    Final Timing, Throughput, and Sizing Analysis    129
5.4.11    Program Slicing    129
5.4.12    Update Software Failure Modes and Effects Analysis    129
5.5    Test Analysis    130
5.5.1    Test Coverage    130
5.5.2    Formal Inspections of Test Plan and Procedures    130
5.5.3    Reliability Modeling    131
5.5.3.1    Criteria for Selecting a Reliability Model    131
5.5.3.2    Issues and Concerns    132
5.5.3.3    Tools    132
5.5.3.4    Dissenting Views    133
5.5.3.5    Resources    133
5.5.4    Checklists of Tests    134
5.5.5    Test Results Analysis    134
5.5.6    Independent Verification and Validation    134
5.5.7    Resources    135
5.6    Operations & Maintenance    135
6.    SOFTWARE DEVELOPMENT ISSUES    136
6.1    Safe Subsets of Languages    137
6.2    Insecurities Common to All Languages    138
6.3    Method of Assessment    139
6.4    Languages    139
6.4.1    Ada83 and Ada95 Languages    140
6.4.2    Assembly Languages    143
6.4.3    C Language    144
6.4.4    C++ Language    148
6.4.5    C# Language    151
6.4.6    Forth Language    153
6.4.7    FORTRAN Language    154
6.4.8    Java Language    155
6.4.6    LabVIEW    157
6.4.7    Pascal Language    158
6.4.8    Visual Basic    159
6.5    Miscellaneous Problems Present in Most Languages    159
6.6    Programming Languages: Conclusions    161
6.7    Compilers, Editors, Debuggers, IDEs and other Tools    162
6.8     CASE tools and Automatic Code Generation    164
6.8.1     Computer-Aided Software Engineering (CASE)    164
6.8.2     Automatic Code Generation    166
6.8.2.1    Visual Languages    166
6.8.2.2    Visual Programming Environments    167
6.8.2.3    Code Generation from Design Models    167
6.9    Software Configuration Management    169
6.9.1     Change control    170
6.9.2    Versioning    170
6.9.3    Status Accounting    171
6.9.4    Defect Tracking    172
6.9.5    Metrics from your SCM system    172
6.9.6    What to include in your SCM system    173
6.10     Operating Systems    174
6.10.1    Types of operating systems    174
6.10.2    Do I really need a real-time operating system (RTOS)?    174
6.10.3    What to look for in an RTOS    175
6.10.4    Commonly used Operating Systems    177
6.11     Distributed Computing    178
6.12    Programmable Logic Devices    181
6.12.1    Types of Programmable Logic Devices    182
6.12.2    “Program Once” Devices    182
6.12.3    “Reprogram in the Field” Devices    183
6.12.4    Configurable Computing    183
6.12.5    Safety and Programmable Logic Devices    184
6.13    Embedded Web Technology    186
6.13.1    Embedded Web Servers    186
6.13.2    Testing Techniques    187
6.14    AI and Autonomous Systems    188
6.14.1    Examples of Intelligent Autonomous Systems (IAS)    189
6.14.2    Problems and Concerns    190
6.14.3    Case Study – Remote Agent on Deep Space 1    191
6.14.3.1    Remote Agent Description    192
6.14.3.2    Testing and Verification of Remote Agent    192
6.14.3.3    In-flight Validation: How well did it work?    194
6.15    Good Programming Practices for Safety    195
6.16     Wrapping it all up    200
7.    SOFTWARE ACQUISITION    201
7.1 Off-the-Shelf Software    202
7.1.1    Purchasing or Reusing OTS Software: Recommendations    204
7.1.2    Integrating OTS Software into your System    208
7.1.2.1    Sticky stuff:  Glueware and Wrapper Functions    208
7.1.2.2    Redundant Architecture    209
7.1.2.3    Adding or Adapting Functionality    209
7.1.2.4    Dealing with Extra Functionality    210
7.1.3    Special Problems with Reused Software    211
7.1.4    Who Tests the OTS? (Us vs. Them)    211
7.1.3.1    Recommended Analyses and Tests    213
7.2 Contractor-developed Software    214
7.2.1    Contract Inclusions    214
7.1.2.1    Safety Process    215
7.1.2.2    Analysis and Test    215
7.1.2.3    Software Assurance and Development Process    215
7.1.2.3    Contractor Surveillance    216
7.1.2.4    Software Deliverables    216
7.1.2.5    Independent Verification and Validation (IV&V)    217
7.1.2.6    Software Change Process    217
7.1.2.7    Requirements Specification    217
7.2.2    Monitoring Contractor Processes    217
7.2.3    Recommended Software Testing    218
8.    REFERENCES    219
APPENDIX A    229
Glossary of Terms    229
APPENDIX B    Software Fault Tree Analysis (SFTA)    249
B.1 Software Fault Tree Analysis Description    249
B.2  Goal of Software Fault Tree Analysis    249
B.3  Use of Software Fault Tree Analysis    251
B.4  Benefits Of Software Fault Tree Analysis    253
APPENDIX C    Software Failure Modes and Effects Analysis    257
C.1    Terminology    257
C.2    Why do an SFMEA?    258
C.3    Issues with SFMEA    258
C.4    The SFMEA Process    260
C.4.1    Identify Project/system Components    260
C.4.2    Ground Rules    261
C.4.3    Identify Failures    263
C.4.3.1     Examination of Normal Operations as Part of the System    264
C.4.3.2     Identify Possible Areas for Faults    264
C.4.3.3     Possible Failure Modes    265
C.4.3.4     Start at the Bottom    265
C.4.4    Identify Consequences of each Failure    266
C.4.5    Detection and Compensation    267
C.4.6    Design Changes    267
C.4.7    Impacts of Corrective Changes    268
C.4.8    Example forms    269
APPENDIX D    Requirements State Machines    270
D.1    Characteristics of State Machines    270
D.2    Properties of Safe State Machines    270
D.3    Input/Output Variables    270
D.4    State Attributes    270
D.5    Trigger Predicates    270
D.6    Output Predicates    270
D.7    Degraded Mode Operation    270
D.8    Feedback Loop Analysis    270
D.9    Transition Characteristics    270
D.10    Conclusions    270
APPENDIX E    270
E.1    Checklists for Off-the-Shelf (OTS) Items    270
E.2    Generic Software Safety Requirements From MSFC    270
E.3    Design for Safety Checklist    270
E.4    Checklist of generic (language independent) programming practices    270
E.5    Checklist of assembly programming practices for safety    270
E.6    Checklist of C programming practices for safety    270
E.7     Checklist of C++ programming practices for safety    270
E.8    Checklist of Fortran programming practices for safety    270
E.9     Checklist of Pascal programming practices for safety    270
E.10    Checklist for Visual Basic    270
E.11    Checklist for selecting an RTOS    270
E.12 Good Programming Practices Checklist    270
E.13 Software Requirements Phase Checklist    270
E.14 Architectural Design Phase Checklist    270
E.15 Detailed Design Phase Checklist    270
E.16 Implementation Phase Checklist    270
E.17 Software Testing Phase Checklist    270
E.18 Dynamic Testing Checklist    270
E.19 Software System Testing Checklist    270


Share