Post #1143050

This is a long-overdue post for @iris_meredith on our process for evaluating and accepting safety analysis software. We start with EPRI guidance for consumer-grade dedication, strip out the inapplicable, intractable, and (IMO) wrong parts then use what remains to set requirements, acceptance methods, and acceptance criteria to define test and inspection plans. https://restservice.epri.com/publicdownload/000000003002002289/0/Product There are some very good parts of the EPRI guidance (section 6.4) and some very bad parts. One of the worst parts of this guidance is the suggestion to use FMEA to determine software safety classification. Software FMEA is effort-intensive and intractable for all but the simplest control software. Using FMEA to analyze complex design and analysis software requires simplifying the FMEA process to the point that it is ineffective. The net effect is to waste limited engineering resources to produce an evaluation appears intensive but which presents a false sense of rigor. This is not simply my opinion, this is backed by both Nancy Leveson in "Safeware" and Chris Hobbs in his book on safety-critical embedded systems now in its third edition https://www.taylorfrancis.com/books/mono/10.1201/9781003598145/embedded-software-development-safety-critical-systems-chris-hobbs Bluntly, using FMEA to evaluate conplex software is a dangerous waste of limited engineering resources. The FMEA process must be simplified to the point of ineffectiveness for the analysis to be completed with the time and resources typically available for V&V activities. Worse, the EPRI guidance only uses FMEA here to determine the safety-significance of _design and analysis_ software. Not instrumentation or control (I&C) software directly connected to an operating plant but the sort of design and analysis software used by reactor vendors and engineering staff outside the protected area of a plant. One can make a strong argument that a failure of design and analysis software cannot have an impact on safety-related plant structures, systems, or components (SSCs) in an operating plant. If MELCOR or RELAP crashes or produces wrong output, it physically cannot affect a concrete wall or pipe run or fuel element due to separation in time and space. If you want to make arguments about diffuse and indirect effects of design and analysis software errors which lead to poor design or licensing decisions, we need to stop and consider whether this QA process (and all of NQA-1) is being applied so far outside its original remit that it is no longer effective or helpful. I'm not arguing against software QA or NQA-1 _in its original intended role of safeguarding operating power plants_. I'm arguing that extending QA standards originally intended to qualify plant hardware (pumps, valves, piping, cable, paint, relays, switches, etc.) leads to confusion and poor outcomes when applied to design and analysis software like MELCOR or RADTRAD or Navisworks. Thankfully IEEE has its own SIL process for assuring I&C systems. That's the realm of Chris Hobbs and why his book is so valuable. Unfortunately, we don't have a better alternative than NQA-1 for dealing with design and anslysis software so we have to severely hack the EPRI guidance to be usable and effective and ensure we're meeting the spirit of the standard. Short answer: Ignore FMEA and use the NITSL impact categorization method. It is no less rigorous than the bastardized oversimplified shadow of FMEA that EPRI suggests while being clear, defensible, and trivial to perform. Sometimes you don't need 150 pages of quantitative analysis to justify an engineering decision, just sayin'...