Categories
Blog software safety

SW Safety Principles Conclusions and References

SW Safety Principles Conclusions and References is the sixth and final blog post on Principles of Software Safety Assurance. In them, we look at the 4+1 principles that underlie all software safety standards.

We outline common software safety assurance principles that are evident in software safety standards and best practices. You can think of these guidelines as the unchanging foundation of any software safety argument because they hold true across projects and domains.

The principles serve as a guide for cross-sector certification and aid in maintaining comprehension of the “big picture” of software safety issues while evaluating and negotiating the specifics of individual standards.

Conclusion

These six blog posts have presented the 4+1 model of foundational principles of software safety assurance. The principles strongly connect to elements of current software safety assurance standards and they act as a common benchmark against which standards can be measured.

Through the examples provided, it’s also clear that, although these concepts can be stated clearly, they haven’t always been put into practice. There may still be difficulties with their application by current standards. Particularly, there is still a great deal of research and discussion going on about the management of confidence with respect to software safety assurance (Principle 4+1).

[My own, informal, observations agree with this last point. Some standards apply Principle 4+1 more rigorously, but as a result, they are more expensive. As a result, they are less popular and less used.]

Standards and References

[1] RTCA/EUROCAE, Software Considerations in Airborne Systems and Equipment Certification, DO-178C/ED-12C, 2011.

[2] CENELEC, EN-50128:2011 – Railway applications – Communication, signaling and processing systems – Software for railway control and protection systems, 2011.

[3] ISO-26262 Road vehicles – Functional safety, FDIS, International Organization for Standardization (ISO), 2011

[4] IEC-61508 – Functional Safety of Electrical / Electronic / Programmable Electronic Safety-Related Systems. International Electrotechnical Commission (IEC), 1998

[5] FDA, Examples of Reported Infusion Pump Problems, Accessed on 27 September 2012,

http://www.fda.gov/MedicalDevices/ProductsandMedicalProcedures/GeneralHospitalDevicesandSupplies/InfusionPumps/ucm202496.htm

[6] FDA, FDA Issues Statement on Baxter’s Recall of Colleague Infusion Pumps, Accessed on 27 September 2012, http://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm210664.htm

[7] FDA, Total Product Life Cycle: Infusion Pump – Premarket Notification 510(k) Submissions, Draft Guidance, April 23, 2010.

[8] “Report on the Accident to Airbus A320-211 Aircraft in Warsaw on 14 September 1993”, Main Commission Aircraft Accident Investigation Warsaw, March 1994, http://www.rvs.unibielefeld.de/publications/Incidents/DOCS/ComAndRep/Warsaw/warsaw-report.html  Accessed on 1st October 2012.

[9] JPL Special Review Board, “Report on the Loss of the Mars Polar Lander and Deep Space 2 Missions”, Jet Propulsion Laboratory”, March 2000.

[10] Australian Transport Safety Bureau. In-Flight Upset Event 240Km North-West of Perth, WA, Boeing Company 777-2000, 9M-MRG. Aviation Occurrence Report 200503722, 2007.

[11] H. Wolpe, General Accounting Office Report on Patriot Missile Software Problem, February 4, 1992, Accessed on 1st October 2012, Available at: http://www.fas.org/spp/starwars/gao/im92026.htm

[12] Y.C. Yeh, Triple-Triple Redundant 777 Primary Flight Computer, IEEE Aerospace Applications Conference pg 293-307, 1996.

[13] D.M. Hunns and N. Wainwright, Software-based protection for Sizewell B: the regulator’s perspective. Nuclear Engineering International, September 1991.

[14] R.D. Hawkins, T.P. Kelly, A Framework for Determining the Sufficiency of Software Safety Assurance. IET System Safety Conference, 2012.

[15] SAE. ARP 4754 – Guidelines for Development of Civil Aircraft and Systems. 1996.

Software Safety Principles: End of the Series

This blog post series was derived from ‘The Principles of Software Safety Assurance’, by RD Hawkins, I Habli & TP Kelly, University of York. The original paper is available for free here. I was privileged to be taught safety engineering by Tim Kelly, and others, at the University of York. I am pleased to share their valuable work in a more accessible format.

If you found this blog article helpful then please leave a review, below. If you have a private question or comments then please connect here.

Categories
Blog Safety Management

Proportionality

Proportionality is about committing resources to the Safety Program that are adequate – in both quality and quantity – for the required tasks.

Introduction to Proportionality

Proportionality is a concept that should be applied to determine the allocation of resource and effort to a safety and environmental argument based on its risk.  It is a difficult concept to attempt to distil into a process as each Product, System or Service will have different risks, objectives, priorities and interfaces that make a ‘one size fits all’ approach impossible.

This section describes an approach that may be used to assist in applying the concept of proportionality; it seeks to guide you in understanding where a proportionate amount of effort can be directed, while at the same time maintaining the overriding principle that Risk to Life must be managed.  Regulators require that a proportional approach is used and there are many methods that try to achieve this.  Some focus on the amount of evidence needed to justify a safety argument; some provide more emphasis on the application of activities that are required to make a safety argument and some consider that fulfilling certain criteria can lead to an assessment of risk, but one requirement that is at the centre of any proportional approach is that safety risks are acceptable. 

A fundamental consideration of a proportional approach is considering compliance against assessment criteria.  The Health and Safety Executive’s view is that there should be some proportionality between the magnitude of the risk and the measures taken to control the risk. The phrase “all measures necessary” should be interpreted with this principle in mind. Both the likelihood of accidents occurring and the severity of the worst possible accident determine proportionality.  Application of proportionality should highlight the hazardous activities for which the Duty Holder should provide the most detailed arguments to support the demonstration [that risk is acceptable].

The following considerations may affect proportionality, in a defence context:

  1. Type of consequence;
  2. Severity;
  3. The stage in the Life cycle;
  4. Intended use (CON OPS/Design Intent);
  5. Material state (degradation);
  6. Historical performance;
  7. Cost of safety;
  8. Cost of realising risk;
  9. Public Relations;
  10. Persons at Risk:
    1. 1st,2nd,3rd Party;
    1. Military
    1. Civilian;
    1. Civil Servants;
    1. Contractors;
    1. General public;
    1. VIPs;
    1. Youths;
  11. Volume;
  12. Geographical spread/transboundary.

Some important points that should be noted regarding safety and environmental proportionality approach are that:

  1. Proportionality is inherent to safety and environmental risk assessment (i.e. use of ALARP, BPEO, etc.);
  2. Proportionality is explicitly linked to risk;
  3. Multiple factors need to be considered when deciding a proportional approach;
  4. ASEMS is the mandated safety and environmental framework; therefore, the framework should be applied; it is not possible to develop a proportional approach that negates any part of ASEMS.

Waterfall Approach Process

The model that should be used to consider a proportional approach is intended to provide guidance and should only be used by competent safety and environmental practitioners.  A degree of judgement should be used when answering questions, particularly where a Product, System or Service may easily be classified in more than one category; this is why the use of competent safety and environmental practioners is required.

The waterfall approach model categorises Product, System or Service risk in accordance with factual questions, presented on the left of the diagram below, which are asked about the intended function and operation.  Each question should be used to define the cumulative potential risk, which may be presented by the Product, System or Service.  The Product, System or Service is categorised into one of three risk bands, which align to those defined in the Tolerability triangle, presented in the right of of the diagram.

During the process two initial questions are asked, where an answer of “yes” will automatically result in a categorisation of high risk, regardless of the answer to subsequent questions.  Further refinement is required for lower risk systems to ensure that the system risk is categorised appropriately.

Figure 1, Proportionality Waterfall Approach Model

The diagram above depicts the proportionality waterfall approach model used for the application of ASEMS.

Adherence to ASEMS is mandatory for DE&S.  As such, it is not possible to develop a proportional approach that negates any individual part of ASEMS and so the procedures described in ASEMS Part 2 – Instructions, Procedures and Support should be followed;  where proportionality may be applied is within each General Management Procedure, Safety Management Procedure or Environmental Management Procedure for the allocation of resource, time or effort.

Once the risk category has been established guidance is defined which prescribes the rigour which should be applied to the safety assessment process in terms of Process, Effort, Competence, Output, Assurance (PECOA):

  1. Process – the amount of dedicated/specific process, level of intervention in the organisational structure the Safety and Environmental Management System are established;
  2. Effort – How much time is afforded to the management of risk;
  3. Competence – the level of competence that is required to conducted appropriate assessment and management of safety and environmental;
  4. Output – The detail of evidence and reporting is cognisant to the level of risk;
  5. Assurance – The level of assurance required which shall be applied to the process.

Guidance for the application of PECOA is provided in the table below.  It should be noted that this is indicative guidance for illustrative purposes only. It is a fundamental requirement of ASEMS safety management principles that all safety decisions made should be reviewed, assessed and endorsed by a Safety and Environmental Management Committee to ensure that the Products, Systems and Services categorisation is correct. The diagram below shows the process that may be applied:

Proportionality Process

It should be remembered that using this low/medium and high categorisation could be misleading as the model takes no account of the population or rate of occurrence of the harm. A simple system that can only cause minor injury could still have a high degree of risk if there are lots of people exposed to the risk and the accident rate was high.  Moreover, acceptance of such a situation could lead to the development of an ineffective safety culture or the bypassing of safety mitigation procedures in order to avoid a high accident/minor injury position.  This is where the application of competent safety and environmental advice is essential to ensure that any proportionality model is not slavishly followed at the expense of proper rigour.   Where this model is useful is assisting those safety and environmental professionals to perform a preliminary assessment regarding what Products, Systems or Services are a priority for the allocation of resource, time or effort.

Stage One – System type and Life Cycle Phase

The first question is used to indicate, at a high level, the likely degree of risk for a project.  It should be noted that this is not a definitive assessment and that Products, Systems or Services could move within the model as the safety or environmental evidence is assessed.  There will be a degree of pre-existing assessment which accompanies a Product, System or Service and this may be used to assist with this initial question. 

The safety and environmental assessment process should be closely aligned with the Product, System or Service development process for newly developed Product, System or Services.  Where Products, Systems or Services are in the Concept, Assessment, Development or Manufacture phase of the CADMID/T cycle, they should be accompanied by a safety and environmental assessment process which utilises quantitative assessment techniques.

Where a Product, System or Service sits in the CADMID/T cycle should not influence the rigour of any safety or environmental argument; this model is provided to assist with any determination of the resource, time or effort that may be applied to the evidence to support the argument.  All Risk to Life should be ALARP, with no exception; what changes is the allocation of resources, time and effort to reach that judgement.

Those Products, Systems or Services where the expected worst credible consequence results in, at worst, a single minor injury should automatically be categorised as LOW risk and a qualitative approach may be adopted.

Commercial Off The Shelf or Military Off The Shelf systems should be accompanied by evidence which may be used in the safety and environmental assessment to demonstrate that they are acceptably safe and environmentally compliant, particularly where these are manufactured for use in the EU, where each Product, System or Service should demonstrate compliance with the applicable EU standards.  That the Product, System or Service is Commercial Off The Shelf or Military Off The Shelf is not, in itself, evidence.

Such evidence should include test evidence, trials evidence or a certificate of conformance.  Where a Commercial Off The Shelf or Military Off the Shelf system is already in the in-service phase and it is established that there is sufficient evidence to form a compelling safety argument that the Risk to Life is ALARP, then the system should be categorised as MEDIUM-LOW.  Where the system is also non-complex then it may be categorised as LOW.

Such Commercial Off The Shelf or Military Off the Shelf evidence should only be relied upon where it is established that this evidence is sufficient to demonstrate that the system is acceptably safe and environmentally compliant and already in existence.  The degree and appropriateness of evidence should be established by a Safety and Environmental Management Committee, with particular emphasis upon the quality of the evidence for high-risk systems.  This approach should be undertaken if the Product, System or Service in its entirety is categorised as Commercial Off The Shelf or Military Off the Shelf.  Where only sub-systems or components are Commercial Off The Shelf or Military Off the Shelf, the Product, System or Service should be categorised as bespoke and assessed accordingly.

Stage Two – Risk estimation and System Complexity

Any estimation of the risk that a Product, System or Service is likely to present should be used to further refine its categorisation.  If the worst credible consequence of a Product, System or Service is multiple fatalities then that Product, System or Service should automatically be categorised as HIGH risk.

If the worst credible consequence is a single fatality or multiple severe injuries then the system complexity should be considered further to refine and inform the categorisation.  Complex or novel system designs should have a higher degree of Suitably Qualified Experienced Personnel to conduct the safety and environmental assessment.  Accordingly, those Products Systems or Services which are complex and novel should also be categorised as HIGH whereas those exhibiting a lower degree of complexity might be categorised as MEDIUM.

Notwithstanding this, those Products, Systems or Services thatare in the Concept, Assessment, Development or Manufacture/Termination phase of the CADMID/T cycle should still be supported by a quantitative safety and environmental process.  The only exceptions are those Products, Systems or Services where the worst credible consequence is a single minor injury.  These should be categorised as LOW risk and may be supported by a qualitative safety and/or environmental process.

LOW risk Products, Systems or Services were the worst credible consequence is at worst a single minor injury should be categorised as LOW-MEDIUM risk where the design is complex or novel, those exhibiting a lower degree of complexity should be categorised as LOW risk.

Once the risk category has been established the rigour which should be applied to the safety assessment process in terms of Process, Effort, Competence, Output, Assurance (PECOA) should be defined.  This is summarised below:

Program ScaleLifecycle Stage
Small scale or no Critical FunctionCADMID/TCADMID/TCADMID/T
Large Scale Capital,

Critical Function or bespoke
CADMID/TCADMID/TCADMID/T
AssessmentHighMediumLow
ProcessA rigorous quantitative safety and environmental assessment process should be applied.Consideration should be given to the application of a qualitative safety and environmental assessment process.  Functional safety/environmental assessment may be required, if identified as a risk control measure.A qualitative safety and environmental assessment process should be appropriate for low risk, low complexity systems.
EffortSignificant effort should be expended developing the safety and environmental case.A medium level of effort should apportioned to development of the safety and environmental case, increasing for newly developed systems.A medium level of effort should be apportioned to development of the safety and environmental case.
CompetenceThe safety and environmental assessment and assurance programme should be led by individuals who are experts.  Remaining personnel should be at least Practitioners who should be provided with oversight where appropriate.Personnel engaged in the safety and environmental assessment and approval should be at least practitioners.Personnel engaged in the safety and environmental assessment and approval should be at least supervised practitioners who should be provided with oversight where appropriate.
OutputA safety and environmental case should be developed which includes a safety argument.  The safety assessment process should be substantiated by quantitative evidence.A safety and environmental case should be developed, which should include a safety and environmental argument for all by simplex low risk systems.  The safety assessment process should be substantiated by quantitative evidence for newly developed systems.A safety and environmental statement may be considered for systems, which are low risk and complexity.
AssuranceThe safety and environmental assessment should be independently assured.Independent assurance should be considered and applied to those projects which are considered to be novel or complex.  Assurance may be conducted at Committee level. Independent assurance is not required.
ASEMS GuidanceSafety and Environmental   Dedicated tailored and full implementation of all Clauses, articulated through adherence to all GMPs, SMPs and EMPs.Safety and Environmental   Apply full implementation of all Clauses, in line with guidance provided for the Functional safety/environmental assessment, as required, if identified as a risk control measure and application of GMPs, SMPs and EMPs.Where Project Teams have an overarching Safety and Environmental Management Systems in place:   Safety Gather sufficient evidence to support safety argument and document in a Safety Case/Assessment in accordance with SMP 04050609 and 12     Environmental Gather sufficient information in order to produce Environmental Impact Statement in accordance with EMP 07 – Environmental Reporting.

Process

The type of safety and environmental process which should be applied is dependent both upon the Product System or Service categorisation and the phase of the CADMID/T cycle that the project is in.  Newly developed MEDIUM-LOW to HIGH category Products, Systems or Services which are in the Concept, Assessment, Development or Manufacture phase of the cycle should have a quantitative safety and environmental assessment process applied, the depth and rigour of the assessment should be proportionate to its classification.  LOW risk Products, Systems or Services where the worst credible consequence is anticipated to be no greater than one minor injury may be assessed qualitatively.

A qualitative safety and environmental assessment process should be applied to Products, Systems or Services, which are in the In-Service, Disposal/Termination phase where it is deemed that there is sufficient evidence already in existence to demonstrate that it is acceptably safe.  In these circumstances a qualitative safety and environmental process should be applied to assess the in-service risks.

The approach uses a systematic and logical approach to categorise the resource, time and effort required to support any argument that a Product, System or Service is acceeptably safe or provides no significant damage to teh environment.  It also advocates the application of ASEMS in its entirety, prescribing the level of rigour, which should be applied in terms of process, effort, competence, output and assurance.

Effort

The effort apportioned to the safety and environmental process should be proportionate to the classification of the system.  A significant amount of rigour should be applied to those projects requiring quantitative assessment processes, particularly those with the highest degree of risk and complexity.

If a Product System or Service is assessed to be in a particularly low category and is simple it may not be necessary to undertake the full scope of risk management procedures.  In these circumstances a certificate of conformance may be sufficient, which may be supported by statement to that effect from the Safety and Environmental Management Committee.

All decisions made regarding the evidence required to justify a safety argument (regardless of risk) should be endorsed by a Safety and Environmental Management Committee.  If this is decision is delegated further for those Products, Systems or Services that are low risk is for the Duty Holder to determine as all decisions regarding to Risk to Life are made on their behalf.

Competence

The safety and environmental lead should be an expert for HIGH category projects or for MEDIUM category projects where the Product System or Service is particularly complex or a novel design.  The remaining personnel engaged on such projects should be at least practitioner level.  A competency assessment should be undertaken which should be endorsed by a Safety and Environmental Management Committee.

The safety and environmental lead for MEDIUM category projects should be at least practitioner level.  The remaining personnel engaged on such projects should be practitioner or supervised practitioner where appropriate supervision is in place.  A competency assessment should be undertaken which should be endorsed by a Safety and Environmental Management Committee.

The safety and environmental lead for LOW category projects should be at least practitioner level or a supervised practitioner with appropriate supervision in place.

Competency requirements relating to specific safety and environmental processes defined in ASEMS should be applied where those processes are undertaken.

Output

A safety and environmental case should be developed for HIGH category projects which includes a safety and environmental argument, developed using Claims Arguments Evidence (CAE) or Goal Structuring Notation (GSN).  The argument should be substantiated by quantitative evidence such as reliability data or the output from quantitative safety assessment processes.

A safety and environmental case should be developed for MEDIUM category projects which includes a CAE or GSN safety argument.  The quality and depth of evidence required to substantiate the safety and environmental argument should be proportionate to the classification of the Product System or Service.   Products, Systems or Services with increased complexity or higher degrees of risk should be substantiated by quantitative evidence

A Safety and environmental case should be developed for MEDIUM-LOW category Products, Systems or Services.  A safety and environmental argument should be included for those Products, Systems or Services which are particularly complex or novel or those which exhibit an increased degree of risk

A Safety and environmental case should be developed for MEDIUM-LOW category Products, Systems or Services.  A safety and environmental argument should be included for those Products, Systems or Services which are particularly complex or novel or those which exhibit an increased degree of risk.

A safety and environmental case or Safety and environmental statement should be developed for LOW category Products, Systems or Services.  A certificate of conformance may be adequate for the lowest risk simple Products, Systems or Services

All decisions made regarding the evidence required to justify a safety argument (regardless of risk) should be endorsed by a Safety and Environmental Management Committee.  If this is decision is delegated further for those Products, Systems or Services that are considered to fall in the low category, then it is for the Duty Holder to determine (as all decisions regarding to Risk to Life are made on their behalf) whether to acept the risks or not.

Assurance

HIGH and MEDIUM category projects should be independently reviewed by a Safety and Environmental Auditor.  The degree of Independent Safety and Environmental Auditor engagement should be proportionate to the project categorisation.

MEDIUM-LOW category projects should be independently reviewed by a Safety and Environmental Auditor where the safety and assessment processes applied are novel or complex.  Justification should be provided where an Independent Safety and Environmental Auditor is not appointed.

It is not necessary for projects categorised LOW to be independently reviewed.

It should be remembered that it is not prudent to take any form of autocratic system or approach without sufficient validation, verification and endorsement by competent and duly authorised individuals, who are considered Suitably Qualified and Experienced Personnel for the role.  Endorsement of decisions should be made by a competent panel or committee, as part of the overall hazard analysis and risk assessment and any variation in opinion from that presented by any proportionality model should be managed by such a panel.

If you found this post on Proportionality helpful, please leave a review.

If this post is missing something you wanted, please let me know!