Measuring Safety Performance of Aviation Safety Reporting Systems
Considering ICAO Definition of Safety
ICAO Annex 19 defines safety as ‘the state in which risks associated with aviation activities, related to, or in direct support of the operation of aircraft, are reduced and controlled to an acceptable level’ and safety performance as ‘a service provider’s safety achievement as defined by its safety performance targets and safety performance indicators’.
Challenges to Establish Confidence in Safety Performance?
Whilst it is usual for safety metrics to focus on serious incidents and accidents it is also possible to gain a false impression regarding the overall level of safety due in part to a system with a low number of high consequence negative outcomes, means the low frequency of such outcomes may give the wrong impression that your system is safe.
Considering final outcomes in isolation does not reveal the presence of systemic factors, hazards or latent conditions that have a potential to result in high consequence negative outcomes.
It is typically not possible to determine a direct relationship between specific parameters or safety actions and the final, aggregate safety outcome. Hence, the absolute measurement of safety is itself not achievable.
Continuous Safety requires a robust system to enable the maintenance of safety in relation to known risks and to be prepared to cope with those risks that are not yet known.
Finally, to be able to compensate for the negative drift over time of risk controls.
How to Measure Safety Performance
Essentially when the resulting output cannot be directly measured, the underlying systems and processes need to be validated instead.
Effective safety management requires both understanding and strong management of your system and processes. Selecting safety performance indicators that consider relevant indicators which ensure the capabilities for safety management can be properly evaluated and improved.
The management of safety relies on the capabilities of your organization to systematically anticipate, monitor and further develop your organizational performance to ensure safe outcomes of your activities.
This implies that you will need to measure performance at all levels of your organization by adopting a broad set of indicators involving key aspects of your system, and operations and allowing to measure those key aspects in different ways.
The Safety Bonus
Effective safety performance measurement supports the identification of opportunities for improvement not only related to safety, but also to efficiency and capacity.
Safety Performance Indicators (SPI)
ICAO defines safety performance indicator as ‘a data-based safety parameter used for monitoring and assessing performance’ and safety performance target as ‘the planned or intended objective for safety performance indicator(s) over a given period.’
Safety performance indicators (SPIs) can be ‘classified’ in accordance with specific features.
Lagging indicator - Metrics that measure safety events that have already occurred including those unwanted safety events we are trying to prevent.
Lagging indicators are mainly used for long-term trending, either at a high level or for specific occurrence types or locations. Because they measure safety outcomes, they can be
used to assess the effectiveness of safety measures, actions, or initiatives and are a way of validating the safety performance of the system.
Leading indicator - Metrics that provide information on the current situation that may affect future Performance. Leading indicators, which are particularly relevant from a management perspective.
Leading Indicators may be used to influence safety management priorities and the determination of actions for safety improvement. You may use this type of indicator to proactively develop (‘drive’) your company’s safety management capabilities, in particular during initial implementation of SMS. This may entail the setting of performance targets.
Sofema Aviation Services www.sassofia.com and our sister company SofemaOnline www.sofemaonline.com provide classroom and online training for regulatory and vocational training fully compliant with EASA requirements. For more information please email office@sassofia.com or online@sassofia.com
