FAA – SMS Safety Performance Monitoring Considerations

Sofema Online (SOL) www.sofemaonline.com considers the process to manage safety performance within the organisations Safety Management System (SMS).

Introduction – Safety Performance and Safety Performance Monitoring

Safety Performance is a balance between the safety system achievements of the organisation, and the effectiveness of the mitigations which are implemented to enable the safety achievements.

SMS General Understanding of Terms:

Safety Policy – A clear statement of the organization’s policy, management principles.

Monitoring and Evaluation:

» Conducting reviews or audits of the SMS processes.
» Applying conventional quality assurance principles.
» Ensuring that remedial actions have been implemented as planned and that the SMS remains effective and relevant to the operation.

Note: Ensure appropriate reports are made to the Accountable Manager to enable management review.

What is a Safety Performance Indicator?

A safety performance indicator (SPI) is defined in the ICAO Safety Management Manual as a measure (or metric) used to express the level of safety performance achieved in a system.

» Generally expressed in terms of the frequency of occurrences of some event causing harm, e.g., A number of events / year or / 000’s of Hrs of operation.
» A set of indicators should also include leading indicators.

Note: These indicators do not require a safety event to take place and are metrics that provide information on the current situation that may affect future performance.

How to Drive Safety Performance within our Organisation?

Step 1 Base Line Identification.

Requires the leadership team to identify the baseline metrics to be set to monitor the goals and objectives based on:

» The Organisations Data.
» Industry Best Practice.
» Regulatory Requirements.

Step 2 – Oversight:

To ensure the collection & processing of data in accordance with Step 1.

To validate on a continuous basis including following Step 3 interventions.

Step 3 – Activation of mitigations to drive the safety improvements.

The Basics of Safety Performance Monitoring

Typically performed by reference to 3 essential elements:

» Developing safety objectives. (Safety objectives represent the organisation’s priorities regarding the management of safety risk.)
» Safety objectives – may be:

o Process-oriented (expected system behaviours),
o Outcome–orientated (containment of incidents / accidents)
o Related to Specific Interventions

Example – To take all necessary measures to minimize any adverse safety events during base maintenance activities / Ground Operations – Through the introduction of additional safety measures.

» Utilizing Safety Performance indicators. (SPIs)
» To demonstrate achievement of our safety performance targets (SPTs).
» SPT Examples:

o During the next 12 months, SMS Training will be delivered to a minimum of 90% of the workforce by the provision of 3 Safety Management System Training Courses.
o During the next 12 months to experience less than 0.55 reportable events / 10,000 ramp Maintenance Service Operations

What is the main difference between Safety Performance Indicators (SPI) and Safety Performance Targets (SPT)?

ICAO defines:

» Safety Performance Indicators as “A data-based parameter used for monitoring and assessing safety performance”.|
» Safety Performance Targets as “The planned or intended objective for safety performance indicator(s) over a given period”.

SPI/SPT Development Considerations

» Specific understanding regarding the selection of the relevant safety concern to be treated to ensure both validity & relevance.
» Avoid complexity wherever possible – simple is better and can clearly reflect relevant safety concerns.
» Use clear definitions to avoid ambiguity.
» Phased approach to implementation – achieve the ultimate objective in stages.
» Avoid high workload analysis.
» Ensure a broad coverage is important do not only pursue extremes (to capture systemic exposure that can agglomerate into more significant events.
» Make sure time frames are clear and understood (data capture rate).
» Avoid quantity over quality measure what is important, not easy.

Leading & Lagging SPIs and SPTs

» Lagging SPIs/SPTs – sometimes known as “Outcome SPIs/SPTs” – reflect negative outcomes:

o Low frequency of high severity negative outcomes (aggregation nationally may be necessary for meaningful analyses). – example runway excursion accidents/10 000 landings.
o SPIs/SPTs for lower severity/higher probability (typically considered also as “precursor” events.) – example – The number of non-stable approaches/1 000 landings.

» Leading SPIs/SPTs – sometimes known as “Process SPIs/SPTs” – measure situations which have the potential to become or contribute to high severity/low probability negative outcomes:

o Leading SPIs support proactive development of the organisation’s safety management System – Capability to improve safety exposure.
o SPI – 0.75 events of foreign object damage FOD) to aircraft per 10,000 ramp service delivery operations.
o SPT: To reduce to 0.5 events of foreign object damage to aircraft per 10 000 ramp service delivery operations with 12 months from the introduction.

Sofema Aviation Services www.sassofia.com & Sofema Online www.sofemaonline.com offer Safety Management System Training as Classroom, Webinar or Online. Please see the websites or email team@sassofia.com