How Does Quality Assurance Audit an EASA Compliant Safety Management System (SMS)?

Sofema Online (SOL) www.sofemaonline.com considers the challenge of performing a QA audit of an SMS system

Introduction

SMS is a business system just like any other, so as part of our Compliance Auditing obligations we will be looking at the following elements as suitable for our auditing activities.

1. Management
2. Documentation
3. Competence
4. Training

What Does SMS Look at?

Essentially the focus of SMS is not in fact compliance it is on Managing Risk and the Purpose of an SMS Audit is to consider Risk. So Safety management goes far beyond traditional compliance led approaches to auditing.

The SMS approach was developed following the identification of shortfalls in the actual process of auditing – you can only identify a discrepancy during a Quality Audit when you have evidence.

What is a Safety Audit?

So a Safety Management System “Audit” aims to identify potential safety risks and Hazards.

At the heart of any effective Safety Management System is a formal Risk Management process that identifies hazards and assesses and mitigates risk.

The next step is to mitigate or manage any risks identified to an acceptable or tolerable level. An essential aspect is the management of risk generated by a Contract or Sub Contracted organization particularly where this may directly impact the organization.

Let’s do some Myth Busting!

a) SMS is just an extension of the Quality System?

Answer – It's not! – Safety Auditing Is very different in fact Safety Auditing is almost holistic in that it is looking at the potential exposures, faced by the Organisational system.

b) SMS is owned by the SAFETY Manager?

Answer – It's not! – The SMS System is owned by the Accountable Manager but each Post Holder is responsible for the effective delivery of the SMS within their business area. Sometimes called a Top-Down Approach.

Additional Key Elements of an effective SMS – which can be subjected to Compliance Audit

(a) Safety Policy and Objectives;

(b) Safety Risk Management;

(c) Safety Assurance;

(d) Safety Promotion.

Each of the above elements is critically important and all must be visible and active within the SMS System.

Key Personnel in an effective SMS

Whist the individual business owners are responsible for safety management an SMS champion is essential for the health of the SMS system. (This person may be the Safety Manager or he may be the Quality manager or another person, the important point is to ensure effective management of the entire process).

Final Note regarding Compliance Auditing

Any audit subject must be referenced against a documented audit criteria (or standard). Compliance Audits are in fact prescriptive – means essentially they are acceptable or not-acceptable (non-conformity)

Let’s consider the purpose of the evidence?

Essentially it is required to convince the Business Area Owner / Nominated Person that the finding is valid. We cannot impose findings we must be able to demonstrate the validity of the finding based on objective evidence.

How much “evidence” is required?

Taking into consideration

The time factor (time is precious) / sufficient evidence for the auditor to form an opinion/ The need to demonstrate to the Auditee the existence of the issue.

Can we trust the evidence?

Evidence that can be considered trustworthy which essentially means it is accurate, credible and where the integrity of the evidence has not been compromised.

Another measure of the effectiveness of the evidence would be its repeatability – will the next auditor see the same issue?

Note that Evidence should be free from any bias (e.g. the auditors' preconceived ideas)

Greater reliance can be placed on evidence which emanates from independent sources

Can we validate the evidence?

Evidence that can be confirmed by cross-checking with other evidence is considered objective

Evidence should accurately show connection and or engagement with the functioning of a system, or part of a system, operated by the auditee.

Factors to consider when judging the quality and quantity of audit evidence

The purpose for which the evidence will be used for example - internal/external or in support of a contract – as part of an accident or incident investigation (in general, the higher the level of audit importance, the higher the standard of evidence that is required)

The same could be said in respect of the potential for legal action, controversy or unexpected issues related to the audit findings driving a need for a higher standard of evidence.

Pay attention to the following considerations

1. It is the responsibility of the auditor as part of the audit preparation to determine the methods that will provide the best quality of evidence for the particular audit.
2. Information in support of the audit may come from a myriad of sources including databases, documents, procedures, processes, instructions, previous audit reports, inspection reports, management reviews, organisational and planning documents.
3. Auditors must determine the reliability of data that is significant to the audit questions by review and corroboration, and by testing the auditee's internal controls over information, including general and application controls over computer-processed data. Provided by third parties
4. The degree to which information may be used as audit evidence depends on the extent to which its quality can be established and its significance in relation to the audit findings.
5. The auditor must be aware that a risk exists that his/her presence may distort or prejudice what would normally occur, thus reducing the quality of the evidence. (do you agree?)

Information was gathered from people through interviews and focus groups. Such information may take the form of written or oral statements.

Oral evidence is generally important in performance audits, as information obtained in this manner is up-to-date and may not be available elsewhere. (However, information should be corroborated and statements confirmed if they are being used as evidence.)

Next Steps

Please see www.sofemaonline.com or email team@sassofia.com

To view course details check here - https://sassofia.com/course/easa-compliant-aviation-quality-assurance-senior-and-lead-auditor-course-5-days/