Considerations Related to the Assessment of Effective EASA Organisational Compliance

Sofema Online (SOL) www.sofemaonline.com considers audit best practices.

With the exception of EASA Part 21 Subpart J Design Organisation Approval which as you will be aware is managed directly by EASA.

The process for all other approvals (Air Carrier – Part 145 / Part M / Part 147 / Part 21 Subpart G POE / ATO) is managed by the Competent Authority.

Essentially works as follows:

Achieving Regulatory Approval (Simplified)

1/ The organisation demonstrates to the Competent Authority (CA) that they satisfy the pre-requisites to be granted the requested approval

Documentation / Facilities / Manpower / Competence / Finance / Oversight

2/ The Accountable Manager of the Approved Organisation signs a statement to confirm his or her responsibility related to ensuring sufficient finance is available to maintain full compliance

3/ The Organisation must set up a process (Independent Quality Assurance) to ensure continuous compliance

4/ The CA periodically assess the organisation for continuous compliance and raises findings for any non – compliance (Level 1 or 2 as appropriate)

Challenges related to the Effective Assessment of a given organisation

1/ When we consider compliance audits whilst it may be satisfactory the reality is that it is unknown if the compliance will be satisfactory next week or next month

2/ A shortfall of the compliance audit is that it does not always test either the process or procedures which are developed to deliver the requirement

(As a result, some assumption is typically made on behalf of the auditor.) Whilst often this does not become an issue it remains a weakness

3/ Compliance Auditing can be misleading depending on the depth of the audit (deeper audits take more time)

4/ When there is a limited time available we should maximise the effectiveness of the criteria – how to do this

Focus on the ability of the organisation to assess it self so an audit which focuses on the quality system 70% - 80% and physical business area audit 20% - 30%.

Use the audit of the Quality Assurance System to understand how effective is the organisations External / Internal audit process.

5/ Who is performing the internal audit? / what is their audit competence – how is this evidenced?

6/ What is the audit standard that is being assessed – how is it documented / how deep is the assessment

Consider the following statement

i/ The organisation shall have a process to ensure training is provided in respect of XYZ

Audit Outcome – the process exists therefore it is acceptable

Please now Consider a Different Scenario

ii/ The organisation shall have a process to ensure training is provided in respect of XYZ

Comments related to the above - When asked to demonstrate how they were measuring the effectiveness of the training – the organisation was not able to provide an adequate response.

5/ Whilst people are receiving training, the relevant question is does it meet both the objectives and specific needs of the organisation (How is this evidenced)

Real-Life Example of A Shortfall

A Group of 20 staff who had completed an internal SMS course were not able to explain the difference between a hazard and a risk – when pressed it was quickly established that there was much confusion

As a result, this group had essentially received “tick the box” training, and the company passes the audit – but is it ok ?.......clearly not!

Taking Steps to improve our Audit Engagement Process

What should we look for in addition to our basic Competence Assessment -  So in general an audit oversight process that looks deeper at the following will be more effective?

How the process is being managed – Means who is managing it and are they training and competent for the task – How is this evidenced?

How the effectiveness of the process is being assessed – Who is measuring the effectiveness of the process & what is the measure employed – Is it successful?

How the assessment process is documented – Where is it a document – Consistent Processes need to be written down so that they are understood and shared by all.

Where is the physical evidence that assessment has taken place?

Are there examples where a shortfall in the process has been identified and how a corrective action was initiated which is shown to be successful?

Next Steps

Please see www.sassofia.com and www.sofemaonline.com or email team@sassofia.com