How to Measure the Effectiveness of a Compliant Quality System

Measuring the effectiveness of your company's quality system should be an essential element of the management of your overall quality management system strategy.

The process starts with a review of the organizational QMS policy together with a detailed gap analysis to assess full compliance with all regulatory obligations.

Having established regulatory compliance a further measure for assessing the strength of QMS is to employ a formal “internal” compliance audit process which will be set against an expected organisational standard.

The effectiveness of any audit is of course heavily dependent on the competency of the auditor, however, to also consider that any external regulatory-driven audit should not show many discrepancies if our “in-house” system is doing a good job.

The number of findings across all audits is also a “coarse” measure of effectiveness.

However, note that it is not just numbers that matter it is a trend showing a reducing level of findings, as well as confirmation of effective closure of the finding. (root cause plays a big part here)

A detailed assessment of the audit program showing a wide program including multiple systems and product audits. (little and often always being more productive)

Another measure of the effectiveness of the QMS relates to the willingness of the employees to engage with and report into the system.

Many employees are lacking in knowledge regarding the purpose of the Quality Assurance System and are even suspicious of its function.

Fully engaged employees will work with the Quality System and provide strength and resilience, much however will depend on the culture of the organisation as well as the top-down leadership.

An inspection of the formal management review will also provide indications regarding the effectiveness of the QMS Typical contents of the management review will include a summary of all internal and external audits with an overview of the status of major findings.

How the organisation has dealt with the major findings provide a key measure that can be utilised as a part of the assessment?

The Engagement of the QMS with external agencies is a key measure, in particular with respect to the management of any Mandatory Occurrence Reports (MOR’s) or any other mandatory reports, in each case satisfactory closure is a key indicator to ensure in each case the appropriate action has been taken.

1/ A review of the previous years' audit system in its entirety is a starting point

a)   How many audits were performed?
b)   How many findings were identified?
c)   What were the major findings?
d)   Were all findings closed out within acceptable time frames?

The simple number are not alone a valid indicator but provide a starting point that facilitates analysis of the available data. A Proactive SMS relies on the amount of data, so too does an effective Quality System.

Dealing with the major findings in a pragmatic way is a leading indicator of an effective Quality System.

Late closing and unclosed findings give rise to concern regarding the effectiveness of the Quality System.

2/ Review the Minutes of the last Annual Meeting

a)   Who attended?
b)   What was on the Agenda?
c)   What action items were raised?
d)   Have all action items been addressed?

The ability of the organization to give visibility to any issues identified and to generate as an action is a measure of the effectiveness of the system.
d) To carry out a review that all identified items have been addressed provides confidence in the management of the Quality System.

3/ A review of the status of the current Quality System

a)   How does this year's system compare with last years?
b)   Is the program on schedule
c)   Does the Quality System have sufficient resources

Notes - How does the quality system measure its resource requirement?
     • Does the Quality System require a manpower plan for the quality system?
           »  Determining the specific need is a measure of the effectiveness of the Quality System.

4/ How does the organization manage competence?

a)   Within the Quality System
b)   Within the organization

Without an effective Quality Audit capability, the organization will not be able to identify issues that impact its effectiveness and ability to ensure full regulatory compliance

Next Steps

Please see www.sassofia.com and www.sofemaonline.com or email team@sassofia.com