Government of Canada
Symbol of the Government of Canada

Audit of the Implementation of the Meat Compliance Verification System (CVS)

Audit Report
Approved: December 4, 2008


Table of Contents


Executive Summary

Introduction

The Audit of the Implementation of the Meat Compliance Verification System (CVS) was undertaken following the Audit of the Food Safety Enhancement Program (FSEP). A number of areas for improvement were noted in the FSEP Audit. CVS was identified as one of the means to address the FSEP audit recommendations.

The CVS was developed to improve the efficiency and uniformity of the inspection approach to verify registered establishment compliance with regulations. To that end, the CVS sets out the verification tasks to be used by CFIA inspection staff to assess compliance with regulatory requirements.

Audit Objective

The objectives of this audit were to provide assurance that CFIA's Compliance Verification System is designed to comply with legislation and has been implemented as prescribed.

CVS Design Features

Over the course of the audit, Internal Audit identified that the design of the Compliance Verification System:

  • Had a stated objective;
  • Was overseen by senior management and had an implementation plan;
  • Included clear procedures;
  • Referenced regulatory requirements;
  • Defined roles and responsibilities within the CVS;
  • Included a training program and a strategy to monitor the effectiveness of training;
  • Included an information systems development plan; and
  • Included system-ability to track Corrective Action Requests.

Audit Opinion

Based on the work performed in the context of the defined audit criteria, Internal Audit has concluded that the management controls over the Compliance Verification System are generally well designed. Two areas for improvement were noted: the finalization of the performance monitoring strategy and clarification of roles and responsibilities related to quality management activities at the regional, area, and national levels.

In this internal audit report, we provide a high, though not absolute, level of assurance to management regarding our findings and conclusions. We do this by designing and applying procedures so that in our professional judgement, the risk of an inappropriate conclusion is reduced to a low level. Use of the term 'high level of assurance' refers to the highest reasonable level of assurance an auditor can provide concerning a subject matter. Absolute assurance is not attainable as a result of factors such as the use of judgement, the use of testing, the inherent limitations of control, and the fact that much of the evidence available to the auditor is persuasive rather than conclusive in nature.

Audit Recommendations

Internal Audit developed two recommendations to strengthen the management control design of the Compliance Verification System, based on the work performed to assess the defined audit criteria:

Recommendation #1

Operations Branch should expedite the completion of the CVS performance monitoring strategy and criteria. The strategy should include a comprehensive communication plan for the performance information.

Recommendation #2

Operations Branch, and in particular the CVS and QMS Management, should clarify the roles and responsibilities in meat inspection Quality Management at the Regional, Area and National levels.

Recommendation for Future Audit

The purpose of this audit was to assess and make recommendations related to aspects of the design and implementation of CVS. The newness of the implementation of the CVS and the limited timeframe and resources available to conduct audit meant that detailed compliance testing was not included in the scope of this audit.

A future audit with a broader scope could include:

  • Compliance with the CVS policy in the Manual of Procedures;
  • The completeness of the CVS as it relates to the FSEP Manual and the Health of Animals Regulations, Consumer Packaging and Labelling Regulations, and Food and Drug Regulations;
  • The effectiveness of follow-up actions on Corrective Action Requests (CARs);
  • The effectiveness of the training and quality management activities;
  • The adequacy of resources to conduct compliance and enforcement activities in the meat program;
  • The efficiency of updates to the Manual of Procedures; and/or
  • The effectiveness and efficiency of external communications with industry.

Introduction

History of Meat Inspection in Canada

Three driving forces influencing the direction of the Canadian meat inspection program over the past 25 years are:

  • The shift from "traditional" organoleptic inspection, as it did not adequately address the food safety concerns represented by contamination from pathogenic micro-organisms and chemical residues. This led to the promotion, by government, to industry of HACCP based quality control programs and their subsequent adoption by industry under the Food Safety Enhancement Program (FSEP).
  • The need to maintain an equivalency status with respect to production standards and inspection procedures to that of the major importer of Canadian meat products, namely the United States.  This was necessary for the economic health of the meat industry, which is a major exporter of meat and meat products and important to the Canadian economy.
  • The need to ensure equitable resourcing with respect to meat inspection across the various CFIA areas and regions, and the recognition that inspection of meat processing establishments and the meat processing areas of slaughter establishments represented the only resourcing flexibility available. This is because inspection procedures and resource requirements on the slaughter line are governed by fixed, internationally recognized standards.

In both Canada and the United States, a number of technological, scientific and quality management developments have occurred in the past quarter century that have had a major impact on the ability of the meat inspection services to react to the forces imposed upon them.

Three of the most important developments are:

  • Inexpensive informatics technology and the increasing improvement in electronic communications have made the development, implementation and management of complex information systems possible in real time.
  • Increasing sophistication of the science of risk analysis and statistical process control procedures and their adaptation to food safety quality control programs.
  • Emphasis on preventative quality control procedures, rather than end product testing, as a means of controlling quality in the production process and generating economic benefits.

The recognition of these developments by Canada and its trading partners and their incorporation into the international standards such as Codex Alimentarius, that govern the production and inspection of food products, have provided an environment that resulted in the development of meat inspection programs such as the Compliance Verification System (CVS) in Canada and the Performance Based Inspection System in the US

Although there have been several individual programs developed and implemented over the past 25 years that impacted the evolution of meat inspection programs, the most recent ones from which the CVS program is derived are the "The Inspection Program" (TIP) and the Food Safety Enhancement Program (FSEP). 

CVS represents a merging of the MCAP and FSEP programs into a single inspection program.  TIP represented a task based inspection program carried out by the in-plant inspector. After an establishment profile (e.g. type of production, size and complexity of operation) was created, a task list of inspection activities was generated to ensure comprehensive coverage of the regulatory requirements that must be inspected for the type of operation. The inspectors in the establishments carried out the inspection tasks generated by the TIP. The tasks were based on requirements set out in regulations and the Meat Hygiene Manual of Procedures.

FSEP is intended to ensure that the conditions under which food products are manufactured and the ingredients used in their manufacture lead to the production of safe food. Since 2005 it has been mandatory for meat processing establishments to develop and operate under HACCP, an internationally recognized tool for adapting traditional inspection methods to a modern, science-based, food safety system. Under HACCP, industry must monitor and verify manufacturing processes on an ongoing basis, maintain records of their HACCP System and update the HACCP system regularly. Within a plant, the HACCP system was verified and recognized by the CFIA via the FSEP program.  In particular, the establishment was monitored by the in-plant inspector and was routinely audited (approximately every 3 months) by an CFIA FSEP Specialist.

With TIP and FSEP as separate programs, issues arose such as duplication and inefficiency. It became evident that improvements were needed, especially with the delivery of TIP and that a single program needed to evolve from the two programs.

The CVS program was created as an amalgamation of the two programs previously mentioned.

Description of CVS

Overview

The Meat Inspection Regulations were updated in November 2005 to put in force the requirement that all federally registered meat establishments develop, implement and maintain a Hazard Analysis and Critical Control Point (HACCP) system. The HACCP system must comply with the requirements of the CFIA's Food Safety Enhancement Program (FSEP).

The adoption of mandatory HACCP was the starting point of the Agency's evolution from its previous inspection system into the Compliance Verification System (CVS). Under the design of CVS, the Agency's inspection programs (FSEP audits and traditional inspection) were merged.

The CVS includes verification tasks that are used by CFIA inspection staff to assess compliance with regulatory requirements. Each verification task includes detailed procedures for inspectors to follow when conducting verifications.

Compliance is normally to be achieved through a cooperative approach between the operator and the inspection staff. This approach generally involves the operator correcting instances of non-compliance through the development and implementation of action plans. When this cooperative approach is not successful, or when the operator is unwilling or unable to correct instances of non-compliance, the CFIA pursues the enforcement options outlined in the Meat Hygiene Manual of Procedures.

Organization, Frequencies, Tasks and Establishment Profiles

Presently, CVS is organized into five sections of verification tasks, as described in Table 1, below.

Table 1:  Organization of Verification Tasks
Section Sub-Sections
1. Food Safety
  1. Critical Controls Points
  2. Prerequisite Programs
  3. Slaughter
  4. Food Safety Current Issues
2. Non Food Safety
  1. Labelling Practices
  2. CFIA Stations and Facilities
  3. Non Food Safety Current Issues
3. Export
  1. Export USA
  2. Export Other than USA
  3. Foreign Country Follow-up Requirements
  4. Export Current Issues
4. HACCP System Design and Reassessment
  1. HACCP System Design and Reassessment
  2. HACCP System Design Current Issues
5. Animal Welfare and Animal Health
  1. Animal Welfare and Animal Health
  2. Animal Welfare and Animal Health Current Issues

Each verification task is assigned an identifying number and a minimum national frequency. These frequencies are determined by considering the:

  • Impact on food safety
  • FSEP Manual guidelines
  • Number of Critical Control Points at the establishment
  • Regulatory requirements
  • Export requirements
  • State of compliance of the industry as a whole

The Verification Task Procedures are described in the Meat Hygiene Manual of Procedures. Each task includes the following information:

  • Section and sub-section for the task
  • Task number
  • Task title that summarizes the task
  • National frequency
  • The date the task was last revised
  • References to the Meat Inspection Act, Meat Inspection Regulations, and other applicable regulations
  • The procedure to follow when completing each task, including what requirements are being assessed by each task; how the inspector has to assess the operator's compliance to requirements; and references to the Manual of Procedures

An Establishment Task Profile is completed by the inspector by selecting the verification tasks that are applicable to the establishment. The profile is reviewed and updated as necessary on an annual basis at the same time the operator's license is renewed. The profile is also completed by inspectors when:

  • A new establishment begins operation
  • Operations change in an establishment
  • Program requirements change which impact the establishment
  • Verification tasks are amended which impact the establishment

Based on the profile of the establishment, a Task Tracking Table is prepared. This document is used by inspection staff to track the delivery of the verification tasks at each establishment. The Task Tracking Table includes:

  • The establishment number
  • The task name, number and frequency
  • Applicable rating or code for each completed task
  • Total number of applicable tasks
  • Average number of tasks to be completed per day or week, as applicable

The Verification Task Process

The Verification Task Process is a seven step process:

  1. Preparation for inspection
  2. Gather information to determine compliance
  3. Assign compliance level
  4. Communicate results and action required
  5. Assess operator's action plan
  6. Follow-up
  7. File maintenance

The process is depicted graphically in Exhibit 1.

Exhibit 1 – The Compliance Verification System - Inspector's Role

Click on image for larger view
The Compliance Verification System - Inspector’s Role
The Compliance Verification System - Inspector’s Role

Authority for the Audit

The Audit of the Implementation of the CVS was conducted with the approval of the CFIA Audit Committee.

The audit was undertaken following the Audit of the Food Safety Enhancement Program (FSEP). A number of areas for improvement were noted in the FSEP Audit. CVS was identified as one of the means to address the FSEP audit recommendations.

Audit Objective and Scope

The objectives of this audit were to provide assurance that CFIA's Compliance Verification System (CVS) is designed to comply with legislation and has been implemented as prescribed.

The scope of the audit was limited to CVS in slaughter and processed meat establishments. Documentation review and interviews were limited to the Ontario and Western Areas.

The audit scope was limited to only Chapter 18 – Compliance Verification System of the Meat Hygiene Manual of Procedures and the Meat Inspection Regulations.

The audit scope did not include an assessment of the completeness of the CVS as it relates to the FSEP Manual and the Health of Animals Regulations, Consumer Packaging and Labelling Regulations, and Food and Drug Regulations.

The period under review was January 2006 to September 30, 2008.

Audit Approach

The audit was initiated in mid-September 2008 and examination ended in mid-November 2008.

The audit assessed the management control design in the CVS. As CVS was only recently implemented and there was a limited timeframe in which to conduct the audit, detailed compliance testing was not included in the scope of the audit. The audit work performed to date will form part of the background to the larger follow-up audit. The recommendations in this report, in part, serve to ensure that actions are taken by management to develop the information required for assessment in the next audit.

The approach used to address the audit objectives included the development of audit criteria against which observations, assessments and conclusions were drawn. The audit criteria developed for this audit are included in Section 2 – Audit Findings of this report. The effectiveness of the management controls of CVS in operation will be assessed at the time of the next follow-up audit.

The main audit techniques used in the conduct of the audit were:

  • Examination of key documents related to the design of CVS
  • Interviews with the CVS team, and management from Policy and Programs Branch (PPB) and Operations Branch
  • A review of the Meat Inspection Regulations and related CVS tasks

Audit Opinion

In this internal audit report, we provide a high, though not absolute, level of assurance to management regarding our findings and conclusions. We do this by designing and applying procedures so that in our professional judgement, the risk of an inappropriate conclusion is reduced to a low level. Use of the term 'high level of assurance' refers to the highest reasonable level of assurance an auditor can provide concerning a subject matter. Absolute assurance is not attainable as a result of factors such as the use of judgement, the use of testing, the inherent limitations of control, and the fact that much of the evidence available to the auditor is persuasive rather than conclusive in nature.


2. Audit Findings

Based on a combination of the evidence gathered through documentation examination, analysis and interviews, each of the audit criteria was assessed by the audit team and a conclusion for each audit criteria was determined. While the design of CVS was found to be generally well developed, Internal Audit identified areas where the CVS management controls can be improved.  Details of these observations are located in the next section.

For each criterion, evidence was gathered and the audit team assessed if the criterion was substantially met, partially met or not met. 

Table 1 – Summary of Audit Findings
Audit Criteria Substantially Met Partially Met Not Met

Clear objectives for the CVS are established.

 

 

Senior management was actively involved in the oversight of the CVS pilot, approval and implementation.

 

 

Clear CVS procedures were developed and communicated.

 

 

The design of CVS references the Meat Inspection Regulations.

 

 

A clear plan for the implementation of CVS was developed.

 

 

A performance monitoring strategy for CVS was developed.

 

 

The principles of Quality Management are embedded into the design of CVS. A process is designed into CVS to allow for improvements based on monitoring and ad hoc suggestions.

 

 

Roles and responsibilities within CVS are clearly defined and communicated.

 

 

The training is delivered to all those who need it. Management has in place a strategy to monitor the effectiveness of CVS training.

 

 

An information systems development and monitoring plan for CVS was created.

 

 

Corrective actions are tracked in a system at the national level.

 

 

More explanation of those criteria rated as "Partially Met" is provided in the following sections of the report. The criteria rated as "Substantially Met" are not described any further as there was sufficient evidence to conclude that those criteria were achieved.

Performance Monitoring Strategy

Criterion

A performance monitoring strategy for CVS was developed.

Procedures and Findings

Internal Audit reviewed key literature and assessed the draft quarterly reporting strategy for the CVS.

During the CVS pilot project, a number of indicators were established and the results were published as part of the Pilot Study Report.

During the first six months post-CVS implementation, ad hoc reports were available to be produced from the current CVS database.

A plan is in place to develop standardized reports that will be produced on a regular basis. Based on the draft quarterly reporting strategy, the standardized reports may include:

  • CVS Tasks Planned versus CVS Tasks Delivered – Highlights where inspector workloads require adjustment.
  • Corrective Action Requests (CARs) Not Closed by the Specified Completion Date – Highlights CARs that are not closed by the due date so that follow-up work can be done to ensure that the CARs are closed or determine the reasons why they are not closed.
  • CARs Issued versus Tasks Assigned – Report organized by region with individual establishment information. Highlights establishments with an excessive number of CARs issued so that a review can be performed to determine if there are underlying issues in the establishment.
  • CVS Visits by Shift – Highlights operation shift coverage to ensure proportional inspection coverage.
  • Task Ratings by Task Section – Summarizes task ratings (i.e. Acceptable or Unacceptable) and is organized by task section (i.e. Food Safety, Non Food Safety, Export, HACCP System Design and Reassessment, and Animal Welfare/Animal Health). Highlights where issues may exist and provides information to adjust task frequencies and allocation of resources, based on risk.

The quarterly reporting strategy will be used to communicate CVS results to Area Operations Committees.

Risk Assessment

Reliable and timely information for management decision-making is critical to ensure the proper control of the Meat Program and CVS.

Recognizing that progress has been made to develop a regular reporting cycle, there is an opportunity to finalize the reporting strategy and begin issuing reports so that any deficiencies noted can be promptly addressed.

Recommendation #1

Operations Branch should expedite the completion of the CVS performance monitoring strategy and criteria. The strategy should include a comprehensive communication plan for the performance information.

Continuous Improvement and Quality Management

Criterion

A process is designed into the CVS to allow for improvements based on monitoring and ad hoc suggestions. The principles of quality management are embedded into the design of CVS.

Procedures and Findings

Internal Audit reviewed the Meat Hygiene Manual of Procedures and the Quality Management System Manual, and conducted interviews with key staff.

The Meat Hygiene Manual of Procedures – Chapter 18 – The Compliance Verification System includes a procedure to amend the verification tasks outlined in Annex A to the chapter. Specifically, all levels of staff working with CVS have the ability to complete a Verification Task Comment Submission Form whenever they identify the need for a change or update in the system. The form itself is available to all employees through the Agency's E-Forms system.

As per Chapter 18, the distribution path for comments depends on who initiates it:

  • Forms initiated by Field staff are distributed from Field staff to the Area CVS Coordinator to the National CVS Coordinator
  • Forms initiated by Programs Network staff are distributed from Program Network staff to the National Program Chief to the National CVS Coordinator

Comments submitted are reviewed by Area CVS Coordinators and Area Program Specialists and proposed revisions, additions or deletions to the CVS are submitted to the National CVS Coordinator for review and acceptance. Changes are to be made in consultation with National Program Chiefs. In addition, changes that coincide with amendments to the Manual of Procedures and/or Acts and Regulations are to be communicated via Manual of Procedure Directives. Inspectors are then required to update their manuals accordingly.

Interviewees were positive about this process, indicating that it was an effective and formalized means for all CVS users to have ownership of the design of the system and to leverage the knowledge and experience of field staff.

In 2004-2005, the Agency initiated a Quality Management System (QMS). The objective was to "result in the continuous improvement of the consistency and overall quality of the regulatory service, and incident management activities delivered by Agency staff."

Nine core QMS activities, including macro and micro steps and criteria, were identified that relate to the meat CVS. These QMS activities address the areas of:

  • CVS Meat Import Product Inspection
  • CVS Meat Export Product Inspection
  • CVS Meat Slaughter Domestic Inspection, Beef
  • CVS Meat Slaughter Domestic Inspection, Hogs
  • CVS Meat Slaughter Domestic Inspection, Horses
  • CVS Meat Slaughter Domestic Inspection, Other
  • CVS Meat Slaughter Domestic Inspection, Poultry and Rabbits
  • CVS Meat Slaughter Verification (Section 5 Tasks, Animal Welfare and Animal Health)
  • CVS Meat Establishment and Slaughter Verification (Section 1, 2, and 3 Tasks)

As part of the Meat Hygiene Manual of Procedures – Chapter 18 – The Compliance Verification System, all inspection supervisors are required to complete QMS verification activities. In slaughter establishments, QMS activities are the responsibility of the Veterinarian-In-Charge and the Regional Veterinarian Officer. In processing establishments, QMS verification activities are the responsibility of the Complex Supervisor.

The other Quality Management responsibilities noted in Chapter 18 are:

  • National Program Chief – Propose revisions, additions or deletions to the QMS verification criteria as required when amendments are made to the MIA, MIR, MOP, FSEP Manual, and other applicable acts and regulations. Forward proposed revisions to the National CVS Coordinator for review and acceptance. Ensure that directives amending the MIA, MIR, MOP or FSEP Manual coincides with the release of any required changes to the QMS criteria.
  • National CVS Coordinator – Accept or suggest revisions to the QMS verification criteria received from the Program Chiefs or Area CVS Coordinators.

Risk Assessment

QMS activities are described within the Meat Hygiene Manual of Procedures - Chapter 18 - The Compliance Verification System at the supervisory level, but not at the Regional, Area or National levels. Roles and responsibilities related to quality management at these levels are not explicit in the context of meat inspection. The potential risk of not formally clarifying these responsibilities is that they will not be appropriately funded, recognized, or completed.

Recommendation #2

Operations Branch, and in particular the CVS and QMS Management, should clarify the roles and responsibilities in meat inspection Quality Management at the Regional, Area and National levels.

Recommendation for Future Audit

The purpose of this audit was to assess and make recommendations related to aspects of the design and implementation of CVS. The newness of the implementation of the CVS and the limited timeframe and resources available to conduct audit meant that detailed compliance testing could not be performed.

A future audit with a broader scope could include:

  • Compliance with the CVS policy in the Manual of Procedures;
  • The completeness of the CVS as it relates to the FSEP Manual and the Health of Animals Regulations, Consumer Packaging and Labelling Regulations, and Food and Drug Regulations;
  • The effectiveness of follow-up actions on Corrective Action Requests (CARs);
  • The effectiveness of the training and quality management activities;
  • The adequacy of resources to conduct compliance and enforcement activities in the meat program;
  • The efficiency of updates to the Manual of Procedures; and/or
  • The effectiveness and efficiency of external communications with industry.

Appendix A – List of Acronyms Used

  • CAR - Corrective Action Request
  • CVS - Compliance Verification System
  • FDA - Food and Drugs Act
  • FSEP - Food Safety Enhancement Program
  • HACCP - Hazard Analysis and Critical Control Point
  • MIA - Meat Inspection Act
  • MIR - Meat Inspection Regulations
  • MOP - Manual of Procedures
  • PPB - Policy and Programs Branch
  • QMS - Quality Management System
  • TIP - The Inspection Program

Management Action Plan as at September 2009

1. Audit Recommendation: Operations Branch should expedite the completion of the CVS performance monitoring strategy and criteria. The strategy should include a comprehensive communication plan for the performance information.

Proposed Management Actions Responsible Person(s) Implementation Date

1) Operations Branch is in process of developing a CVS performance monitoring strategy including the reporting of program delivery and compliance information.  Reporting frameworks are being finalized for incorporation into management processes.

National Inspection Manager

Implemented

2) A communication plan is being developed in association with the reporting framework to ensure senior management in both Operations Branch and Program and Policy Branch have access to performance information and utilize this information in program management and delivery.

National Inspection Manager and Executive Director, Operations Strategy and Delivery

Implemented

2. Audit Recommendation: Operations Branch, and in particular the CVS and QMS Management, should clarify the roles and responsibilities in meat inspection Quality Management at the Regional, Area and National levels.

Proposed Management Actions Responsible Person(s) Implementation Date

1) Operations Branch will reconfirm QMS responsibilities at each level of the Branch.

National Quality Manager and Manager, National Operations

Implemented

2) Operations Branch will continue to embed the QMS into CVS procedures.

National Quality Manager and National Inspection Manager

Implemented