ARCHIVED - Audit of Laboratory Services

This page has been archived.

Internal Audit Report
December 2010

Table of Contents

• Executive Summary
• 1.0 Introduction
  • 1.1 Background
  • 1.2 Risk Assessment
  • 1.3 Audit Objective and Scope
  • 1.4 Approach and Methodology
  • 1.5 Audit Criteria
  • 1.6 Statement of Assurance
• 2.0 Audit Opinion
• 3.0 Findings, Recommendations and Action Plan
  • 3.1 Management of Quality
  • 3.2 Safeguarding of Assets
• Appendix A – Divisions of the Science and Engineering Directorate and respective responsibilities
• Appendix B – Flowchart of Process for Receiving, Analysis and Shipping and Storage of Samples
• Appendix C – Audit Criteria
• Appendix D – Acronyms

---

Executive Summary

Background

The Canada Border Services Agency's (CBSA) Science and Engineering Directorate (SED) provides a broad range of scientific, engineering and advisory services. This audit focussed on the controls, procedures and guidelines for managing the quality of the services that the SED's laboratories provide. It also looked at client satisfaction and turnaround times for analyses, and at the Directorate's security regime for samples and other items that it was holding.

Significance of this Audit

The audit should be of interest to management, first because it looked at the Directorate's process for managing quality, i.e. its Quality Management System (QMS). The soundness of this system will be central to obtaining accreditation (i.e. Standards Council of Canada) for the SED's laboratory facilities. Second, the audit noted gaps with respect to the security and control of certain samples in the SED's custody. These gaps warrant management's attention because they represent non-compliance with security-related standards and guidelines from the Treasury Board Secretariat (TBS).

Objective and Scope

The audit objective was to provide assurance that controls were in place for managing quality and the safeguarding of assets. The audit scope included examining established quality-management procedures and guidelines, client satisfaction and the timeliness of service (turnaround times), the security of samples, and inventory control.

Audit Opinion

The management of quality was adequate, although the laboratory had not completed a documented Quality Management System. The security of samples and exhibits in the laboratory needed strengthening with respect to inventory control and compliance with TBS security policies.

Key Findings

Key elements associated with ensuring high-quality laboratory services were in place. These included mentoring and supervising junior staff, and continuously testing the proficiency of laboratory staff. However, the SED had not completed and implemented a fully documented QMS and had not established performance indicators for quality. Of note is that the SED must develop and implement a complete QMS as a prerequisite to having its laboratories accredited. Although the Directorate lacked a formal QMS, responses to client surveys indicated clearly that most clients (89%) were satisfied with the SED's laboratory services.

Regarding security, the audit found that external control of access to facilities was adequate. Inside the facilities, exhibits requiring a chain of custody, i.e. exhibits relating to prosecutions, were kept locked up. The audit noted, however, that staff was generally not familiar with government security standards, and that procedures for safeguarding other samples while in the laboratory's custody did not fully comply with security-related policies. As well, inventory control at the main facility needs strengthening. This would help the laboratory to better manage and locate samples, and to reduce both delays in analysis and the risk that samples could be misplaced or lost.

Statement of Assurance

The audit engagement was planned and conducted in accordance with the Internal Auditing Standards for the Government of Canada.

Management Response

The CBSA agrees with the findings and recommendations within this audit. The SED will continue efforts related to the development and implementation of the laboratories' QMS and inventory control systems. The SED is also fully committed to improving compliance with government security policies related to the safeguarding of samples.

1.0 Introduction

1.1 Background

The Canada Border services Agency's (CBSA) Audit Committee approved this Audit of Laboratory Services as part of the Three-year Risk-based Audit Plan for Fiscal Years 2009–10 to 2011–12.

The Science and Engineering Directorate (SED) is a key component of the CBSA's approach to managing the border and the commodities which cross that border. To this end, the SED provides a range of scientific, engineering, analytical, technical and advisory services. It works with other Canadian government departments, scientists, and the United States and other international partners. The scientific services that the SED provides support the health and safety of Canadians, national security and trade priorities.

The Directorate has three Divisions: 

• the Analytical and Forensic Services Division analyses samples and document exhibits;
• the Applied Research and Development Division conducts research with a view to developing border management solutions; and
• the Science and Engineering Support and Operations Division provides administrative, national maintenance services and project support.

Appendix A indicates the respective responsibilities of each Division. The SED has two laboratories: a main facility encompassing most operations, and a satellite location housing the Forensic Document Examination Section.

Clients of the SED send samples and document exhibits to the laboratory Analytical and Forensic Sciences Division for analysis. Regular clients are the CBSA Operations Branch, the Canada Revenue Agency and the Royal Canadian Mounted Police. Samples include taxable and dutiable items such as food items, tobacco and textile products or suspected contraband. The SED analyses them to verify that they are what they are purported to be. The laboratory also analyses documents to verify their authenticity.

The flowchart in Appendix B provides an overview of the processes for receiving, handling, shipping and storing samples and exhibits. The SED developed a long-term vision, Border Science: Going Beyond – 5‑Year Vision / Strategy, based on its 2007 consultations with stakeholders. Future directions include applying science and technology to the fullest extent possible to better manage the border, and strategic positioning to anticipate emerging border threats. The CBSA approved this vision and was working to implement it at the time of the audit.

1.2 Risk Assessment

The risk assessment conducted during the planning phase identified the following key risks:

Management of Quality

• There is a risk that the SED may not provide accurate, reliable and timely laboratory service to its clients, undermining the reputation of the CBSA.
• There is a risk that the absence of a documented Quality Management System may hinder the SED's efforts to become accredited, with implications for its perceived credibility. The ability to measure and report on performance and quality of service could be compromised.

Safeguarding of Assets (Stewardship)

• There is a risk that samples or document exhibits sent for analysis could be lost or misplaced, which could compromise prosecutions or tax cases, undermining the reputation of the CBSA.

1.3 Audit Objective and Scope

The audit objective was to provide assurance that controls were in place for managing quality and the safeguarding of assets. The audit scope included examining established quality-management procedures and guidelines, client satisfaction and the timeliness of service (turnaround times), the security of samples, and inventory control.

The audit was carried out in both SED laboratories in Ottawa. The audit team examined closed cases from fiscal year 2009–10 to determine average turnaround for completing the analyses requested. Open inventory-control testing included goods currently on hand within the SED, regardless of age.

The audit did not examine the CBSA's governance mechanisms with respect to science and technology. It offers no opinion, positive or negative, on the validity of scientific analysis methodologies, and it did not directly examine the work of the Applied Research and Development Division. As well, the SED main laboratory facility has multiple deficiencies, but this was not examined as management had already completed a review.

1.4 Approach and Methodology

The examination phase of this audit used the following approach:

• Reviewing and analyzing policies, procedures, guidelines, standards and reports.
• Conducting interviews with key stakeholders, specifically with managers and staff in the Science and Engineering Directorate of the Information, Science and Technology Branch.
• Analyzing laboratory procedures and assessing the management of quality.
• Inspecting laboratory sample storage facilities, obtaining floor plans, and assessing whether facilities met baseline security guidelines and requirements.
• Performing a walkthrough of procedures for receiving, handling and return shipping of samples to determine whether these procedures ensured control of access to samples and exhibits, chain of custody and the continuity of evidence.
• Determining average turnaround times (timeliness) of closed cases from fiscal year 2009–10.
• Surveying selected clients to assess whether they were satisfied and their needs had been met.
• Conducting an aging analysis of open inventory and reconciling goods on hand (log to shelf, and shelf to log) to assess whether inventory-control procedures were sufficient to prevent and detect the loss of samples.

1.5 Audit Criteria

Appendix C lists the audit lines of enquiry and criteria.

1.6 Statement of Assurance

The audit engagement was planned and conducted in accordance with the Internal Auditing Standards for the Government of Canada.

2.0 Audit Opinion

The management of quality was adequate, although the Science and Engineering Directorate had not completed a documented Quality Management System. The security of samples and exhibits in the laboratories needed strengthening with respect to inventory control and compliance with Treasury Board Secretariat security policies.

3.0 Findings, Recommendations and Action Plan

3.1 Management of Quality

Demonstrating a commitment to quality

According to the audit criterion for this area, senior management should be able to demonstrate that the organization is committed to quality in all its projects and activities. A formal, documented Quality Management System (QMS) is a key tool for managing quality, and the appropriate staff should clearly understand the system and follow it. A QMS is a documented set of Quality System Procedures and Quality System Instructions explaining general laboratory methodologies and detailing instructions for product analysis. A commitment to quality also includes the establishment of performance indicators and standards for quality, as well as clearly defined responsibilities for meeting these standards.

The audit found that the SED had been working to implement a QMS; however, this work was incomplete at the time of the audit. Two key reasons why the SED had not completed its QMS was that it did not have a formal project plan for doing so, and competing priorities, such as keeping up with the analytical work, prevented the SED from allocating sufficient resources to this project.

A documented QMS is a prerequisite to obtaining accreditation for the SED. Without accreditation, the SED's credibility could be affected. As well, without performance measures, the SED cannot measure and report on its performance.

It is important to note that despite the lack of a fully documented QMS, clients have consistently judged the quality of the SED's analytical services as being highly satisfactory. The "Client Satisfaction" section, below, discusses this finding in more detail.

Monitoring and supervision

Even though the SED has not implemented a formal QMS, audit work clearly indicates that the Directorate has instituted a number of measures aimed at monitoring the quality of the analytical work. For example, from 1998 to 2006 the SED implemented proficiency testing for the key areas of drug analysis, spirit and food testing, and document examination. All affected employees are subject to this testing on a rotating basis. Tests are scored and evaluated by third-parties who provide proficiency-testing services. This objective, independent process provides assurance on the quality of the analytical work and enables the SED to correct any weaknesses that testing may identify. To date, proficiency testing results have been good to excellent. Proficiency testing to verify the technical competencies of a laboratory is a best practice and is part of the accreditation process.

The laboratories also routinely conduct instrumentation checks to ensure that equipment is kept in good working order. Junior chemists are closely mentored and supervised for two to three years, and senior chemists consult with their peers and management as needed on complex cases.

Client Satisfaction

The audit criterion for this area requires the SED to understand the needs of its clients and to meet these needs.

The Directorate surveys client satisfaction annually. As noted earlier, clients indicated that they were highly satisfied with the quality of the analytical work. Our audit included a survey which confirmed that more than 89% of clients were satisfied with the SED's responsiveness, communication, accessibility and reliability. The only area for which satisfaction rates fell below 75% was timeliness ― i.e., the turnaround times to complete analytical work. Nevertheless, some 73% of clients were satisfied with this aspect of service, while 18% had "no opinion," and only 9% were dissatisfied.

The audit team noted that the SED compensates for timeliness issues by prioritizing analytical work on a case-by-case basis, depending upon urgency. This practice meets urgent client needs, but does not improve the overall turnaround times for fulfilling analytical requests. The audit acknowledges that various factors have contributed to the delays in turnaround. For example, workload has increased. As well, since 1976, the SED has been housed in a retrofitted warehouse. Numerous concerns with the building have previously been noted. These have included employee capacity (overcrowding), a leaking roof, mould growth, and ventilation deficiencies, which, taken together, have affected levels of service.

Recommendation

The Vice-President of the Information, Science and Technology Branch should prepare a formal project plan with expected timelines for implementing a Quality Management System that includes performance indicators for quality, timeliness targets for fulfilling analytical requests, and procedures and instructions covering laboratory methodology.

Management Action Plan	Expected Completion Date
The Science and Engineering Directorate (SED) will develop a project plan and implement a Quality Management System based on ISO 17025 within its SED during the course of 2011-12.	April 2012
The Quality Management System (QMS) will incorporate performance measures for quality, timeliness and procedures or instructions that are required for proper functioning and validation of the QMS.	April 2012 and ongoing

3.2 Safeguarding of Assets

The criterion for this line of enquiry requires the SED to ensure that it keeps the samples and documents in its custody secure, in accordance with all government and CBSA requirements. As well, it should have an adequate inventory-control system in place.

Physical Security

The audit team found that access to laboratory facilities was adequately controlled, with both locations protected by alarm systems. Inside the laboratories, samples and document exhibits requiring a chain of custody, such as those involving a criminal investigation, were kept locked up with restricted access. However, the security measures and controls for other samples, such as trade samples, did not comply with the Treasury Board Secretariat requirement to keep them locked up in accordance with the Operational Security Standard on Physical Security and the Guideline for Employees of the Government of Canada: Information Management (IM) Basics.

In interviews with laboratory staff, the audit team found that they were generally not familiar with government security standards and the appropriate level or classification of security for particular items. The audit also noted that the physical facilities themselves were contributing to security issues. The main laboratory facility is aging and does not meet requirements for space or security.

Inventory Controls

Inventory control and verification is a good management practice that helps in preventing and detecting loss. The audit found that inventory-control procedures for document exhibits at the satellite facility were satisfactory. However, at the main facility, the process for receiving and tracking samples while in the laboratory did not provide enough information on the exact whereabouts of particular samples. Specifically, the SED's main laboratory did not have an inventory-control system, and staff could not generate reliable reports on what samples were on hand. Staff stated that it can be difficult and time-consuming to locate samples for analysis.

The audit noted instances in which samples had been misplaced. If a sample were to be lost or misplaced, thus compromising a prosecution or tax case, the reputation of the Agency could be affected.

Disposal

The audit noted that laboratory policies did not discuss legislated disposal requirements and that laboratory employees may not all be aware of disposal requirements.

Recommendations

The Vice-President of the Information, Science and Technology Branch should review security and determine how best to ensure that staff understand and comply with all government security requirements.

The Vice-President of the Information, Science and Technology Branch should develop adequate inventory-control procedures, including disposal instructions for regulated commodities.

Management Action Plan	Expected Completion Date
The SED will review and disseminate all relevant government security requirements related to samples (assets) storage and inventory control.	June 2011
Inventory procedures will be re-examined and revised to ensure effective tracking and disposal.	June 2011

Appendix A – Divisions of the Science and Engineering Directorate and Respective Responsibilities

Division	Responsibilities
Analytical and Forensic Services Division	- Analysis of commodities, alcohol, precursor chemicals, suspected drugs and other suspected contraband; provision of technical information on composition and origin. - Scientific authority for the Canada Revenue Agency and the Royal Canadian Mounted Police for the authentication of alcohol and tobacco products for investigative purposes; for tobacco products the SED is the only government laboratory in Canada with this expertise (cost-recovery basis). - Forensic document examination and court testimony; most work is performed for the Canada Revenue Agency for taxation investigative purposes (cost-recovery basis). - Scientific, technical and advisory services and support.
Applied Research and Development Division	- Applied research and engineering to develop or assess emerging technological border management solutions, such as non-intrusive examination technologies, detection of chemical and biological materials, video surveillance, biometrics, mathematics and experimental computing. - Scientific support and assistance for the radiation detection portals installed at marine container facilities. - Scientific, technical and advisory services and support.
Science and Engineering Support and Operations Division	- Accountable for coordinating and directing the development and implementation of the SED 5-year vision / strategic action plan. - Advancing, coordinating and supporting efforts that focus on the strategic and operational components of providing laboratory services, solutions and advice. - Special projects (New CBSA Laboratory, Business Continuity Planning). - General administration, accommodations, finance, human resources, procurement, shipping and receiving, sample room management, continuity of evidence and short- to long-term storage of samples. - National maintenance of detection technology. - Operations and maintenance of the radio telecommunications program. - Library, performance measurement, health and safety.

Appendix B – Flowchart of Process for Receiving, Analysis and Shipping and Storage of Samples

Summary of the process:

• Receiving of samples:
  
  
  1. A client fills out applicable "Request for Analysis" form (Y15, Y15A, FACTS or Excise Form) for a sample which is prepared as per the laboratory instructions for shipping. The client sends the sample to the laboratory with the form attached to the outside of the evidence container or bag.
  2. An administrative services clerk receives the package, and compares goods received to information on the form. Once confirmed that the content of the evidence container or bag matches the information on the form, the administrative services clerk completes his part in a chain of custody form if needed and attaches to the package (i.e. seized alcohol and tobacco, suspected contraband, document exhibits).
  3. The information indicated on the form is entered into the Laboratory Analysis Support System (LASS) to generate the LASS number for the sample. If the sample received is a suspected contraband or urgent request, it is sent to the analysis unit. If not, the sample is stored in an appropriate location pending analysis.
• Analysis of samples:
  
  
  1. Each chemist has his/her own section in the laboratory. If the sample is a chain of custody controlled sample, it is locked up in an appropriate location.
  2. Once the analysis is complete, the result is entered into LASS and the final analysis report is generated in LASS. A paper and electronic copy of the final report is sent to the client. Unused portion of the sample is sent to shipping and storage.
• Shipping and storage of samples:
  
  
  1. The request for analysis is closed in LASS once a paper and electronic copy of the final report is sent to the client.
  2. Unused portion of the sample is prepared for shipping or for storage, depending on the type of the sample. If the sample is the chain of custody controlled sample, the unused portion is returned to the client. If not, it is stored in the laboratory for three years and destroyed.

This document is also available for download in PDF format (179 Kb)  [help with PDF files]

Note: For the purpose of this flowchart, references to "samples" include document exhibits and other evidentiary items sent for analysis.

Appendix C – Audit Criteria

Lines of Enquiry	Audit Criteria
Management of Quality (Adapted from ISO 9000 / 1400 – Quality management principles and the SED Quality System Manual)	Planning and commitment SED management is committed to quality at all steps and in all aspects of its analysis activities: A quality management system (policies, manuals, procedures, structures, etc.) is documented by SED management and communicated to, understood by and implemented by the appropriate staff member. Quality and performance measures are identified and established. Responsibilities and performance quality expectations to which managers and supervisors are held accountable are formally defined and clearly communicated. Steps are taken by management to ensure that the appropriate national or international quality standard accreditation will be achieved and maintained.
	Client Focus The SED meets client requirements and expectations as follows: User satisfaction measurement is conducted on an ongoing basis and is used to set improvement targets. Follow-up procedures for feedback received from stakeholders are managed in a timely fashion, and means are in place for client comments and feedback. Reports on results of the evaluation of client satisfaction are provided to senior management and acted upon accordingly.
	Monitoring and continuous improvement Effectiveness of controls is assessed by SED management and the results are communicated to those to whom it is accountable as follows: Responsibility for monitoring and updating performance measures is clear and communicated. Monitoring is conducted in a regular basis to identify quality control deficiencies and performance results are documented and reported to the required management level.
Safeguarding of Assets (Adapted from the Treasury Board policies, Core Management Controls for Stewardship, November 2007 and SED Quality System Manual)	Assets (samples) are protected as follows: CBSA (or equivalent) security guidelines / requirements (physical construction and zone) for the storage of samples are identified. These are applied by the SED, and these meet baseline governmental requirements. Internal security procedures (Quality System Manual and Quality System Procedures) including control of access, are in place to protect samples from external and internal threats (theft or loss) and these cover all aspects of the handling process (receiving, storage, transfer and shipping).
	Assets (samples) and records are periodically verified as follows: Inventory control is adequately robust, promotes loss prevention and enables loss to be detected through periodic inventory verifications.

Appendix D – Acronyms

CBSA	Canada Border Services Agency
SED	Science and Engineering Directorate
TBS	Treasury Board Secretariat
QMS	Quality Management System