Interim Evaluation of the National Immunization Strategy

April 2003 to June 2007

Cat.: HP5-79/1-2009E-PDF ISBN: 978-1-100-11692-1

PDF Version* (Full Document) (28 pages, 309 KB)

Executive Summary

This report presents the results of the Phase 1 interim evaluation of the National Immunization Strategy (NIS), implemented from May to July 2007. The specific objectives of the evaluation are as follows:

• measure progress towards achievement of short-term outcomes;
• identify areas of future improvement;
• inform future decisions regarding immunization program planning, design and implementation;
• ensure accountability; and
• provide evidence to support decision making on improvements to the NIS.

The evaluation involved both qualitative and quantitative methods. For the first, interviews were conducted with key informants¹ to obtain their views on the relevance, design and delivery, and success of the NIS so far. Similar interview questions were used in an on-line survey of local health care providers contacted through an e-mail invitation to P/T networks of health professionals. The quantitative component of the evaluation aimed to assess the progress the NIS has made in achieving expected outcomes and impacts, and it was guided by several Indicators of Success:

• number of new immunization programs launched or expanded;
• amount of money spent by P/Ts on four new vaccines² before and after receipt of new federal funding;
• level of vaccine coverage for the four publicly funded vaccines before and after funding;
• incidence of the four vaccine preventable diseases before and after funding.

The necessary data, which will be used as a baseline for future evaluations, were obtained from an extensive review of relevant documents, as well as interviews with members of the (then) Immunization and Respiratory Infections Division (IRID), Public Health Agency of Canada (PHAC).

According to the responses of 48 key informants and 145 on-line surveys, the NIS continues to be relevant at both federal and P/T levels, and there is no identified need at present to revise or realign its overall goals and objectives. Four gaps identified by respondents were lack of public health research, less than optimal coordination between the Canadian Immunization Committee (CIC) and the National Advisory Committee on Immunization (NACI), insufficient attention to special populations (mobile populations, immigrants, Aboriginal peoples) and lack of sustainable funding models. The design and implementation of the NIS were considered effective overall, but further thought might be given to certain aspects: the strain on the resources of smaller jurisdictions involved in many working groups; the need for working groups to be informed about each other's activities; the greater role that NGOs would be willing to undertake on working groups; the need for working group budgets to allow activities to move forward without undue reliance on PHAC staff; and the desire for a long-term NIS vision.

The quantitative evaluation revealed that progress has been made in several key areas. The structure of the NIS with its many working groups and F/P/T representation has contributed to flourishing partnerships and collaborations on immunization issues. With federal funding, all P/Ts have now introduced programs for the four vaccines, a significant increase in coverage for three of those vaccines has been achieved since 2004, and the incidence of IPD, pertussis and IMD has decreased since that time.

Recommendations

The evaluation team suggested a number of recommendations that might improve the performance of the NIS further:

1. Develop a monitoring system for select key areas and indicators using the Performance Measurement Framework³ (PMF) in order to build on current knowledge, establish baselines and prepare for future NIS evaluation.
2. Enhance communications between the CIC and NACI.
3. Increase the interaction between the NIS working groups.
4. Consider additional roles for NGOs on working groups.
5. Allocate specific resources directly to working groups.
6. Continue to develop and focus on a long-term vision.

Action plan

The Centre for Immunization and Respiratory Infectious Diseases (previously IRID) has developed a Management Response Action Plan that outlines the steps needed to address the recommendations of this interim evaluation. The Plan includes implementation of the PMF, development of sustainable funding options, review of CIC and NACI terms of reference, consideration of funding mechanisms for a coordinated research program, and improvement of national surveillance strategies.

 

Footnotes

¹ A broad representation of federal/provincial/territorial (F/P/T) jurisdictions and non-government organizations (NGOs) represented on the Canadian Immunization Committee (CIC) and its working groups

² Vaccines against invasive pneumococcal disease (IPD), pertussis, varicella and invasive meningococcal disease (IMD)

³ The PMF lists the indicators and data sources needed to measure desired outcomes and impacts.

 

For more information, or if the above information and documents are not accessible to you, please contact:

Estelle Arseneault, Publication coordinator
Public Health Agency of Canada
Centre for Immunization and Respiratory Infectious Diseases
Immunization Division
A.L. 5902A, Rm 30, 380 Hunt Club
380 Hunt Club Road, Ottawa, Ontario K1A 0K9

Tel: (613) 998-8862
Fax: (613) 998-8934
E-mail: estelle.arseneault@phac-aspc.gc.ca

* How to download PDF documents.

---

Prepared by the Program Monitoring and Evaluation Unit, Centre for Immunization and Respiratory Infectious Diseases, Public Health Agency of Canada

Qualitative assessment and interviews with key informants pPreformed and prepared by Goss Gilroy Inc., Management Consultants

Original submitted September 2007. Approved July 2008

Published by authority of the Minister of Health