Program evaluation respondents indicated that the monthly reporting system is simple and 80% felt that the follow-up study questionnaires were easy to complete. As only non-nominal, non-identifiable data is collected through the CPSP, 90% of those who reported a case did not hesitate to provide clinical information. Even with a total of 583 reported cases in 2003, the majority of participants (1,975 of 2,335, 84.6%) had 'nil' cases to report. The importance of zero reporting must however be re-emphasized. As studies come and go, the workload shifts to different subspecialties. The number of reported cases was higher this year due to the neonatal hyperbilirubinemia and vitamin D deficiency rickets studies and the inclusion of the early-onset eating disorders study to the program.
Figure 3 illustrates the number of cases reported by respondents in 2003. It shows that most participants (84.6%) had no cases to report and checked off the 'nothing to report' box each month. In fact, 11.1% of participants reported one case and 3.2% reported two or three cases. Only 25 participants (1.1%) completed four or more questionnaires. It is interesting to note that 125 of the 583 reported cases were duplicates, validating CPSP ascertainment. The CPSP is extremely grateful
that the majority of participants faithfully complete the detailed questionnaires subsequent to reporting cases. This demonstrates that they appreciate the enormous value of the scientific data collected. On their part, the Steering Committee continues to insist on short, precise and pertinent detailed questionnaires.
The CPSP is available as an inexpensive tool to survey participants on a one-time-basis in order to identify the prevalence of a problem or to answer a specific question. Once approved by the CPSP Steering Committee, the one-time survey question is sent to all participants with a monthly initial reporting form. Once collected, results are forwarded to the investigator for data analysis.
Results of the 2003 one-time survey question on lap-belt syndrome are found on page 49.
TABLE 2 |
|
---|---|
Rarity | Disorders of such low incidence or prevalence that national ascertainment of cases is needed (less than 1,000 cases a year). |
Public health importance | Clearly addressing a public or paediatric health issue. |
Scientific importance | Demonstrated scientific interest and importance. |
Uniqueness | Proposal must demonstrate a clear need for data on a condition or disorder for which there is only limited information and for which surveillance is the most appropriate means of collecting the data. |
Quality of proposal | Proposal must state clear and achievable objectives, practicability, patient confidentiality, adequate resources, clear questionnaire and method of evaluation. |
Workload of paediatricians | Steering Committee must be convinced that reporting will not make excessive additional demands on the workload of paediatricians. |
Priority will be given to diseases that are not currently notifiable or, if notifiable, have sufficient indication of under-notification. Investigators are expected to demonstrate that potential funding is available. |
The CPSP can offer investigators the use of a timely, active surveillance system to increase awareness of childhood disorders that are high in disability, morbidity, mortality and economic costs to society, despite their low frequency. The CPSP provides an innovative means of identifying and obtaining data on low-frequency diseases and conditions from approximately 2,335 participants. The program is committed to a case ascertainment rate of over 90% and boasts a high response rate of 96% on detailed reports (Table 1), due to follow-up reminders to participants who have not responded. The CPSP offers an opportunity for international collaboration with other paediatric surveillance
units worldwide and a chance to make a difference in the health and well-being of Canadian children and youth.
Individual researchers are encouraged to submit proposals for new studies once they have reviewed the Criteria considered for inclusion of studies
TABLE 3 Format for submission |
---|
Proposals for new studies should include:
|
(Table 2) and the Format for submission (Table 3). The Steering Committee reviews submissions at its spring and fall meetings, giving preference to studies that have either strong public health importance or could not be undertaken any other way. Studies must receive ethical approval and have funding in place before final acceptance to the program.
The CPSP is pleased to see established faculty members mentoring young researchers with their study proposals.
As previously mentioned in the Overview section, the CPSP is available to investigators as a cost-effective tool to survey participants on a one-time basis in order to identify the prevalence of a problem or to answer a specific question.
TABLE 4 CPSP studies timeline (by end date) |
||||
---|---|---|---|---|
Studies | Start date | End date | Total confirmed cases to December 31, 2003 | |
Group B strep | January 1996 | December 1996 | 178 | |
Neural tube defects | January 1997 | December 1998 | 107 | |
Creutzfeldt-Jakob disease | January 1997 | June 1999 | 1 | |
Hemorrhagic disease of the newborn | January 1997 | December 2000 | 6 | |
Subacute sclerosing panencephalitis | January 1997 | December 2000 | 3 | |
Cerebral edema in diabetic ketoacidosis | July 1999 | June 2001 | 23 | |
Progressive intellectual and neurological deterioration | July 1999 | June 2001 | 59 | |
Anaphylaxis | January 2000 | June 2001 | 732 | |
Hemolytic uremic syndrome | April 2000 | March 2002 | 140 | |
Smith-Lemli-Opitz syndrome | January 2000 | December 2002 | 35 | |
Hepatitis C virus infection | February 2001 | January 2003 | 58 | |
Neonatal liver failure/perinatal hemochromatosis | February 2001 | January 2003 | 10 | |
Necrotizing fasciitis | September 2001 | August 2003 | 37 | |
Neonatal herpes simplex virus infection | October 2000 | September 2003 | 58 | |
Neonatal hyperbilirubinemia - severe | July 2002 | June 2004 | 203 | |
Vitamin D deficiency rickets | July 2002 | June 2004 | 69 | |
CHARGE association/syndrome | September 2001 | August 2004 | 90 | |
Acute flaccid paralysis | January 1996 | December 2004 | 370 | |
Congenital rubella syndrome | January 1996 | December 2004 | 9 | |
Prader-Willi syndrome | January 2003 | December 2004 | 31 | |
Osteogenesis imperfecta | January 2004 | December 2004 | N/A | |
Early-onset eating disorders | March 2003 | February 2005 | 63 | |
Lap-belt syndrome | September 2003 | August 2005 | 3 | |
Adverse drug reactions - serious and life-threatening | January 2004 | December 2005 | N/A | |
Severe combined immunodeficiency | April 2004 | March 2006 | N/A | |
Acquired demyelinating syndromes of the central nervous system | April 2004 | March 2007 | N/A | |
Acute rheumatic fever | April 2004 | March 2007 | N/A |
Preamble
The CPSP evaluation process began early in 2003 with the creation of an Expert Advisory Group (EAG). Individual members were selected based on their experience and expertise in the fields of public health, paediatrics, epidemiology, surveillance and administration. The CPSP Evaluation Working Group held a preparatory meeting in May with the chair and circulated extensive background materials to all EAG members prior to a face-to-face meeting in September, followed by in-camera deliberations. The chair of the EAG presented the following final report to the CPSP Steering Committee in November 2003.
Members of the Expert Advisory Group
Dr. Robert McMurtry (Chair), Former Dean of Medicine, The
University of Western Ontario; Former Assistant Deputy Minister of
Health
Dr. Margaret Berry, Neonatologist, The Montreal Children's
Hospital
Dr. Jeffrey Davis, Chief Medical Officer, Wisconsin Division of
Public Health
Dr. Philippe Duclos, Project Leader, Immunization Safety, World
Health Organization
Dr. Monika Naus, Epidemiologist, BC Centre for Disease Control;
Chair, National Advisory Committee on Immunization
The EAG was unanimous in their opinion that the CPSP represents excellent value for money. The achievement in this respect was seen as excellent and unsurpassed by any comparable program known to the EAG. The CPSP was seen as representing an important collaborative tool for surveillance, research and policy development. In this role, it was perceived as unique in Canada. In other words, it provides an important activity that would disappear in its absence, unless a much larger investment is made to replace it.
The core activity of surveillance of low-frequency, high-impact conditions affecting children has created a network that reaches into all parts of Canada. This not only generates crucial information of these conditions ("they are on target"), but it has also established a mechanism to provide important public health information quickly and inexpensively on a national basis. Examples include the work on hemorrhagic disease of the newborn, confirming the Canadian recommendation of vitamin K as the gold standard for prevention, and on baby walker injuries, confirming the need for a commercial product safety ban on these devices.
The EAG was impressed by the survey of clinicians (paediatricians) that affirmed a change in practice pattern by some, and a high degree of recognition by all. The publications generated by the program also received accolades. The CPSP is encouraged to increase its reach to include nurse practitioners and northern communities and territories.
Finally, the EAG underlined the importance of providing more evidence of impact on public health policy and clinical practice. Annual evaluation of the effectiveness of the Steering Committee was also recommended.
The CPSP has done well in regards to its current objectives. It has initiated programs of national scientific significance and developed an effective surveillance system to monitor the health of Canadian children in relationship to low-frequency, high-impact conditions. Nonetheless, there may be an advantage to rewording the program objectives to reflect emerging priorities and new realities (e.g., changes in federal leadership, positive changes in federal/provincial/territorial relations).
Some specific wording for the program objectives was suggested as follows:
The EAG commended the CPSP on performing its core function so well and emphasized that important additional roles, such as responding to public health emergencies and international collaboration, may require additional resources.
The evaluation process was seen as exemplary, and the EAG was impressed with the surveys of the four stakeholder groups and the CDC (Centers for Disease Control and Prevention) framework. The provided integrated background material was well
done and contained both quantitative and qualitative information of value. The logic frameworks provided an interesting context. However, the program goals were not seen as serving CPSP well and could be deleted without ill consequence.
The case in support of the excellent value for money represented by the CPSP might be strengthened, especially in view of the new federal fiscal reality that will likely be similar to the Program Review of 1994-95. The EAG is convinced that the case can be made and, furthermore, an effort to duplicate the essential work of the program by another means would be considerably more expensive.
The events of 2003 have been characterized by large-scale change and high impacts. All provinces east of Alberta held elections this year with new governments being elected in Ontario, Quebec, and Newfoundland and Labrador. Most observers feel that together with the change in federal leadership, a more collaborative approach at federal/provincial/territorial forums can be anticipated. In addition, a significantly negative economic impact was felt from SARS and the case of one animal with BSE (bovine spongiform encephalopathy). Both were low-frequency, high-impact events and, accordingly, both of these latter developments underscore the importance of public health and the crucial importance of surveillance.
In the reviewers' opinion, the asset that the CPSP represents is relevant to these realities. It is a national program and an important mechanism for surveillance of human health as observed in the health and well-being of one of the most vulnerable populations in Canada, our children.