Instructions for Use
Advertising Campaign Evaluation Tool
Introduction
To improve the Government of Canada’s ability to consistently evaluate major advertising campaigns, and to assist departments in complying with a key requirement in the Communications Policy (http://publiservice.tbs-sct.gc.ca/pubs_pol/sipubs/comm/siglist_e.asp and http://publiservice.tbs-sct.gc.ca/pubs_pol/sipubs/comm/cpgcp-ppcgc_e.asp), an Advertising Campaign Evaluation Tool (ACET) for use in telephone surveys has been developed in partnership with an interdepartmental committee and the Government Advertising Committee.
The Interdepartmental ACET Committee includes the public opinion research coordinators from those departments with historically-higher advertising expenses, including Health, Human Resources Development, National Defence, Environment, Natural Resources, Citizenship and Immigration, Foreign Affairs and International Trade, Canadian Heritage and Public Works and Government Services.
When to use
Since fall 2002, all departments are required by the Government Advertising Committee to conduct a quantitative ACET evaluation of major advertising campaigns exceeding $400K in media buy, as well as any other campaigns of this magnitude which are horizontal in nature (i.e., involving more than one department). Despite the emphasis on >$400K campaigns, departments are encouraged to evaluate all campaigns (of whatever magnitude).
As a guideline, the survey fieldwork period should begin in the days following the end of the campaigns. Otherwise, if it begins during the campaign, the results will need to be correctly reported depending on the survey period in relation to the campaign.
Note that the ACET is intended to be a post-campaign assessment, and its use is not mandatory for more periodic assessments not tied to specific campaigns.
How to use
The ACET (including demographic questions strongly suggested to facilitate comparisons) should not be altered in any way, including the wording, order and coding of questions. No questions must be deleted, and questions must be added only at the end, after the core questions have been asked. The core questions must be the first part of the survey. No questions must come before them. The ACET evaluation must not be undertaken as part of an omnibus survey.
Media consumption questions (e.g., "In the past week, have you read a newspaper?") will be asked at the conclusion of the ACET questionnaire, so as not to contaminate responses to the question "Where did you notice this ad?"
For general population studies, it was agreed to adopt a minimum N=1,000 national sample. A larger sample is desirable for a significant regional representation. All campaign evaluations must be on general population, except where special conditions apply.
In the interest of consistent reporting and analysis, top-box scores from the 7-point scale will be 5, 6 and 7, and low-box scores 1, 2 and 3.
Departments are reminded to contact their Public Works and Government Services Canada public opinion research advisor prior to contacting a research supplier, as directed in the Communications Policy. The advisor will assist you in setting research objectives, selecting an appropriate methodology and supplier, and will assign a registration number prior to the project being contracted by Public Works and Government Services Canada on your behalf.
Reporting Results
Final reports should be submitted to Co-ordination and Advisory Services – Public Opinion Research Directorate at Public Works and Government Services Canada, as directed in the Communications Policy, no more than three months (ideally one month) following completion of survey fieldwork. The actual English and French questionnaire used and tables and the data file (in SPSS format) must also be submitted to Public Works and Government Services Canada, as must the following information:
- short description of campaign and its objective(s) measured by the evaluation;
- evaluation methodology used, sampling, and timing of ACET evaluation vis-à-vis campaign;
- media buy (costs by media and, if available, by region; coverage and duration (GRP)).
More detailed information on each campaign, including any public opinion research used in preparation, are also required for the
Advertising Management Information
System (AdMIS).
Sampling frames which screen-in only the specific target audience of the advertising campaign (as opposed to a random sample of the general public) provide different evaluation results than general population studies. For the purposes of analysis, two ‘classes’ of studies will be created – one general public and one specific-screened. Data sets from mixed frames – i.e., those that combine a general public sample with an oversampling of specific-screened -, should clearly separate results for each.
Results of post-campaign ACET evaluations will be analysed periodically, and those of each individual campaign, will be shared with all departments and agencies.
Further Information
Additional guidelines will be developed as necessary, and will be posted on this site once complete. For clarification on any of the above, or to discuss the evaluation component of a particular campaign, kindly contact your Public Works and Government Services Canada Public Opinion Research Advisor directly, or the Research and Analysis section of the Privy Council Office Communications and Consultation Secretariat at: 957-5486.
|
