Evaluation of the 2007-2008 International Youth Program's Advertising Campaign

Advertising Evaluation
Executive Summary
March 2008

Prepared for:
Foreign Affairs and International Trade Canada
Lester B. Pearson Building
125 Sussex Drive
2nd floor
Ottawa (Ontario) K1A 0G2
kevin.chappell@international.gc.ca

Contract #:08170-070470/001/CY

Table of Contents

• Executive Summary
  • Background
  • Research Objectives
  • Methodology
  • Questionnaire Development
  • Notes on Reading This Report
  • Key Findings

Executive Summary

Background

Foreign Affairs and International Trade Canada oversees agreements that permit Canadian students to work or travel abroad, and for foreign students to come to Canada. The IYP advertising campaign took place from January 2008 to February, 2008. The campaign's primary objectives were to:

• Increase general knowledge and awareness of the IYP by 15%;
• Increase participation in IYP by 10%; and,
• Achieve an overall IYP advertising recall of 25%.

Research Objectives

The objectives of this research were to establish:

• A baseline of awareness of the IYP programs among youth aged 15 to 35 prior to the prior to the campaign launch;
• Recall of the International Youth Program advertisements, including unaided recall, aided recall, where the respondents saw the advertisement, awareness of who is responsible for creating the ads, action taken as a result of the advertisements including contact with the 1-877 number and the website www.international.gc.ca/experience (among the adult general public to fulfill the Department's obligations under the guidelines for assessing Government of Canada advertising); and,
• Post-campaign awareness of both the advertising and the program among the target audience (youth aged 15 to 35).

Methodology

In order to achieve the research objectives, Ipsos-Reid conducted three phases of research.

• The first phase consisted of a 5-minute telephone survey using Computer Assisted Telephone Interviewing (CATI) conducted among n=500 youth aged 15 to 35. This phase was designed to establish baseline awareness among the target audience pre- campaign. The field window extended from January 8^th to January 14^th 2008.
• The second phase of research consisted of a 10-minute survey using Computer Assisted Telephone Interviewing (CATI) conducted among n=500 youth aged 15 to 35. This phase established a post-campaign awareness of both the advertising and the program among the target audience. The field window extended from February 25^th to March 6^th 2008.
• The third phase, which ran concurrently to the second phase, consisted of a 10-minute survey using Computer Assisted Telephone Interviewing (CATI) conducted among n=1,000 adult Canadians. This phase fulfills the Department's obligations under the guidelines for assessing Government of Canada advertising. The field window extended from February 25^th to March 6^th 2008.

Questionnaire Development

The study required the development of three different questionnaires, one for each of the phases. The third phase questionnaire included the mandatory Government's Advertising Campaign Evaluation Tool (ACET) and attempted to gauge:

• Unaided and aided recall of the ads;
• Awareness and use of information channels in the ads; and,
• Perceived messages of the ads, as well as the perceived effectiveness, suitability, credibility and success of the ad in increasing awareness of the IYP program.

Notes on Reading This Report

Reporting on the Results of Scaled Questions

Several questions in this report asked respondents to answer based on a scale. For example, we asked respondents to evaluate the performance of the Government of Canada on a scale of 1 to 7, where 1 means terrible, 7 means excellent and 4 means neither good nor bad. When reporting on these data we collapse ratings on the scale to indicate overall responses. For example, on the question of performance of the Government of Canada, ratings of 5, 6 or 7 are reported as "positive" responses.

A note on small base sizes

In cases where base sizes are exceptionally small, rather than displaying percentages which could be misleading, responses are presented in frequencies.

Key Findings

Overview of Wave One Findings

• Just over a third (36%) of Canadians aged 15 to 35 say that they are aware of international youth programs in Canada that enable young Canadians to work and travel in other countries. When those who are aware of such programs are asked to name these programs, two in five (42%) do not name any specific programs. Those who do name specific programs refer most commonly to high school or university exchange programs (10%), and variations of IYP programs such as Canada World Youth (10%) and others (6% SWAP/ Student Work Abroad Program, 2% the International Youth Program and 2% WHP/Working Holiday Program).
• When provided with a description of the International Youth Programs, roughly one in five youth says they are aware of them (13% yes, 9% maybe).
• After having heard more about IYPs, respondents were asked to rate their level of interest in having more information about these programs. Interest is high; 15 percent say they are very interested in having more information about IYPs, and another three in ten (29%) say they are somewhat interested.
• Two in five (39%) youth aged 15 to 35 years old say they are interested in participating in an IYP (14% very, 25% somewhat). However, three in five (61%) are either not very interested (25%) or not at all interested (36%). Of those who are interested, the most common reasons for this interest are travel experience (38%), learning about different cultures (29%) and learning about different countries and seeing new places (25%). Of those who are not interested, the most common reasons given are that they are employed or that they already have a good job (22%), that they have already started families (20%) or that they are too busy (19%).

Overview of Wave Two and Three Findings

• Unaided awareness is low with four in five saying they do not recall any advertising on international youth programs. Specifically, this includes:
  • 83 percent among the general public;
  • 82 percent of those aged 18-34 among the general public; and,
  • 84 percent among the youth oversample, aged 15-35.
• Among those who do recall advertising on this topic, three in ten recall things unrelated to the IYP advertising. But, half recall themes that seem to be related to the IYP campaign.
• In terms of where those who recall the advertising saw it, half (50%) say they recall seeing these advertisements on television and a quarter (24%) say they recall seeing them in newspapers.
• Just over a quarter (28%) of general public respondents recall seeing at least one of the treatments;
  • Just over a quarter (28%) of general public respondents recall seeing at least
  • Recall was highest for the television ad with one in five (20%) who recall seeing it; and,
  • Among the youth oversample, awareness was slightly higher with 41 percent who saw at least one ad and 33 percent who recall seeing the television ad.
• When asked if they recall the sponsor of the advertisement they saw, a plurality do not recall the sponsor for each ad.
• When those who say they saw the television ad are asked what they think was the main point the ad was trying to get across, a third (33%) say that they do not know. Of those who do recall a message, most recall messages directly related to the campaign such as "work abroad" (18%), "youth program/for young Canadians/for students" (11%) or "travel experience/travel abroad/see the world" (6%).
  • Nearly half (19 of the total n=41 responses) cannot recall the main point of the Power Lunch print ad, and almost as many (14 of the total n=41 responses) mention messages unrelated to the campaign and IYPs.
  • The Rush Hour ad garnered similar results, with 37 percent who do not recall the main point and 22 percent who recall messages unrelated to the campaign and IYPs.
  • Just over one fifth (6 of the total n=32 responses) cannot recall the main point of the Internet ad, and a similar proportion (7 of the total n=32 responses) mention messages unrelated to the campaign and IYPs.

Comparison of Pre and Post Campaign Findings, Among Youth

• General awareness among youth of international youth programs that allow young Canadians to work and travel in other countries decreased post campaign: 26 percent (Yes/Maybe) compared to 36 percent (Yes/Maybe) pre campaign.
  • When asked if they could name any of these programs, top-of-mind unaided awareness of IYPs increased five percentage points post campaign with 15 percent naming IYPs compared to 10 percent pre campaign.
• Aided awareness of IYPs among youth on the other hand has increased post campaign with a third (32%: 23% yes and 9% maybe) who say the are aware of the program compared to one in five (21%: 13% yes and 9% maybe) pre campaign.
• The proportion of youth who are interested in having information on IYPs remained about the same compared to the pre campaign sounding (41% somewhat or very interested compared to 44% pre campaign), as did interest in participating (39% somewhat or very interested compared to 39% pre campaign).
• Travel experience (28% compared to 38% pre campaign) still tops the list of reasons for interest in participating, though almost as many say they would be interested in either learning about different cultures (25% compared to 29% pre campaign) or having a different or life changing experience (23% compared to 17% pre campaign).
• Already having a job no longer tops the list of reasons for not being interested in participating in an IYP. Instead, post campaign results indicate that being too busy, having too many other plans and having no time tops the list with a quarter (25%) of those who are not interested citing this as the reason.

Performance of the Government of Canada - General Public

• Overall, a plurality of Canadians (n=1000 general public) are undecided when asked to rate the performance of the Government of Canada in providing programs that help Canadian youth to work and travel abroad, with 42% who rate this performance neither negatively nor positively (4 on a seven-point scale). One in five (22%) give a positive rating (5, 6, or 7 on a seven-point scale) and 29% give a negative rating (1, 2, or 3 on a seven-point scale).
• A quarter (25%) give the Government of Canada's performance in general a negative rating (1, 2, or 3 on a seven-point scale), 35% are undecided (4 on a seven-point scale) and two in five (38%) offer a positive rating (5, 6, or 7 on a seven-point scale).
• A third (32%) give the Government of Canada's performance in providing information to the public about government services that are available to Canadians a negative rating (1, 2, or 3 on a seven-point scale), 24 percent are undecided (4 on a seven-point scale) and two in five (43%) offer a positive rating (5, 6, or 7 on a seven-point scale).

Attributes of Government of Canada service - General Public

• When asked whether they agree or disagree with several statements about the Government of Canada, a majority (65%) of Canadians agrees (5, 6 or 7 on a seven-point scale) that the Government of Canada delivers its services and information in a respectful way.
• Half (48%) agree (5, 6 or 7 on a seven-point scale) that you can count on the Government of Canada for reliable service and information, with three in ten (27%) who disagree (1, 2, or 3 on a seven-point scale).
• Views are mixed with regard to the ease of contacting the Government of Canada for information and service (40% agree that it is easy, 21% neither agree nor disagree and 36% disagree that it is easy), and whether the Government of Canada is using new and innovative ways to provide information and services to citizens (36% agree that it is, 36% neither agree nor disagree and 26% disagree that it is).