Global Affairs Canada (GAC) - ACTED

Travel and Border - COVID 19 Post-Advertising Campaign Evaluation Tool (ACET)

Methodological Report

Prepared For:
Global Affairs Canada

Contract Information

  • Supplier Name: Advanis Inc.
  • Contract number: EP363-140002/001/CY
  • Contract Value: $24,961.43 (including HST)
  • Award date: March 10, 2022
  • Delivery date: May 25, 2022
  • Registration number: POR 131-21

For more information on this report, please contact Global Affairs Canada at: POR-ROP@international.gc.ca

Ce rapport est aussi disponible en français.

Canada Logo

This report presents the methodological details for the Travel and Border – COVID 19 Post-Advertising Campaign Evaluation Tool (ACET) conducted by Advanis Inc. on behalf of Global Affairs Canada (GAC). The survey was administered among 2,072 members of the adult Canadian general public aged 18 or older, between April 4 and April 27, 2022.

Ce rapport est aussi disponible en français sous le titre: Outil d'évaluation de post-campagnes publicitaires (OECP) – COVID 19 sur les voyages et les frontières : rapport méthodologique

This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Global Affairs Canada. For more information on this report, please contact Global Affairs Canada at: POR-ROP@international.gc.ca

Lester B. Pearson Building Tower B3
125 Sussex Drive
Ottawa, Ontario K1A 0G2
Canada

1. Executive Summary

1.1 Background

As the COVID-19 pandemic evolves and restrictions change, the Government of Canada must communicate relevant travel information and provide resources to Canadians to make informed decisions around travel.

Feedback from social media comments, general inquiry mailboxes and 1-800 calls to Global Affairs Canada (GAC) and Treasury Board Secretariat (TBS), indicated that Canadians are finding it difficult to find and understand travel information that pertains to their situation during the pandemic. A high volume of inquiries, calls and emails to the GAC general information lines have been questions related to quarantine and isolation. The overall strategy of the ad campaign is to ensure that Canadians have all the up-to-date information and resources they need to make safe travel decisions by driving them to the https://travel.gc.ca/travel-covid site, the hub for all government travel and border information, which was launched on July 10, 2020. A mix of paid and organic media tactics across different digital channels was used to promote the resources Canadians need to make safe travel decisions.

The site reminds Canadians of the global travel advisories and raises awareness of the tools, advice and assistance at the disposal of Canadians to make informed decisions. The site also provides information on how to stay safe should they decide to travel abroad. It informs foreign nationals and Canadian citizens about entry restrictions, what they need to know before they travel or leave Canada, and what they can expect at the border. The latest Public Health Agency of Canada/Health Canada (PHAC/HC) information is also on this site. PHAC/HC is running a $1.5 million advertising campaign to support the ArriveCAN app, a key tool that will support Canada Border Services Agency (CBSA) operations.

Global Affairs Canada has launched an ad campaign to promote safe travel and border information during the pandemic. Travel and border post-campaign feedback will help to inform advertising campaigns in the COVID-19 era. This study collected feedback on the creative digital advertising campaigns and short-term outcomes. The results of this study will help to ensure GAC’s delivery of important messaging to Canadians in future advertising campaigns. Details of each digital advertising campaign are as follows:

Ad 1

If you must travel, follow Canada's travel requirements

In market:

Platforms:

Ad 2

Are you entering or returning to Canada? Learn about our entry requirements.

In market:

Platforms:

Ad 3

Are you planning to travel? Read more about Canada's re-entry requirements before you go.

In market:

Platforms:

Ad 4

Avoid non-essential travel.

In market:

Platforms:

1.2 Objective

The purpose of this quantitative study is to collect feedback from Canadians in order to:

1.3 Methodology

This study was completed from April 4 to April 27, 2022, in the form of a post-campaign survey through the use of Advanis’ General Population Representative Sample (GPRS) sample. Advanis sought a probability-based sample of 2,000 Canadian adults aged 18 or older drawn from the general population. Participants were invited via email or SMS to take part in a Web survey. The study was drawn from a random sample and can be extrapolated to the broader population only on a national level.

1.4 Contract Value

The contract value for this study was $24,961.43 (including HST).

1.5 Political Neutrality Requirement

Political neutrality certification

I hereby certify as Senior Officer of Advanis that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Policy on Communications and Federal Identity and the Directive on the Management of Communications.

Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.

Signed
Signature
Nicolas Toutant
Vice-President, Research and Evaluation
Advanis

2. Sampling

2.1 Project Characteristics

This study measured awareness, recall and impact of the ads among the target population. More precisely, the study aimed to assess:

The post-campaign survey was conducted from April 4 to April 27, 2022. There was no pre-campaign survey conducted. Canadians were invited to the survey from Advanis’ General Population Representative Sample (GPRS), which is a representative sample source recruited via probability sampling. Overall, 2,072 people completed the survey, with a response rate of 20.09%, and a margin of error of 2% (19 times out of 20 at a 95% confidence interval).

2.2 Sampling Plan

The target audience for this project was Canadians, across all provinces and territories, who are 18 years and older. The targeted number of completed surveys for the post-campaign survey was 2,000 Canadian adults.

Questions within the survey further filtered out responses from anyone under the age of 18, who did not live in Canada, and anyone working for a market research firm, a magazine or a newspaper, an advertising agency, a graphic design firm, a political party, a radio or television station, a public relations company or the federal/provincial governments.

Table 1: Sample plan by province/region of Canada
Region Percentage of population Target population
NFLD & Labrador 1.4% 28
PEI 0.45% 9
NS 2.6% 52
NB 2.1% 42
QC 22.9% 458
ON 38.7% 774
MB 3.6% 72
SK 3.2% 64
AB 11.6% 232
BC 13.1% 262
Territories 0.35% 7
Total 100% 2000

3. Invitations

3.1 Details of email and SMS invites/reminders

Invitations were sent by SMS and grouped by province, to ensure that they were sent out during appropriate hours within each time zone.

After sending the initial invitation, a reminder message was sent 3 days later to applicants who did not complete a survey or who were not screened out of the survey. A maximum of 2 reminder messages were sent, except for Ontario residents who received a maximum of 3 reminders. Overall, 26,042 SMS were sent during data collection for this study.

Table 2: Number of SMS invitations/reminders sent 
Message ID Purpose Total Sent
1 Invitation (EN) 8,399
2 Invitation (FR) 1,774
3 Reminder 1 (EN) 6,323
4 Reminder 1 (FR) 1,344
5 Reminder 2 (EN) 4,200
6 Reminder 2 (FR) 1,390
7 Reminder 3 (EN) (Ontario only) 2,612
Total 26,042

A test pre-launch was conducted on April 4, 2022. Respondents were asked to provide feedback on the survey. A total of 18 English and 24 French completed surveys were captured during this pre-test period. Neither language pre-tests highlighted any difficulties with the survey itself or with data collection. The data collected during the pre-test was retained for the analysis.

A support email address and phone number were provided in the survey itself for any related technical issues. Each survey had a unique number embedded in the hyperlink to eliminate the possibility of duplicate responses from one participant.

4. Completes and Targets

4.1 Completes and targets

Overall, 10,173 Canadians were invited to participate in the web survey. The average survey length was 7 minutes. In total, 2,072 web questionnaires were completed during the data-collection period.

Table 3: Number of completes and targeted number of completes per province
Province/Region Completed the web survey Target
Nova Scotia 73 52
Newfoundland & Labrador 16 28
New Brunswick 43 42
PEI 13 9
Quebec 472 458
Ontario 777 774
Manitoba 80 72
Saskatchewan 64 64
Alberta 260 232
British Columbia 268 262
Territories 6 7
Total 2072 2000

4.2 Screened-out participants

Table 4: Number of screened-out participants
Working in ineligible sector Does not live in Canada Younger than 18 Total
258 3 1 262

5. Weighting

Overall, 2,072 web interviews were conducted during data collection. The weighting of the final file was based on 3 variables: age, gender and region. A weight value of 1 was attributed to the 26 people who were neither of male nor female genders. The population sizes are based on the latest Statistics Canada census results published—the 2016 census.

Table 5: The weights
Weight category Unweighted N Weighted N Weight
BC & Yukon, male gender, 18 to 34 27 37 1.380
BC & Yukon, male gender, 35 to 54 43 45 1.046
BC & Yukon, male gender, 55 or older 46 53 1.153
BC & Yukon, female gender, 18 to 34 27 37 1.361
BC & Yukon, female gender, 35 to 54 45 48 1.066
BC & Yukon, female gender, 55 or older 80 59 0.737
Alberta & NWT, male gender, 18 to 34 17 37 2.199
Alberta & NWT, male gender, 35 to 54 66 42 0.640
Alberta & NWT, male gender, 55 or older 38 36 0.935
Alberta & NWT, female gender, 18 to 34 10 36 3.638
Alberta & NWT, female gender, 35 to 54 83 42 0.502
Alberta & NWT, female gender, 55 or older 45 38 0.845
Prairies & Nunavut, male gender, 18 to 34 25 20 0.816
Prairies & Nunavut, male gender, 35 to 54 29 22 0.766
Prairies & Nunavut, male gender, 55 or older 20 23 1.171
Prairies & Nunavut, female gender, 18 to 34 15 20 1.329
Prairies & Nunavut, female gender, 35 to 54 27 22 0.832
Prairies & Nunavut, female gender, 55 or older 28 26 0.938
Ontario, male gender, 18 to 34 122 108 0.884
Ontario, male gender, 35 to 54 110 130 1.186
Ontario, male gender, 55 or older 160 139 0.867
Ontario, female gender, 18 to 34 87 108 1.236
Ontario, female gender, 35 to 54 118 140 1.183
Ontario, female gender, 55 or older 166 159 0.958
Quebec, male gender, 18 to 34 26 62 2.366
Quebec, male gender, 35 to 54 79 80 1.012
Quebec, male gender, 55 or older 96 92 0.956
Quebec, female gender, 18 to 34 37 61 1.650
Quebec, female gender, 35 to 54 87 80 0.919
Quebec, female gender, 55 or older 142 104 0.736
Atlantic, male gender, 18 to 34 5 16 3.226
Atlantic, male gender, 35 to 54 20 22 1.119
Atlantic, male gender, 55 or older 32 29 0.894
Atlantic, female gender, 18 to 34 11 16 1.475
Atlantic, female gender, 35 to 54 26 24 0.921
Atlantic, female gender, 55 or older 51 32 0.631
Any other gender 26 26 1

6. Database and Banners

The database was cleaned to remove any errors at the end of the data-collection phase, and all unique identifiers in the client profiles were removed in the final data set provided to GAC. All survey answers have been matched and compiled into banner tables.

New variables were created to include in the banner tables. Using the responses to the survey questions, Advanis created variables for the following subgroups to facilitate further analysis:

7. Survey design

The survey draft and the French translations were provided by GAC and were programmed using SurveyBuilder, a software program that is proprietary to Advanis. The surveys were available to be completed online and were compatible with both desktop computers and mobile devices (tablets and smartphones). The surveys were housed on a website hosted by Advanis.

Respondents were shown 4 image advertisement banners (static image with text) and were asked follow-up questions about their exposure to the ads, their reaction to the ads and their assessment of the ads.

The surveys were designed to include multiple-choice questions, including scaled, open-ended and demographic questions. Skip logic was applied throughout, including 3 thank-you messages used for the screening out of ineligible participants (having an occupation that makes them ineligible to participate, not living in Canada, or being under the age of 18). The survey was thoroughly pre-tested to ensure that skip patterns and survey questions were correctly programmed.

Several strategies were employed to increase response rates and reduce the effects of non-response bias. This includes recruiting potential respondents by telephone to increase response rates as emails are easy to ignore. Sending an SMS to recruits also assures a seamless transition from the telephone survey to the online survey as receipt can be confirmed in real time and encourages respondents to continue as soon as the call ends. When calling, techniques such as out-pulsing a local phone number rather than a toll-free number and routing calls from respondents to a live interviewer when they call the number that we out-pulse were also used to increase response rates.

8. Appendices

8.1 Appendix A: Survey English Questionnaire

Travel ACET Web: Government of Canada

Advanis logo

LoginTCH
Advertising Campaign Survey

The Government of Canada is conducting this survey dealing with current issues of interest to Canadians.

Your participation is voluntary and your responses will be kept entirely confidential. The survey takes about 5 minutes to complete.

La version française est disponible en cliquant sur le bouton ci-dessus.

Click here if you wish to verify the authenticity of this survey. You will need to enter the Project code: 20220317-AD712.

If you require any assistance, please contact POR-ROP@international.gc.ca

For technical assistance, please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net

If you get interrupted while doing the survey, you can click on the same link to pick up right where you left off.

QA

Does anyone in your household work for any of the following organizations?

Click here if you require any technical assistance: *?*

Select all that apply

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

Show if QA Any Selected

Thnk1

Thank you for your interest. This survey is intended for those who are not working in any of the sectors mentioned in the previous question.

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

QB

Are you…

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

QC

In what year were you born?

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

Minimum: 1900, Maximum: 2006

__________

Show if QC Not Answered

QD

Would you tell us in which of the following age categories you belong?

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

Show if QC or QD less than 18yo

Thnk2

Thank you for your interest. This survey is intended for those who are 18 or over.

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

QE

In which province or territory do you live?

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

Show if QE No Answer

Thnk3

Thank you for your interest. This survey is intended for those who live in a Canadian province or Territory.

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

Q1

Over the past three weeks, have you seen, read or heard any advertising from the Government of Canada?

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

Show if Q1 Yes

Q3

Think about the most recent Government of Canada ad that comes to mind. What do you remember about this ad?

Click here if you require any technical assistance: *?*

Please do not enter personally-identifying information (e.g., name, email address, phone number, mailing address), as anything you enter may be shared with the sponsor of this research.

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

__________________________________________________

__________________________________________________

__________________________________________________

__________________________________________________

__________________________________________________

T1A

Over the past three weeks, have you seen, read or heard any Government of Canada advertising about about international travel?

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

Show if T1A Yes

T1B

Where have you seen, read or heard this ad about travel?

Select all that apply

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

T1C

What do you remember about this ad?

Click here if you require any technical assistance: *?*

Please do not enter personally-identifying information (e.g., name, email address, phone number, mailing address), as anything you enter may be shared with the sponsor of this research.

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

__________________________________________________

__________________________________________________

__________________________________________________

__________________________________________________

T1D

When it comes to Government of Canada entry requirements, where do you typically get your information?

Select all that apply

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

T1HBanners1

Here are some ads that have recently been broadcast on various media. Please review these ads carefully before proceeding to the next question.

Click here if you require any technical assistance: *?*

If you must travel, follow Canada's entry requirements.

Are you entering or returning to Canada? Learn about our entry requirements.

Are you planning to travel? Read more about Canada's re-entry requirements before you go.

Avoid non-essential travel.

Click on the right arrow to go to the next page.

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

T1H

Over the past three weeks, have you seen, read or heard these ads?

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

Show if T1H Yes

T1I

Where have you seen, read or heard these ads?

Select all that apply

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

T1J

What do you think is the main point these ads are trying to get across?

Click here if you require any technical assistance: *?*

Please do not enter personally-identifying information (e.g., name, email address, phone number, mailing address), as anything you enter may be shared with the sponsor of this research.

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

__________________________________________________

T1K

Please indicate your level of agreement with the following statements about these ads: *?*

*?* If you require any technical assistance, please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

Levels marked with * are randomized

D1

Which of the following categories best describes your current employment status? Are you…

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

D2

What is the highest level of formal education that you have completed?

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

D3

Are there any children under the age of 18 currently living in your household?

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

Ex1

Have you travelled internationally for either business or leisure in the past year?

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

Show if D3 yes and Ex1 yes

Ex2

Did you travel internationally with your family during March break/Spring break 2022?

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

Show if D1 student and Ex1 yes

Ex3

Did you travel to a warmer destination during March break/Spring break 2022?

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

Show if Ex1 no

Ex4

Would you say you have an interest in international travel for either business or leisure?

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

D4

Which of the following categories best describes your total annual household income, including income from all household members, before taxes are deducted?

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

D5

Where were you born?

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

Show if D5 Outside Canada

D5elsewhere

Please select the country in which you were born.

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

Show if D5 Outside Canada

D6

In what year did you first move to Canada?

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

Minimum: 1900, Maximum: 2022

__________

D7

What is the language you first learned at home as a child and still understand?

Click here if you require any technical assistance: *?*

Select up to two

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).

EndTCH

That concludes the survey. This survey was conducted on behalf of Global Affairs Canada. In the coming months the report will be available from Library and Archives Canada. We thank you very much for taking the time to answer this survey, it is greatly appreciated.

If you'd like to see results from other studies conducted by Advanis via TellCityHall, please visit tellcityhall.ca/surveys (http://www.tellcityhall.ca/surveys.html).

Click here if you require any technical assistance: *?*

*?* please contact Sue Day, by telephone at 1-866-542-6921 or by e-mail at survey+usborderadv2022@advanis.net (mailto:survey+usborderadv2022@advanis.net).