Post ACET for COVID-19 Vaccination Advertising Campaigns
Methodological Report

Prepared for Health Canada
Supplier name: Ipsos Public Affairs
Contract Number: HT372-213292/001/CY
Contract value: $39,051.44
Award Date: January 15, 2021
Delivery Date: November 1st, 2021.
Registration number: POR 009-21

For more information on this report, please contact Health Canada at:
hc.cpab.por-rop.dgcap.sc@canada.ca.

Ce rapport est aussi disponible en français

POLITICAL NEUTRALITY STATEMENT

I hereby certify as Senior Officer of Ipsos that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.

Mike Colledge
President
Ipsos Public Affairs

Post ACET COVID-19 Vaccination Advertising Campaigns
Methodological Report

Prepared for Health Canada and the Public Health Agency of Canada
Supplier name: Ipsos Public Affairs
October 2021

This public opinion research report presents the methodology of the Post COVID-19 Vaccination Advertising Campaigns Evaluation online survey conducted by Ipsos Public Affairs on behalf of Health Canada and the Public Health Agency of Canada. The research study was conducted with n=2,005 Canadians aged 18+ between September 29th and October 12th, 2021.

Cette publication est aussi disponible en français sous le titre : Post-évaluation (OECP) des campagnes publicitaires sur la vaccination contre la COVID-19.

This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Health Canada. For more information on this report, please contact Health Canada at hc.cpab.por-rop.dgcap.sc@canada.ca or at:

Communications Branch
Health Canada
200 Eglantine Driveway
AL 1915C, Tunney's Pasture
Ottawa, ON K1A 0K9

Catalogue Number:
H14-385/2021E-PDF

International Standard Book Number (ISBN):
978-0-660-41083-8

Related publications (registration number: POR 009-21)
Catalogue Number H14-385/2021F-PDF (Final Report, French)
ISBN 978-0-660-41084-5

^©Her Majesty the Queen in Right of Canada, as represented by the Minister of Health, 2021

Executive Summary

1. Background

In December 2019, a cluster of cases of pneumonia was reported in Wuhan, China; the cause was confirmed as a new coronavirus (COVID-19) that had not previously been identified in humans. The situation quickly evolved into a worldwide pandemic. By October 2021, there were more than 1,676,846 confirmed cases in Canada and over 28,468 deaths. COVID-19 daily epidemiology update - Canada.ca

The COVID-19 response is an urgent Government of Canada priority. The planned vaccination advertising campaign has been approved for $40.5M.

Overall, the campaign has sought to achieve the following objectives:

• In phase 1: Provide Canadian populations with the information they need to make an informed decision to get the COVID-19 vaccines when they become available to them by increasing confidence and trust in the COVID-19 vaccines.
• In phase 2: Increase awareness among all Canadians of the importance of the COVID-19 vaccine to protect themselves and motivate action.

The research consisted of online surveys conducted in English and French, taking place between September 29th and October 12th, 2021.

Ipsos programmed, hosted, and provided sample management services, while HC and PHAC provided the online questionnaires. Ipsos was responsible for data collection and data storage in Canada, data processing and data weighting. The total contract value of this research was $39,051.44, including HST.

2. Research Objectives

PHAC is looking to assess the impact of its COVID-19 Vaccination Advertising Campaign in raising awareness, motivating interest, and generating engagement with federal information and resources and other Canadians about the COVID-19 vaccines at Canada.ca/covid-vaccine.

This research will help to evaluate the impact of this advertising campaign, such as:

• Unaided and aided awareness of the ads and their placement
• Assess message recall
• Determine if the messaging and visuals resonated with the target audience
• Understand the influence of the ad, in terms of visiting Canada.ca/covid-vaccine to access information and resources.
• Understand if the campaign motivated audiences to share information and or take an action as a result of seeing the ads.

The results of this research will allow PHAC to evaluate the performance of this multi-phase ad campaign. It will also help PHAC to improve future phases of this ongoing COVID-19 vaccination campaign.

The Government of Canada's Policy on Communications and Federal Identity requires the evaluation of all advertising campaigns exceeding $1 million in media through the Advertising Campaign Evaluation Tool (ACET).

The following campaigns were evaluated through this ACET:

1. COVID-19 vaccines: We can all help by getting vaccinated (launched May 17 and ended July 4, 2021)
2. COVID-19 Vaccine Questions (launched June 5 and ended July 31, 2021)
3. COVID-19 Vaccine Questions – Parents of Youth Edition (launched August 2 and ended September 30, 2021)
4. COVID-19 vaccines: It's time for kids to make memories again (launched August 25 and ended October

The objectives of the research are as follows:

• Determine if people have seen the advertisements associated with the campaign
• Determine where the ads have been seen
• Measure recall of specific elements of the campaign
• Identify attitudinal changes as a result of the advertising campaign

The target audience of the research was as follows:

• General Canadian population 18 years of age or more
• Young adults 18-24 years of age
• Young adults 25-34 years of age
• At-risk ethnic communities
• Indigenous communities
• Parents of children 12-17

3. Methodology

3.1 Online Sample

Respondents for the survey samples were drawn from the Ipsos i-Say panel, consisting of over 200,000 Canadians recruited in all provinces and territories. The survey sample of n=2,005 was drawn from Canadians 18+.

A pre-test was conducted on September 29th on a sample of n=40 respondents (27 in English and 13 in French), to confirm survey length before fully deploying the questionnaire. In the pre-test, an open-ended question was asked at the end of the survey where any problems, questions, or unclear questions could be brought to our attention. No issues were flagged during the pre-test that required revision to the original questionnaire.

The general population sample consisted of n=2,005 respondents. Quotas were set to ensure representation by region, age and gender, according to the latest Census information.

Within this sample, we set up minimum quotas of n=200 for each of five target groups (except Indigenous communities: target of n=100 and parents of children 12-17: target of n=300). In the end, within natural fallout we obtained our targets without having to oversample (note that one person can fall into more than one category).

• Young adults 18-24 years of age (n=212)
• Young adults 25-34 years of age (n=326)
• Ethnic communities (n=288)
• Indigenous communities (n=139)
• Parents of children 12-17 (n=335)

Interviews were conducted in English and French, with respondents being able to complete the survey in either official language of their choice.

Respondents were invited to participate in the survey via email, with a unique URL link. This link could only be used once, with respondents being allowed to pause during completion and return to complete it at a later time. On average, the online interviews took 11 minutes to complete.

3.2 Participant Recruitment

Sample Source

The Ipsos i-Say panel was the primary source for recruiting respondents and, where necessary, a supplemental sample from a third-party vendor, Dynata, was used for additional completes. Ipsos lead the data collection process in conjunction with Dynata. Outgoing sample lists were coordinated between Ipsos and Dynata to ensure that an individual was not allowed to complete the same survey twice, in the event that there are panel members who are on both companies' panels. The Ipsos i-Say online panel is a diversely-sourced and actively maintained panel of over 200,000 Canadian adults. Using this panel allowed us to confidently build online samples based on known population characteristics, such as ones based on Census data. Invitations to participate in each survey are adjusted to reflect the most recent current population data provided by Statistics Canada.

Online panel participants were recruited and screened using rigorous double and triple opt-in checks to ensure that respondents want to receive contact, such as invitations to participate in research surveys. The Ipsos i-Say panel's Terms and Conditions of Membership details the respondents' rights to, at any time and for any reason, ask to be removed from the panel and stop receiving emails from Ipsos and its clients.

Panel recruitment consists of the following steps:

• Various "wide net" methodologies (e.g., email campaigns, affiliate networks, banner ads, text ads, search engine, co-registration, offline-to-online, specialized websites); and,
• Customized incentives and materials for recruiting special targeted groups such as parents of children 12-17, etc.

It is important to note that any panel – no matter how well recruited – does not enable random selection in the sense that not all members of the public are likely to join an online survey panel. This applies to panels recruited through online approaches as well as panels recruited via traditional telephone Random Digit Dialing (RDD) in that the sampling naturally skews towards those who are amenable to answering a survey.

Rigorous panel maintenance procedures mean only those participants who actively participate in online surveys at least once every six months are included. Participants can choose whether to enter a survey or not. While participating in a survey, the respondent can simply exit the survey by closing the survey Internet browser window at any time without submitting their responses if they no longer want to participate.

To capture responses from low incidence populations, we use what is called a Webcruiser on our iSay Panel. A Webcruiser asks a set number of questions to our entire panel and typically takes approximately three weeks to complete. Panelists who meet the criteria we are looking for are then flagged and can be contacted for further research.

Incentives and Quality Control Measures

Respondents to Ipsos's online surveys are offered a number of innovative incentive programs in the forms of a points-based system where participants can redeem points for various items. We do not reward our panelists using cash payments.

Extensive quality-control procedures are in place within IIS (Ipsos Interactive Services, who manage our panel) to ensure that the survey inputs (sample and questionnaire design) allow for high-quality survey data. These processes span the life cycle of a panelist and are in place for all Ipsos online surveys. IIS experts are constantly monitoring and reviewing the performance of our quality measures and updating and integrating new ones as respondents' behaviours and the online landscape evolve.

Are panelists who they say they are?

• Double opt-in approach to confirm identity
• Country validation via Geo-IP
• Mismatch between device settings and geolocation
• Anonymous proxy detection
• Detection of robots via Captcha code
• Detection of "5 minute" emails (temporary email addresses)
• Detection of data anomalies and patterns
• Maintenance of Ipsos blacklist
• RealAnswer™- detection of pasted and robot answers

Have they recently participated in similar surveys?

• Strict panel usage rules to avoid interviewing the same people too often and prevent them from becoming too used to a type of survey or product category
• Duplicate device identification through digital Fingerprinting (RelevantID^®) and web/Flash cookies

Are they completing surveys seriously?

• Monitoring survey taking behavior: speeding, straight lining, open-ends quality evaluation
• Panelists' history monitored across surveys and used for panel purge removing "bad" or inactive respondents

Can they only take the survey once?

• Identification of duplicate email addresses
• Identification of duplicate devices through digital fingerprinting (RelevantID^®) and web/Flash cookies
• Identification of duplicate contact details

Sample Weighting

The table below shows the unweighted and weighted distributions of the online sample. Weighting was applied to the sample to ensure that the final data reflects the general adult population by region, age, and gender according to the 2016 Census. A Random Iterative Method (RIM) technique was applied for weighting.

The figures presented in the table above show minimal differences between the unweighted and weighted samples. As a result of the strong representativeness of the unweighted data the largest weight factor that was applied for any respondent is 1.09, which is well within acceptable ranges for a survey of the general population.

Email Statistics

For this survey, a sample router was used. Therefore, a response rate cannot be calculated. However, the participation rate for the survey was 94%. Participation rate is calculated as follows: (qualified completes + over quota + terminates)/click-through).

Non-Response Analysis

The results of this survey are not statistically projectable to the target population because the sampling method used does not ensure that the sample represents the target population with a known margin of sampling error. Reported percentages are not generalizable to any group other than the sample studied, and therefore no formal statistical inferences can be drawn between the sample results and the broader target population it may be intended to reflect. The data have been weighted to reflect the demographic composition of Canadian adults aged 18 years and over.

The table below compares the unweighted survey samples to the 2016 Census results by region, age, and gender. Overall, the sample is highly representative of the national adult population.

* Denotes variables included in the weighting scheme.

Differences among the variables used in the weighting scheme are minimal. The distribution in the sample is consistent, with only slight differences observed between the unweighted percentages and the 2016 Census data.

4. Appendix – Survey Questionnaire

4.1 English Questionnaire

INTRODUCTION

Thank you for taking the time to complete this survey dealing with current issues of interest to Canadians. Do you prefer to continue in English or French? / Préférez-vous continuer en français ou en anglais?

• English / Anglais
• French / Français

Part of this research involves hearing about an advertising campaign related to the COVID-19 Vaccines. Do you agree to participate in this research? The survey will take about 15 minutes to complete.

• Yes
• No - THANK AND TERMINATE

Your participation is voluntary, and your responses will be kept entirely confidential.

When you are ready to answer the survey, click on the following button to access the questionnaire.

START SURVEY

To view our privacy policy, click here.

If you require any technical assistance, please contact Daniel Kunasingam at daniel.kunasingam@ipsos.com

A) Does anyone in your household work for any of the following organizations?

SELECT ALL THAT APPLY

• A marketing research firm
• A magazine or newspaper
• An advertising agency or graphic design firm
• A political party
• A radio or television station
• A public relation company
• The federal or provincial government
• None of these organizations

IF "NONE OF THESE ORGANIZATIONS" CONTINUE, OTHERWISE THANK AND TERMINATE.

B) Are you …

• Male gender
• Female gender
• Gender diverse

C) In what year were you born?

YYYY

• I prefer not to answer

ADMISSIBLE RANGE 2002 and less

ASK D IF QUESTION C = "I prefer not to answer"

D) Would you be willing to indicate in which of the following age categories you belong?

• 18 to 24
• 25 to 34
• 35 to 44
• 45 to 54
• 55 to 64
• 65 or older
• Prefer not to answer [Thanks and Terminate]

E) In which province or territory do you live?

• Alberta
• British Columbia
• Manitoba
• New Brunswick
• Newfoundland and Labrador
• Northwest Territories
• Nova Scotia
• Nunavut
• Ontario
• Prince Edward Island
• Quebec
• Saskatchewan
• Yukon
• I prefer not to answer

IF "I prefer not to answer", THANK AND TERMINATE

ETHNIC
E2) Do you identify as any of the following? Select all that apply

White
Indigenous
South Asian
Chinese
Filipino
Southeast Asian
West Asian
Korean
Japanese
Black
Latin American
Arab
Other
None of the above [Exclusive]

PARENT

F) Are you the parent or guardian of a child or children under 18 years of age living in your household?

1- Yes
2- No
99- Prefer not to answer

[ASK IF QF=1 at PARENT ask QG, else skip to D5A]

G) How old is/are the child/children? Select all that apply

1-Under 6
2-6 to 11
3-12 to 17
99- Prefer not to answer

D5A:

Do you identify as any of the following?

SELECT ALL THAT APPLY

An Indigenous person (First Nations, Inuit or Métis)
A member of an ethnocultural or a visible minority group other than an Indigenous person
A member of the LGBTQ2 community
A person with a disability
None of the above
I prefer not to answer [Exclusive]

CAMPAIGN SPECIFIC QUESTIONS

ASK ALL RESPONDENTS

T1A:
Over the past six weeks, have you seen, read or heard any Government of Canada advertising about the COVID-19 Vaccines?

• Yes
• No => GO TO T1J

T1B:
Where have you seen, read or heard this ad about the COVID-19 Vaccines?

SELECT ALL THAT APPLY

• Cinema
• Facebook
• Internet website
• Magazines
• Newspaper (daily)
• Newspaper (weekly or community)
• Outdoor billboards
• Pamphlet or brochure in the mail
• Public transit (bus or subway)
• Radio
• Television
• Twitter
• YouTube
• Instagram
• LinkedIn
• Snapchat
• Spotify
• Tik Tok
• Shopping malls
• Digital screens (like in gas stations, convenience stores, residences, digital billboards)
• Other, please specify:

T1C:
What do you remember about this ad?

[RANDOMIZE BLOCKS – COVID-19 vaccines: We can all help by getting vaccinated]

COVID-19 vaccines: We can all help by getting vaccinated – TV - All respondents

T1J:
Here is an ad that has recently been broadcast on various media.

[Please add this before the video 'Please confirm your volume is on before watch the video']

[INSERT VIDEO – COVID-19 vaccines: We can all help by getting vaccinated]
https://www.youtube.com/watch?v=CG3yL7eP7jI

DESCRIPTION:
Image shows a scene from the ad.

[CLICK TO GO TO THE NEXT PAGE]

Over the past six months, have you seen this ad?

• Yes
• No => GO TO T1L

T1K:
Where have you seen this ad?

SELECT ALL THAT APPLY

• Cinema
• Facebook
• Internet website
• Magazines
• Newspaper (daily)
• Newspaper (weekly or community)
• Outdoor billboards
• Pamphlet or brochure in the mail
• Public transit (bus or subway)
• Radio
• Television
• Twitter
• YouTube
• Instagram
• LinkedIn
• Snapchat
• Spotify
• Tik Tok
• Shopping malls
• Digital screens (like in gas stations, convenience stores, residences, digital billboards)
• Other, please specify:

T1L: (ASK ALL)
What do you think is the main point this ad is trying to get across?

T1M: (ASK ALL)
Please indicate your level of agreement with the following statements about this ad?

COVID-19 Vaccine Questions –TV (General Population)/ COVID-19 Vaccine Questions –Digital (Parents of Youth Edition) – All respondents

T1JP:
Here is an ad that has recently been broadcast on various media.

[Please add this before the video 'Please confirm your volume is on before watch the video']

[INSERT ADS – COVID-19 Vaccine Questions –TV (General Population)/ COVID-19 Vaccine Questions –Digital (Parents of Youth Edition)]

All Respondent except parents: show one of these, rotate:

1. https://www.canada.ca/en/health-canada/services/video/ask-experts-covid-19-vaccines.html#a3
Do COVID-19 vaccines have side effects?

DESCRIPTION:
Image shows a scene from the video.

2. https://www.canada.ca/en/health-canada/services/video/ask-experts-covid-19-vaccines.html#a2
How was it possible to develop safe COVID-19 vaccines so quickly?

DESCRIPTION:
Image shows a scene from the video.

3. https://www.canada.ca/en/health-canada/services/video/ask-experts-covid-19-vaccines.html#a1
How do we know the COVID-19 vaccines are safe?

DESCRIPTION:
Image shows a scene from the video.

Parents: show one of these, rotate:

1. https://www.youtube.com/watch?v=8xpZDopn8i4
How do I know a COVID-19 vaccine is safe for my youth without long-term data?

DESCRIPTION:
Image shows a scene from the video.

2. https://www.youtube.com/watch?v=DYdkTAeUOVM
Should youth get a COVID-19 vaccine as soon as it's available to them?

DESCRIPTION:
Image shows a scene from the video.

[CLICK TO GO TO THE NEXT PAGE]

Over the past six months, have you seen this ad or an ad similar to this where an expert answers a question related to COVID-19 vaccination?

• Yes
• No => GO TO T1LP

T1KP:
Where have you seen, read or heard these ads?

SELECT ALL THAT APPLY

• Cinema
• Facebook
• Internet website
• Magazines
• Newspaper (daily)
• Newspaper (weekly or community)
• Outdoor billboards
• Pamphlet or brochure in the mail
• Public transit (bus or subway)
• Radio
• Television
• Twitter
• YouTube
• Instagram
• LinkedIn
• Snapchat
• Spotify
• Tik Tok
• Shopping malls
• Digital screens (like in gas stations, convenience stores, residences, digital billboards)
• Other, please specify:

T1LP:
What do you think is the main point these ads are trying to get across?

T1M1:
Please indicate your level of agreement with the following statements about these ads.

COVID-19 vaccines: It's time for kids to make memories again – TV (Youth Parents Edition) – Parents of child/children 12-17

T1JP2:
Here is an ad that has recently been broadcast on various media.

[Please add this before the video 'Please confirm your volume is on before watch the video']

[INSERT ADS – COVID-19 vaccines: It's time for kids to make memories again - parents – TV]
https://www.youtube.com/watch?v=TkaiXadCukA

DESCRIPTION:
Image shows a scene from the video.

[CLICK TO GO TO THE NEXT PAGE]
Over the past six weeks, have you seen this ad?

• Yes
• No => GO TO T1LP2

T1KP2:
Where have you seen, read or heard these ads?

SELECT ALL THAT APPLY

• Cinema
• Facebook
• Internet website
• Magazines
• Newspaper (daily)
• Newspaper (weekly or community)
• Outdoor billboards
• Pamphlet or brochure in the mail
• Public transit (bus or subway)
• Radio
• Television
• Twitter
• YouTube
• Instagram
• LinkedIn
• Snapchat
• Spotify
• Tik Tok
• Shopping malls
• Digital screens (like in gas stations, convenience stores, residences, digital billboards)
• Other, please specify:

T1LP2:
What do you think is the main point these ads are trying to get across?

T1M2:
Please indicate your level of agreement with the following statements about these ads.

[ASK TO RESPONDENTS WHO HAVE SEEN OF THE ADS PRESENTED]

T1N

Did you do any of the following as a result of seeing any of these ads? Randomize

• Visited the Canada.ca/covid-vaccine website
• Discussed COVID-19 vaccines with a friend or family member
• Shared the information
• Got vaccinated
• Encouraged others to get vaccinated
• Talked about the ad to others
• Looked for information on COVID-19 vaccination
• Other, specify ___________________[ANCHOR]
• I didn't do anything as a result of seeing the ad [EXCLUSIVE] [ANCHOR]

Additional questions – Ask all

The following questions are about the COVID-19 vaccines

Q1 Have you received a COVID-19 vaccine?

Yes, I have received three doses of the COVID-19 vaccines
Yes, I have received two doses of the COVID-19 vaccine
Yes, I have received one dose of the COVID-19 vaccine
No, but I plan to get vaccinated
No, I will not get vaccinated against COVID-19

ASK IF NOT VACCINATED

Q2 What is the main reason you have not received a COVID-19 vaccine? [Open-ended]

[ASK IF PARENTS WHO HAVE AT LEAST ONE CHILD "12-17" IN CHILD AGE]

Q3 Thinking about your child/children aged 12-17, will you choose to get your child/children vaccinated?

My child/children is/are already vaccinated
Yes, as soon as it is available
Yes, but will wait a bit
No, I will not get my child/children vaccinated
I do not make these decisions
Not sure

[ASK IF PARENTS WHO HAVE AT LEAST ONE CHILD 11 years of age or younger]

Q4 Thinking about your child/children under the age of 12, if a COVID-19 vaccine becomes available to your child/children, would you choose to get your child/children vaccinated?

Yes, as soon as it is available
Yes, but will wait a bit
No, I will not get my child/children vaccinated
I do not make these decisions
Not sure

[ASK IF "No, I will not get my child/children vaccinated" or "Not sure" IN Q3 and/or Q4]

Q5 What are the main reasons for not choosing to have the child(ren) under your care or guardianship vaccinated with a COVID-19 vaccine or for being not sure? Select up to 3 reasons. [Randomize]

It will be challenging to make or get to the vaccine appointment
I have concerns about possible side effects or that the vaccine will make my child(ren) sick
I am worried about the long-term effects of the vaccine on my child(ren)
I do not trust the newness of the vaccine
I need more information
It is not needed (e.g., virus will naturally disappear)
There is not enough testing or research that has been done on children
I do not believe it will be safe
If they get COVID-19, they are unlikely to get very sick
I think "natural immunity" is better for children than immunity from vaccines
I do not believe it will be effective
My child is very anxious or fearful of vaccinations
I am wary of vaccinating my child(ren) in general
Don't know [Anchor and Exclusive]
Other (please specify)

DEMOGRAPHIC QUESTIONS

D1:
Which of the following categories best describes your current employment status? Are you…

SELECT ONE ONLY

• Working full-time (30 or more hours per week)
• Working part-time (less than 30 hours per week)
• Self-employed
• Unemployed, but looking for work
• A student attending school full-time
• Retired
• Not in the workforce (Full-time homemaker or unemployed but not looking for work)
• Other employment status

D2:
What is the highest level of formal education that you have completed?

SELECT ONE ONLY

• Some high school or less
• High school diploma or equivalent
• Registered Apprenticeship or other trades certificate or diploma
• College, CEGEP or other non-university certificate or diploma
• University certificate or diploma below bachelor's level
• Bachelor's degree
• Postgraduate degree above bachelor's level

D3:
Which of the following categories best describes your total annual household income, including income from all household members, before taxes are deducted?

SELECT ONE ONLY

• Under $20,000
• Between $20,000 and $40,000
• Between $40,000 and $60,000
• Between $60,000 and $80,000
• Between $80,000 and $100,000
• Between $100,000 and $150,000
• $150,000 and above
• Prefer not to say

D4:
Where were you born?

• Born in Canada
• Born outside Canada
  Specify the country:

ASK IF D4=BORN OUTSIDE CANADA

D5:
In what year did you first move to Canada?
YYYY

ADMISSIBLE RANGE: 1900-2021

D6:
What is the language you first learned at home as a child and still understand?

SELECT UP TO TWO

• English
• French
• Other language, specify ___________________

That concludes the survey. This survey was conducted on behalf of Health Canada. In the coming months the report will be available from Library and Archives Canada. We thank you very much for taking the time to answer this survey, it is greatly appreciated.