METHODOLOGICAL REPORT
Prepared for Health Canada
Supplier name: Ipsos Public Affairs
Contract Number: HT372-215043/001/CY
Contract value: $32,733.30
Award Date: January 20th, 2022
Delivery Date: March 31st, 2022
Registration number: POR 109-21
For more information on this report, please contact Health Canada at:
hc.cpab.por-rop.dgcap.sc@canada.ca.
Ce rapport est aussi disponible en français
POLITICAL NEUTRALITY STATEMENT
I hereby certify as Senior Officer of Ipsos that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.
Mike Colledge
President
Ipsos Public Affairs
Prepared for Health Canada and the Public Health Agency of Canada
Supplier name: Ipsos Public Affairs
March 2022
This public opinion research report presents the methodology of the Post-Campaign ACET survey for COVID-19 Public Education Campaigns in 2021-22: Normalizing Public Health Measures and Personal Risk Assessment online survey conducted by Ipsos Public Affairs on behalf of Health Canada and the Public Health Agency of Canada. The research study was conducted with n=2,003 Canadians aged 18 -39 between February 15th and March 8th, 2022.
Cette publication est aussi disponible en français sous le titre : Sondage post-campagne OECP pour les campagnes d'éducation du publique COVID-19 en 2021-22 : Normalisation des mesures de santé publique et évaluation des risques personnels.
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Health Canada. For more information on this report, please contact Health Canada at hc.cpab.por-rop.dgcap.sc@canada.ca or at:
Communications Branch
Health Canada
200 Eglantine Driveway
AL 1915C, Tunney's Pasture
Ottawa, ON K1A 0K9
Catalogue Number:
H14-403/2022E-PDF
International Standard Book Number (ISBN):
978-0-660-43761-3
Related publications (registration number: POR 20-30)
Catalogue Number H14-403/2022F-PDF (Final Report, French)
ISBN 978-0-660-43762-0
©Her Majesty the Queen in Right of Canada, as represented by the Minister of Health, 2022
Throughout the COVID-19 pandemic situation in Canada public health measures continue to play an important role in limiting the spread of the virus and variants. However, there is increasing public fatigue with continuing to practice these measures, particularly among youth and young adults.
It is important to communicate and normalize the practice of public health measures to help Canadians stay healthy and to continue preventing the spread of COVID-19. To achieve this, COVID-19 public education efforts continue to play an important role as part of the Government of Canada's ongoing pandemic response. From December 2021 to mid-January 2021, HC and PHAC launched a +$2 million national advertising campaign to increase awareness of and interest in practicing public health measures. The campaign targeted Canadians aged 18 to 39 years using a mix of media including digital, social media, billboards, and cinema.
Overall, the campaign sought to achieve the following objectives:
The target populations of the campaign were Canadian youth and young adults aged 18 to 39 years.
The research involved conducting an online post-campaign survey to evaluate the campaign using the Government of Canada's ACET (Advertising Campaign Evaluation Tool). The objectives of the post-campaign survey were as follows:
The target audience of the research was as follows:
The survey was conducted online with a sample of n=2,003 Canadians between the ages of 18 and 39 years old. Quotas were set to ensure representation by region, age and gender, according to the latest Census information. Respondents were drawn from the Ipsos i-Say panel and a partner vendor panel, Canadian Viewpoint.
Within the sample of n=2,003, a minimum quota of n=100 Indigenous peoples with some speaking/reading one of these languages: Inuktitut, Cree, or Ojibway was achieved. As well, a minimum quote of n=300 respondents self-identifying as part of specific ethnic communities, including a minimum of 100 speaking/reading or living with someone speaking/reading one of these languages: Arabic, Cantonese, Farsi, Hindi, Korean, Mandarin, Punjabi, Spanish, Tagalog, Tamil, Urdu, Vietnamese) was achieved.
In the end, we obtained our targets through natural fallout, without having to oversample (note that one person can fall into more than one category).
Indigenous peoples, ethnic communities and languages were identified using these questions:
Interviews were conducted in English and French, with respondents being able to complete the survey in either official language of their choice.
The sampling methodology utilized email invitations + router technology to invite participants. Each participant received a unique URL link. This link could only be used once, with respondents being allowed to pause during completion and return to complete. On average, the online survey took 7 minutes to complete.
A pre-test was conducted on February 15th with a sample of n=27 respondents (16 in English and 11 in French), to confirm survey length before fully deploying the questionnaire. In the pre-test, an open-ended question was asked at the end of the survey where any problems with the clarity of the survey questions could be brought to our attention. No issues were flagged during the pre-test.
Sample Source
Ipsos partnered with Canadian Viewpoint Inc., on the fieldwork and in obtaining the required sample. Ipsos and Canadian Viewpoint have over 200,000 active panelists. Like Ipsos, Canadian Viewpoint uses mixed-mode strategies (phone/online/in-person) to build its panel permitting more representative samples. Canadian Viewpoint's online consumer panel is also recruited using post screeners during telephone surveys, Facebook, online vetted list and during in-person studies. Incentives are not used for recruitment purposes to ensure quality, but respondents are incentivized for completing survey directly proportionate to the amount of time taken to complete the survey and with comparable incentives offered by other online panel sources.
The comprehensive background profiling data gathered when respondents join a panel allows for the targeting of respondents based on key criteria, such as region, age, gender, education and income level, intention to buy a home within 12 months, profession, and other characteristics.
The data excludes any duplicate respondents based on IP capture and excludes panelists who have completed another Government of Canada survey in the previous 30 days as a member of that panel.
Sample Weighting
The table below shows the unweighted and weighted distributions of the online sample. Weighting was applied to the sample to ensure that the final data reflects the general adult population by region, age, and gender according to the 2016 Census. A Random Iterative Method (RIM) technique was applied for weighting. Note: totals may not add to 100% due to rounding.
| Unweighted Sample Size | Weighted Sample Size | |
|---|---|---|
| Canada | 2003 | 2003 |
| Region | ||
| British Columbia/Territories | 272 | 280 |
| Alberta | 226 | 220 |
| Prairies (MB/SK) | 131 | 140 |
| Ontario | 786 | 761 |
| Quebec | 472 | 461 |
| Atlantic Canada | 116 | 140 |
| Gender | ||
| Male | 974 | 992 |
| Female | 1005 | 992 |
| Diverse | 24 | 20 |
| Age | ||
| 18-24 | 605 | 624 |
| 25-29 | 469 | 444 |
| 30-34 | 468 | 476 |
| 55+ | 461 | 459 |
| Gender by Age | ||
| Male 18 - 24 | 290 | 317 |
| Male 25 - 29 | 230 | 218 |
| Male 30 - 34 | 230 | 238 |
| Male 35 - 39 | 224 | 218 |
| Female 18 - 24 | 304 | 297 |
| Female 25 - 29 | 229 | 218 |
| Female 30 - 34 | 238 | 238 |
| Female 35 - 39 | 234 | 238 |
The figures presented in the table above show minimal differences between the unweighted and weighted samples. As a result of the strong representativeness of the unweighted data the largest weight factor that was applied for any respondent is 1.20, which is well within acceptable ranges for a survey of the general population.
Email Statistics
For this survey, a non-probability sample was used. Therefore, a response rate cannot be calculated.
However, the participation rate for the survey was 91%. Participation rate is calculated as follows: Participation Rate = R/(R+IS+U).
| Completions | Post-campaign Survey |
|---|---|
| Invalid Cases | 833 |
| Unresolved (U) | 0 |
| In-scope non-responding (IS) | 297 |
| Responding units (R) | 2884 |
| Participation Rate | 91% |
Online survey cases can be broken down into four broad categories:
Invalid Cases
These can include only clearly invalid cases (for example, invitations mistakenly sent to people who did not qualify for the study, incomplete or missing email addresses in a client-supplied list).
Unresolved (U)
These include all the cases where it cannot be established whether the invitation was sent to an eligible or an ineligible respondent or unit (for example, when email invitations bounce back or remain without an answer before the candidate could be qualified).
In-scope non-responding (IS)
These include all refusals, either implicit or explicit; all non-contacts and early break-offs of known eligible cases; and other eligible non-respondents (due to illness, leave of absence, vacation or other).
Responding units (R)
These include cases who have participated but who were disqualified afterwards (for example, when admissible quotas have been reached). It also includes all completed surveys or partially completed surveys that meet the criteria set by the researcher to be included in the analysis of the data.
Unresolved (U), in-scope (IS), and responding units (R) are all included in the broad category of "potentially eligible" cases. However, invalid cases are not included in the calculation of outcome rates.
For this survey a router was used to screen potential respondents and assign them to one of several surveys. Given this, it is not possible to estimate the number of cases "invited" to participate and whether they were eligible or not. Therefore, it is not possible to estimate the "unresolved" cases. For this survey, responding units is broken out as follows.
| Completions | Post-campaign Survey |
|---|---|
| - Over quota | 881 |
| - Qualified Completes | 2003 |
| Responding units (R) | 2884 |
The sample routing technology uses weighted randomization to assign surveys to participants. Upon entry into the system, panelists are checked to ensure they have not exceeded survey participation limits. A list of potential survey matches is determined for each panelist based upon the information we know about them. Panelists may be asked additional screening questions within the system to ensure they meet the project criteria. Priority may be given to surveys that are behind schedule; however, this is kept to a minimum as survey randomization must remain in place as a key element for preventing bias. In this case, limited prioritization was applied during the field window, therefore there is a low chance of sample bias.
Non-Response Analysis
The results of this survey are not statistically projectable to the target population because the sampling method used does not ensure that the sample represents the target population with a known margin of sampling error. Reported percentages are not generalizable to any group other than the sample studied, and therefore no formal statistical inferences can be drawn between the sample results and the broader target population it may be intended to reflect. The data have been weighted to reflect the demographic composition of Canadian adults aged 18 to 39 years and over.
The table below compares the unweighted survey samples to the 2016 Census results by region, age, and gender. Overall, the sample is highly representative of the national adult population.
| Unweighted Percentage | Census 2016 Proportions (adults) | |
|---|---|---|
| Region* | ||
| British Columbia/Territories | 14% | 14% |
| Alberta | 11% | 11% |
| Prairies (MB/SK) | 7% | 7% |
| Ontario | 39% | 38% |
| Quebec | 24% | 23% |
| Atlantic | 6% | 7% |
| Gender* | ||
| Male | 49% | 50% |
| Female | 50% | 50% |
| Diverse Gender | 1% | <1% |
| Age* | ||
| 18-24 | 30% | 31% |
| 25-29 | 23% | 22% |
| 30-34 | 23% | 24% |
| 35-39 | 23% | 23% |
| Gender by Age | ||
| Male 18 - 24 | 15% | 16% |
| Male 25 - 29 | 12% | 11% |
| Male 30 - 34 | 12% | 12% |
| Male 35 - 39 | 11% | 11% |
| Female 18 - 24 | 15% | 15% |
| Female 25 - 29 | 11% | 11% |
| Female 30 - 34 | 12% | 12% |
| Female 35 - 39 | 12% | 12% |
| Diverse Gender | 1% | <1% |
* Denotes variables included in the weighting scheme.
Differences among the variables used in the weighting scheme are minimal. The distribution in the sample is consistent, with only slight differences observed between the unweighted percentages and the 2016 Census data.
INTRODUCTION
Thank you for taking the time to complete this survey dealing with current issues of interest to Canadians. Do you prefer to continue in English or French? / Préférez-vous continuer en français ou en anglais?
Your participation is voluntary, and your responses will be kept entirely confidential. The survey takes about 7 minutes to complete.
START SURVEY
To view our privacy policy, click here.
If you require any technical assistance, please contact Ipsos at pasurveys@ipsosasks.ca
a) Does anyone in your household work for any of the following organizations?
SELECT ALL THAT APPLY
IF "NONE OF THESE ORGANIZATIONS" CONTINUE, OTHERWISE THANK AND TERMINATE.
b) Are you...
c) In what year were you born?
ADMISSIBLE RANGE 1982-2004
IF > 2004 OR IF < 1982, THANK AND TERMINATE
ASK D IF QUESTION C IS LEFT BLANK
d) In which of the following age categories do you belong?
SELECT ONE ONLY
IF "LESS THAN 18 YEARS OLD" OR '40 OR OLDER' "BLANK", THANK AND TERMINATE
e) In which province or territory do you live?
SELECT ONE ONLY
IF NONE OF THE ABOVE IS SELECTED, THANK AND TERMINATE
SCREENER QUESTIONS TO IDENTIFY Indigenous peoples, ethnic communities and languages
f) Do you identify as any of the following? [Select all that apply]
g) [ASK ONLY IF f =2] Do you identify as any of the following? Select all that apply
h) [ASK ONLY IF f=1 OR f=2] Do you, or someone you live with, speak, or read fluently any of the following languages? Select all that apply
CORE QUESTIONS
ASK ALL RESPONDENTS
Q1:
Over the past six weeks, have you seen, read or heard any advertising from the Government of Canada?
Q3:
Think about the most recent Government of Canada ad that comes to mind. What do you remember about this ad?
CAMPAIGN SPECIFIC QUESTIONS
ASK ALL RESPONDENTS
T1A:
Over the past six weeks, have you seen, read or heard any Government of Canada advertising about keeping up various public health measures to slow the spread of COVID-19?
T1B:
Where have you seen, read or heard this ad about keeping up public health measures to slow the spread of COVID-19?
(Select all that apply) [Script question as below and expand each +sign with items]
+Print (e.g. flyer/newspapers/magazine)
+Broadcasting (radio/TV)
+Online / digital / Social media
+Indoor and outdoor signage
T1C:
What do you remember about this ad?
ASK ALL RESPONDENTS
T1D:
If public health measures were lifted in your area, how likely would you be to continue to do the following?
| Very likely | Likely | Unlikely | Very unlikely | Don't know/ Not applicable | |
|---|---|---|---|---|---|
| wear a mask - Indoors anytime in public | |||||
| wear a mask - Outdoors anytime in public | |||||
| wear a mask - Indoors in large public crowds | |||||
| wear a mask - Outdoors in large public crowds | |||||
| practice physical distancing | |||||
| Frequently wash your hands for at least 20 seconds with soap and warm water or use hand sanitizer containing at least 60% alcohol, if soap and water aren't available. | |||||
| stay home when you're feeling sick |
T1E:
In a post-pandemic time, meaning about a year from now and COVID-19 is behind us, how important is it to you personally to continue to practice individual public health measures when the risk of respiratory viruses are highest (e.g., during seasonal flu/cold seasons)?
T1F:
In post-pandemic times, how likely would you be to follow public health measures such as mask wearing and practicing physical distancing in each of the following scenarios?
| Very likely | Likely | Unlikely | Very unlikely | Don't know/Not applicable | |
|---|---|---|---|---|---|
| Health care or public health capacities are becoming overloaded | |||||
| The spread of a virus is increasing in your area | |||||
| There are respiratory viruses in circulation (e.g. seasonal flu) | |||||
| A virus is in circulation (e.g. seasonal flu) and you are gathering with individuals at high-risk of infection (e.g. seniors, immunocompromised, etc.) | |||||
| A virus is in circulation (e.g. seasonal flu) and you are attending a crowded event/location. | |||||
| When in public places (e.g. shopping, events, etc.) during cold and flu season. |
T1G:
Also, in a post-pandemic time, what do you think would motivate you most to continue practicing various individual public health measures such as mask wearing and practicing physical distancing?
Choose all that motivate you most. Knowing that you were…
[Randomize]
T1H:
Here is one public health measures COVID-19 ad that has recently been broadcast on various media. Click here to watch.
Scripter: please embed the correct video (into the English or French survey per language)
EN https://www.youtube.com/watch?v=qbVzVqPxEEM
FR https://www.youtube.com/watch?v=gA3W0aCD3eU
[CLICK TO GO TO THE NEXT PAGE]
Over the past six weeks have you seen this ad?
T1I:
Where have you seen, read or heard this ad? (Select all that apply)
T1J:
What do you think is the main point this ad is trying to get across?
T1K:
Please indicate your level of agreement with the following statements about this ad?
RANDOMIZE STATEMENTS
| 1 Strongly Disagree |
2 | 3 | 4 | 5 Strongly Agree |
|
|---|---|---|---|---|---|
| This ad catches my attention | |||||
| This ad is relevant to me | |||||
| This ad is difficult to follow | |||||
| This ad does not favour one political party over another | |||||
| This ad talks about an important topic | |||||
| This ad provides new information | |||||
| This ad clearly conveys how to help slow the spread of COVID-19 | |||||
| This ad motivated me how to keep practicing different public health measures | |||||
| This ad prompted/reminded me to practice physical distancing in public | |||||
| This ad prompted/reminded me to wear a well-fitting mask in public places | |||||
| This ad prompted/reminded me that vaccination is important, whether against COVID-19 or the seasonal flu | |||||
| This ad prompted/reminded me to wash my hands more frequently with soap and water | |||||
| This ad prompted/reminded me to clean and disinfect frequently touched surfaces and objects | |||||
| This ad prompted me to use a rapid test if I think I might have COVID-19 |
[Ask those seen the AD in T1H, else skip to T1LB]
T1LA.
Did you do any of the following as a result of seeing the ad? Please select all that apply
[RANDOMIZE]
[Ask those not seen the AD in T1H, else skip to D1]
T1LB.
Will you do any of the following as a result of seeing the ad? Please select all that apply
[RANDOMIZE]
DEMOGRAPHIC QUESTIONS
[ASK ALL]
D1:
Which of the following categories best describes your current employment status? Are you…
SELECT ONE ONLY
D2:
What is the highest level of formal education that you have completed?
D3:
Are there any children under the age of 18 currently living in your household?
D4:
Which of the following categories best describes your total annual household income, including income from all household members, before taxes are deducted?
SELECT ONE ONLY
D5:
Where were you born?
ASK IF D5=BORN OUTSIDE CANADA
D6:
In what year did you first move to Canada?
ADMISSIBLE RANGE: 1900-2022
D7:
What is the language you first learned at home as a child and still understand?
SELECT UP TO TWO
That concludes the survey. This survey was conducted on behalf of Health Canada. In the coming months the report will be available from Library and Archives Canada. We thank you very much for taking the time to answer this survey, it is greatly appreciated.