Ipsos Public Affairs
Prepared for Health Canada
Supplier name: Ipsos Public Affairs
Contract Number: HT372-203632/001/CY
Contract value: $40,358.76
Award Date: November 5, 2020
Delivery Date: January 15, 2021
Registration number: POR 057-20
Ipsos Public Affairs
1 Nicholas St, Suite 1400
Ottawa ON K1N 7B7
Tel: 613.241.5802
Fax: 613.248.7981
www.ipsos.ca
For more information on this report, please contact Health Canada at: hc.cpab.por-rop.dgcap.sc@canada.ca.
Ce rapport est aussi disponible en français.
I hereby certify as Senior Officer of Ipsos that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.
Mike Colledge
President
Ipsos Public Affairs
Post-Seasonal Flu Vaccination Awareness Campaign Evaluation
Methodological Report
Prepared for Health Canada
Supplier name: Ipsos Public Affairs
January 2021
This public opinion research report presents the methodology of the post-seasonal flu vaccination awareness campaign ACET online survey conducted by Ipsos Public Affairs on behalf of Health Canada. The research study was conducted with n=2,535 Canadians aged 18+ between December 4th and 22nd, 2020.
Cette publication est aussi disponible en français sous le titre : Évaluation post-campagne de sensibilisation à la vaccination contre la grippe saisonnière.
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Health Canada. For more information on this report, please contact Health Canada at hc.cpab.por-rop.dgcap.sc@canada.ca or at:
Communications Branch
Health Canada
200 Eglantine Driveway
AL 1915C, Tunney’s Pasture
Ottawa, ON K1A 0K9
Catalogue Number:
H14-356/2021E-PDF
International Standard Book Number (ISBN):
978-0-660-37172-6
Related publications (registration number: POR 057-20)
Catalogue Number H14-356/2021F-PDF (Final Report, French)
ISBN 978-0-660-37173-3
© Her Majesty the Queen in Right of Canada, as represented by the Minister of Health, 2021
The 2020-21 Seasonal Influenza (Flu) Vaccination Awareness Campaign is an important component of PHAC's COVID-19 response. A robust advertising campaign was needed to maximize flu vaccine uptake and reduce pressure on the health care system in the event of a second wave of COVID-19 illness during flu season. Strong flu vaccine uptake is critical to decreasing hospitalizations due to flu-related complications.
To this end, Health Canada launched an advertising campaign to raise awareness of the importance of getting the flu vaccine, especially amongst high-risk populations.
The campaign primarily targeted adults 65+ and those with underlying medical conditions. Secondary audiences included pregnant women, parents of kids under 5 years of age and young adults 18-34 years of age.
The campaign was largely digital, predominately based on web banners, social media and search engine marketing. There was also some out-of-home elements and audio ads on Spotify.
The research consisted of online surveys conducted in English and French, taking place between December 4 and 22, 2020.
Ipsos programmed, hosted and provided sample management services, while Health Canada provided the online questionnaires. Ipsos was responsible for data collection and data storage in Canada, data processing and data weighting. The total contract value of this research was $40,358.76, including HST.
The 2020-21 Seasonal Flu Vaccination Awareness Campaign is an important component to PHAC's COVID-19 response. The campaign aimed to increase uptake of the flu vaccine, particularly in high-risk groups, to help mitigate the impact on the health care system during a second wave of the COVID-19 pandemic.
The Government’s Policy on Communications and Federal Identity requires the evaluation of advertising campaigns exceeding $1 million in media buy using the Advertising Campaign Evaluation Tool (ACET). The ACET was created in 2002 following a Cabinet directive identifying the need for a standard advertising evaluation approach across departments. The main objectives were to bring rigor and consistency to ad campaign evaluation and to develop norms on metrics against which campaigns could be evaluated.
The specific research objectives were as follows:
The target audience of the research was as follows:
Respondents for the post-wave samples were drawn from the Ipsos i-Say panel, consisting of over 200,000 Canadians recruited in all provinces and territories. The post-wave sample of n=2,535 was drawn from Canadians 18+ and included an oversample of n=535 (see detailed below regarding oversampling).
The general population sample consisted of n=2,000 respondents. Quotas were set to ensure representation by region, age and gender, according to the latest Census information.
In addition to the natural fallout from the general population sample, oversamples in each of the following four target groups were carried out in order to reach a minimum of n=500 in each of these groups. In the end, we obtained the following number of completes, including both natural fallout and oversample (note that one person can fall into more than one category):
For the target group of pregnant women, we attempted to obtain as many as possible, with the goal of reaching a minimum of n=50. In the end, we reached n=69 pregnant women.
Interviews were conducted in English and French, with respondents being able to complete the survey in either official language of their choice.
Respondents were invited to participate in the survey via email, with a unique URL link. This link could only be used once, with respondents being allowed to pause during completion and return to complete it at a later time. On average, the online interviews took 9 minutes to complete.
The Ipsos i-Say panel was the primary source for recruiting respondents and, where necessary, a supplemental sample from a third-party vendor, MARU, was used for additional completes. The Ipsos i-Say online panel is a diversely-sourced and actively maintained panel of over 200,000 Canadian adults. Using this panel allowed us to confidently build online samples based on known population characteristics, such as ones based on Census data. Invitations to participate in each survey are adjusted to reflect the most recent current population data provided by Statistics Canada.
Online panel participants were recruited and screened using rigorous double and triple opt-in checks* to ensure that respondents want to receive contact, such as invitations to participate in research surveys. The Ipsos i-Say panel’s Terms and Conditions of Membership details the respondents’ rights to, at any time and for any reason, ask to be removed from the panel and stop receiving emails from Ipsos and its clients.
Panel recruitment consists of the following steps:
It is important to note that any panel – no matter how well recruited – does not enable random selection in the sense that not all members of the public are likely to join an online survey panel. This applies to panels recruited through online approaches as well as panels recruited via traditional telephone Random Digit Dialing (RDD) in that the sampling naturally skews towards those who are amenable to answering a survey.
Rigorous panel maintenance procedures mean only those participants who actively participate in online surveys at least once every six months are included. Participants can choose whether to enter a survey or not. While participating in a survey, the respondent can simply exit the survey by closing the survey Internet browser window at any time without submitting their responses if they no longer want to participate.
In order to capture responses from low incidence individuals, such as pregnant women who are vaccine-hesitant, we use what is called a Webcruiser on our iSay Panel. A Webcruiser asks a set number of questions to our entire panel and typically takes approximately three weeks to complete. Panelists who meet the criteria we are looking for are then flagged and can be contacted for further research.
Incentives and Quality Control Measures
Respondents to Ipsos’s online surveys are offered a number of innovative incentive programs in the forms of a points-based system where participants can redeem points for various items. We do not reward our panelists using cash payments.
Extensive quality-control procedures are in place within IIS ( Ipsos Interactive Services, who manage our panel) to ensure that the survey inputs (sample and questionnaire design) allow for high-quality survey data. These processes span the life cycle of a panelist and are in place for all Ipsos online surveys. IIS experts are constantly monitoring and reviewing the performance of our quality measures and updating and integrating new ones as respondents’ behaviors and the online landscape evolve.
Are panelists who they say they are?
Have they recently participated in similar surveys?
Are they completing surveys seriously?
Can they only take the survey once?
Sample Weighting
The table below shows the unweighted and weighted distributions of the post-wave online samples. Weighting was applied to the sample to ensure that the final data reflects the general adult population by region, age, and gender according to the 2016 Census. A Random Iterative Method (RIM) technique was applied for weighting.
| Unweighted Sample Size | Weighted Sample Size | |
|---|---|---|
| Canada | 2535 | 2535 |
| Region | ||
| British Columbia | 345 | 344 |
| Alberta | 303 | 284 |
| Prairies (MB/SK) | 172 | 165 |
| ON | 1002 | 974 |
| QC | 544 | 595 |
| Atlantic Canada | 169 | 173 |
| Gender | ||
| Male | 1093 | 1237 |
| Female | 1432 | 1288 |
| Diverse | 10 | 10 |
| Age | ||
| 18-34 | 856 | 710 |
| 35-54 | 800 | 862 |
| 55+ | 879 | 963 |
The figures presented in the table above show minimal differences between the post-wave unweighted and weighted samples. As previously noted, the post-wave research was drawn from a sample of adult Canadians, aged 18 years an over. This resulted in a higher weight ratio of 1.28:1, which remains within acceptable ranges for a survey of the general population.
Email Statistics
The table below presents general statistics regarding the response rate for the email phase of the research. Overall, a response rate 13% was achieved, which is within normal ranges for a survey of the Canadian adult population.
| Calculation for Data Collection | Post-Wave |
|---|---|
| Total Email Invitations Issued | 26,685 |
| Invalid (incomplete/incorrect email address, email invitation bounce backs) | 0 |
| Unresolved (U) (no response at all) | 22,324 |
| In-scope - non-responding (IS) | 945 |
| In-scope - Responding units (R) | 3,416 |
| Over quota | 630 |
| Other disqualified | 251 |
| Completed questionnaires | 2,535 |
| Response Rate = R/(U+IS+R) | 13% |
Non-Response Analysis
As with any probability sample, there exists within the current sample the possibility of non-response bias. In particular, this survey does not include members of the population who do not have access to a computer with an Internet connection (either at home or at work) or who are not capable of responding to a survey in either English or French. In addition, some groups within the population are systemically less likely to answer surveys.
The table below compares the unweighted post-wave samples to the 2016 Census results by region, age, and gender. Overall, the sample is highly representative of the national adult population, except for a few gaps, which are described below.
| Unweighted Percentage | Census 2016 Proportionsh (adults) |
|
|---|---|---|
| Region* | ||
| BC | 13.6% | 13.6% |
| Alberta | 12.0% | 11.2% |
| Prairies (MB/SK) | 6.8% | 6.5% |
| Ontario | 39.5% | 38.4% |
| Quebec | 21.5% | 23.5% |
| Atlantic | 6.7% | 6.8% |
| Gender* | ||
| Male | 43.1% | 48.5% |
| Female | 56.5% | 51.4% |
| Diverse | <1% | <1% |
| Age* | ||
| 18-34 | 33.8% | 27.3% |
| 35-54 | 31.6% | 34.1% |
| 55+ | 34.7% | 38.6% |
* Denotes variables included in the weighting scheme.
Differences among the variables used in the weighting scheme are minimal; however, there are a few notable differences. Young adults (18-34) are overrepresented in the unweighted sample. This is due to the intentional oversampling of this population and is to be expected. Additionally, females are also overrepresented. However, this is due to the oversampling of pregnant women in the research and is also to be expected. The remaining distribution in the sample remains consistent, with only slight differences observed between the unweighted percentages and the 2016 Census data. Regional distributions similarly demonstrate slight, but not significant, differences between the unweighted percentages and the 2016 Census data.
INTRODUCTION
Thank you for taking the time to complete this survey dealing with current issues of interest to Canadians. Do you prefer to continue in English or French? / Préférez-vous continuer en français ou en anglais ?
Your participation is voluntary, and your responses will be kept entirely confidential . The survey takes about 5 minutes to complete.
START SURVEY
To view our privacy policy, click here.
If you require any technical assistance, please contact Daniel Kunasingam at daniel.kunasingam@ipsos.com
a) Does anyone in your household work for any of the following organizations? Select all that apply.
IF “NONE OF THESE ORGANIZATIONS” CONTINUE, OTHERWISE THANK AND TERMINATE.
b) Are you…
c) In what year were you born?
YYYY
ADMISSIBLE RANGE 1900-2002
IF > 2002, THANK AND TERMINATE
ASK D IF QUESTION C IS LEFT BLANK
d) In which of the following age categories do you belong?
SELECT ONE ONLY
IF “LESS THAN 18 YEARS OLD”, THANK AND TERMINATE
E) In which province or territory do you live?
SELECT ONE ONLY
F1. Are you the parent or legal guardian of a child age 4 years old or younger?
[If Yes in F1 ask F2 , else skip to QG]
F2. Please enter the age of your child(ren) below. If your child is under 1 years old, please enter 0
[INSERT CHILD BOXES FOR NUMBER OF CHILDREN - CHILD 1, CHILD 2, CHILD 3,…CHILD 10]
QG. Has a doctor or other health care professional ever told you that you have any of the following? Select all that apply
[RANDOMIZE]
[If anyone select options 1 ,2,4,6, 11 to 16 consider they have ‘People with underlying health conditions’.]
IF FEMALE ask QH
H) Are you currently pregnant?
CORE QUESTIONS
ASK ALL RESPONDENTS
Q1:
Over the past three five weeks, have you seen, read or heard any advertising from the
Government of Canada?
Q2:
Think about the most recent ad from the Government of Canada that comes to mind. Where have you seen, read or heard this ad? Select all that apply.
Q3:
What do you remember about this ad?
Q4:
How did you know that it was an ad from the Government of Canada?
CAMPAIGN SPECIFIC QUESTIONS
ASK ALL RESPONDENTS
T1A:
Over the past five weeks, have you seen, read or heard any
Government of Canada advertising about Seasonal Influenza (Flu) Vaccination?
T1B:
Where have you seen, read or heard this ad about Seasonal Influenza (Flu) Vaccination? Select all that apply.
T1C:
What do you remember about this ad?
ASK ALL RESPONDENTS
Pretest1: Was anything in the survey today confusing?
T1D. How worried are you, if at all, that you or someone else in your family may get the seasonal flu this fall or winter?
T1E. Do you plan to get, or have you already received, the seasonal flu shot this upcoming flu season, this fall and winter?
[If No ask T1F, else skip to T1G]
T1F. Why are you not planning to get the seasonal flu shot this upcoming flu season? Select all that apply.
[Ask all]
T1G. How does or did, the COVID-19 pandemic affect your likelihood of getting the seasonal flu shot this upcoming flu season, this fall and winter? Would you say you are:
T1H:
Here are some ads that have recently been broadcast on various media. Click next to watch.
[INSERT PRINT AND RADIO ADS ON SEPARATE PAGES AS PER THE LOGIC MENTIONED IN THE QUOTA CHART. SAME RADIO AD SHOWN TO ALL. ALL PRINT ADS TOGETHER ON ONE PAGE.]
[CLICK TO GO TO THE NEXT PAGE]
Over the past five weeks, have you seen, read or heard these ads?
T1I:
Where have you seen, read or heard these ads? Select all that apply.
T1J:
What do you think is the main point these ads are trying to get across?
T1K: Please indicate your level of agreement with the following statements about these ads?
RANDOMIZE STATEMENTS
| 1 Strongly Disagree |
2 | 3 | 4 | 5 Strongly Agree |
|
|---|---|---|---|---|---|
| These ads catch my attention | o | o | o | o | o |
| These ads are relevant to me | o | o | o | o | o |
| These ads are difficult to follow | o | o | o | o | o |
These ads do not favour one political party over another |
o | o | o | o | o |
| These ads talk about an important topic | o | o | o | o | o |
| These ads provide new information | o | o | o | o | o |
These ads made me more likely to think that seasonal influenza (flu) vaccination is important |
o | o | o | o | o |
These ads prompted me to get the seasonal influenza (flu) vaccination |
o | o | o | o | o |
T1L. Did you do any of the following as a result of seeing the ads? Please select all that apply
[RANDOMIZE]
I didn’t do anything as a result of seeing the ad [EXCLUSIVE] [ANCHOR]
Pretest2: Was anything in the survey today confusing?
DEMOGRAPHIC QUESTIONS
D1:
Which of the following categories best describes your current
employment status? Are you…
SELECT ONE ONLY
D2: What is the highest level of formal education that you have completed?
SELECT ONE ONLY
D3: Are there any children under the age of 18 currently living in your household?
D4:
Which of the following categories best describes your total annual
household income, including income from all household members,
before taxes are deducted?
SELECT ONE ONLY
D5: Where were you born?
Ê Specify the country:
ASK IF D5=BORN OUTSIDE CANADA
D6: In what year did you first move to Canada?
YYYY
ADMISSIBLE RANGE: 1900-2020
D7:
What is the language you first learned at home as a child and still understand?
SELECT UP TO TWO
That concludes the survey. This survey was conducted on behalf of Health Canada. In the coming months the report will be available from Library and Archives Canada. We thank you very much for taking the time to answer this survey, it is greatly appreciated.
*The Ipsos online panel is recruited utilizing a double and triple opt-in screening process to ensure maximum return from an engaged and representative audience. After the initial opt-in to the Ipsos online panel, respondents are sent a second and third email to confirm they want to join our panel.