Supplier name: Ipsos Public Affairs
Contract Number: HT372-174272/001/CY
Contract vaue: $49,941.48
Award Date: 2018-02-16
Delivery Date: February 2019
Methodological Report
Prepared for Health Canada
Registration number: POR 105-17
For more information on this
report, please contact:
hc.cpab.por-rop.dgcap.sc@canada.ca
Ce sommaire est aussi
disponible en français.
POLITICAL
NEUTRALITY STATEMENT
I hereby certify as Senior Officer of Ipsos that
the deliverables fully comply with the Government of Canada political
neutrality requirements outlined in the Communications Policy of the Government
of Canada and Procedures for Planning and Contracting Public Opinion Research.
Specifically, the deliverables do not include information on electoral voting
intentions, political party preferences, standings
with the electorate or ratings of the performance of a political party or its leaders.
Mike Colledge
President
Ipsos Public Affairs
Prepared for Public Health Agency of Canada
Supplier name: Ipsos Public Affairs
February 2019
This public opinion research report presents the results of baseline and post-campaign ACET online surveys conducted by Ipsos Public Affairs on behalf of Public Health Agency of Canada. The research study was conducted with 1603 "vaccine-hesistant" Canadians aged 18 and over in total, 881 in the baseline survey and 722 in the post-campaign survey. The research focused on three groups of vaccine hesistant Canadians - parents of children aged 6 and under, women who are currently pregnant and women who are planning to become pregnant in the next 12 months. The baseline survey was conducted between March 8th and 25th, 2018 and the post-campaign was conducted between December 10th, 2018 and January 11th, 2019.
This publication may be reproducted for non-commercial purposes only. Prior written permission must be obtatined from Public Health Agency of Canada. For more information on this report, please contact Public Health Agency of Canada at: hc.cpab.por-rop.dgcap.sc@canada.ca
Executive Summary
1. Background
1.1 Summary Statement
2. Purpose of the Research
2.1 Research Objectives
Appendices
Participant Recruitment
Sample Source
Incentives
Sample Details
Details Related to Target
Population
Description of Sample Frame
Case Dispositions
Survey Instrument
The research consisted of quantitative online surveys, a baseline and a post-campaign survey with a sample of vaccine-hesitant Canadians aged 18 and over. Vaccine-hesitant was defined as those who vaccinate late or selectively, and acceptors who have doubts about their decision. The research focused on three groups of vaccine hesistant Canadians - parents of children agred 6 and under, women who are currently pregnant and women who are planning to become pregnant in the next 12 months. In total, n=881 online surveys were conducted in the baseline and n=722 in the post-campaign survey. Below is a breakdown of the three groups for each survey:
|
Baseline
Survey |
Post-Campaign
Survey |
|
|
Total
|
881 |
722 |
|
Parents
of children 6 and under |
688 |
611 |
|
Women,
currently pregnant |
52 |
24 |
|
Women,
planning pregnancy in next 12 months |
141 |
87 |
The surveys were conducted in English and French with the pre-campaign taking place between March 8th and 25th, 2018 and the post-campaign survey taking place between December 10, 2018 and January 11, 2019.
As these surveys were not among the general population, and the true proportions of vaccine-hesitant Canadians is unknown, weighting was not employed for either survey.
Ipsos programmed, hosted and provided sample
management services only, while the Public Health Agency of Canada provided the
online survey. Ipsos was responsible for data
collection and data storage in Canada, data processing, and data weighting. The
total contract value of this research was $49,941.48 including HST.
The purpose of this research is to assess the effectiveness of the advertising campaign on the vaccine hesitant targets. Specific research objectives included:
The research may also be used to uncover what attitudinal changes the
Public Health Agency of Canada may need to continue to work to change in order
increase vaccination rates among this target group.
The Ipsos i-Say panel is the primary source for recruiting respondents. Where necessary, a supplemental sample from third party vendor panels is also used as a source. The Ipsos i-Say online panel is a diversely-sourced and actively maintained panel of over 200,000 Canadian adults. Utilizing this panel allows us to confidently build online samples based on known population characteristics like the ones available through Census data. Invitations to participate in each survey are adjusted to reflect the most recent Current Population Data provided by Statistics Canada.
Online panel participants are recruited and screened using rigorous double and triple opt-in checks1 to ensure that respondents want to receive contact such as invitations to participate in research surveys. Ipsos i-Say panel Terms and Conditions of Membership details the respondents' rights to, at any time, for any reason ask to be removed from the panel and stop receiving emails from Ipsos and its clients.
Panel recruitment consists of the following steps:
It is important to note that any panel - no matter how well recruited - does not enable random selection in the sense that not all members of the public are apt to join a panel. This applies to panels recruited through online approaches as well as panels recruited via traditional telephone Random Digit Dialing.
Rigorous panel maintenance procedures mean only those participants who actively participate in online surveys, at least once every six months, are included. Participants can choose whether to enter a survey or not. While participating in a survey, the respondent can simply exit the survey by closing the survey Internet browser window at any time without submitting their responses if they no longer want to participate.
In order to capture responses from low incidence individuals, such as
vaccine-hesitant women who are currently pregnant, we use what is called a Webcruiser on our iSay Panel. A Webcruiser asks a
set number of questions to our entire panel and typically takes approximately
three weeks to complete. Panelists who meet the criteria we are looking for are then flagged and can be contacted for further research.
To ensure a high participation rate from our panelists, the
i-Say panel provides a number of innovative
incentive programs. Ipsos' i-Say
online panel participants are incentivized through a
points program whereby respondents collect points that can be redeemed for
rewards of their choice from a collection of products from online retailers
such as gift cards and prizes. Each incentive is tailored to the specific
requirements of each survey, depending on the length of the survey, the subject
matter of the study, and the time required to complete
a minimum number of interviews.
A sampling frame was devised to reach n=700 vaccine parents of children 6
and under (with a skew toward women), n=150 vaccine-hesitant women who are
currently pregnant and n=150 vaccine hesitant women who are planning to become
pregnant in the next 12 months. In order to qualify, respondents had to be aged
18 and over, and fall into one of these three vaccine-hesitant groups. Due to
the difficulty of reaching these low incidence groups, these sample quotas were
not met. As mentioned, as the true proportions of vaccine-hesitant Canadians is
unknown, we did not put hard quotas on age or region, but instead used loose
quotas in order to ensure we reached Canadians across the country.
The Ipsos i-Say online panel consists of over 200,000 Canadian adults recruited through diverse sources and is constantly refreshed. Background data gathered when respondents join the panel allows us to finely target respondents on the basis of key criteria such as region (national samples are typically divided into over 40 submarkets nationwide ensuring accurate distributions within provinces), age, and gender. The results of quota based sampling against these criteria allow us to construct samples that are reflective of Canadian population characteristics. In this instance, sampling quotas were set according to the respondents' region, gender, and age to match a target profile based on Census 2016 data. This serves to underpin the representativeness of the survey results.
All sample surveys and polls may be subject to sources of error, including, but not limited to coverage error and measurement error. Traditional margins of error don't apply to online panel-based samples of general public audiences as the traditional margin of error calculation assumes that the sample measured is a probability sample. As there is no way to measure the probability that any given member of the public may join or be a part of an online panel, these samples are inherently non-probabilistic.
Non-Response Analysis: There exists within the current sample the possibility of non-response bias. In particular, this survey would not include members of the population who do not have access to the Internet or who are not capable of responding to a survey in either English or French.
Extrapolation: The table below shows the unweighted sample size and proportions by vaccine hesitant sub-group, as well as geographical and demographic distribution of the sample.
|
Baseline Survey |
Post-Campaign Survey |
|||
|
Sample Size |
Sample Proportions |
Sample Size |
Sample Proportions |
|
|
Total |
881 |
100% |
722 |
100% |
|
Vaccine-Hesitant Sub-Group |
||||
|
Parents
of children 6 and under |
688 |
78% |
611 |
85% |
|
Women,
currently pregnant |
52 |
6% |
24 |
3% |
|
Women,
planning pregnancy in next 12 months |
141 |
16% |
87 |
12% |
|
Region |
||||
|
British
Columbia |
99 |
11% |
72 |
10% |
|
Alberta |
100 |
11% |
78 |
11% |
|
Saskatchewan |
30 |
3% |
17 |
2% |
|
Manitoba |
27 |
3% |
23 |
3% |
|
Ontario |
342 |
39% |
274 |
38% |
|
Quebec |
229 |
26% |
218 |
30% |
|
Newfoundland
and Labrador |
8 |
1% |
6 |
1% |
|
Prince
Edward Island |
6 |
1% |
5 |
1% |
|
Nova
Scotia |
20 |
2% |
10 |
1% |
|
New
Brunswick |
20 |
2% |
19 |
3% |
|
Gender |
||||
|
Male |
145 |
16% |
118 |
16% |
|
Female |
736 |
84% |
604 |
84% |
|
Age |
||||
|
18-24 |
80 |
9% |
32 |
4% |
|
25-34 |
409 |
46% |
307 |
43% |
|
35-39 |
227 |
26% |
221 |
31% |
|
40-45 |
112 |
14% |
123 |
17% |
|
46+ |
53 |
5% |
39% |
5% |
The following table provides the response dispositions and response rate calculation, as per the MRIA's empirical method of calculating response rates for online surveys.
|
Calculation for Data
Collection2 |
Baseline Survey |
Post-Campaign Survey |
|
Total
Email Invitations Issued |
33,355 |
31,666 |
|
Unresolved
(U) (no response) |
23,050 |
26,761 |
|
In-scope
- non-responding (IS) - Qualified respondent break-off (incomplete)> |
330 |
128 |
|
In-scope
- Responding units (R) |
9,975 |
4,777 |
|
Over
quota |
340 |
29 |
|
Other
disqualified |
8,754 |
4,061 |
|
Completed
Interviews |
881 |
722 |
|
Response
Rate = R/(U+IS+R) |
29.9% |
15% |
Sampling is carried out via a proprietary sampling application that allows us to construct complex samples based on the target and screening requirements. To develop the survey sample and minimize non-response, we use a large and diversely sourced panel. In addition, we have the capability of broadening the sample frame further through the inclusion of other actively managed panel resources as needed to ensure the sample requirements are met among the hardest-to-reach audiences (such as younger males).
This drop in response rate is likely laregely due
to the field period being over the Christmas holidays.
'Unresolved (U)' indicates no response received to email invitation; 'In-scope - non-responding (IS)' indicates qualified respondents who started but did not complete the survey (incomplete); 'In-scope - responding units (R)' indicates qualified respondents who started the survey but were disqualified as over-quota or disqualified for other reasons. Valid completes are counted as 'Completed interviews'.
Baseline Survey
YEAR/MONTH. What is your date of birth?
RESP_GENDER_ca. What is your sex?
Country10 (hidden question) . In which country do you live?
QMktSize_CA. REQUIRED What is your postal code? (example: A8A8A8)
SCREENER QUESTION TO IDENTIFY IF HAVE CHILDREN UNDER AGE 7 (SKEW FEMALE 70%)
S1. Are you the parent or legal guardian of a child age 6 years old or younger?
Yes (please enter number of children) [NUMERIC TEXT BOX]
No
[IF MALE AND S1=NO, TERMINATE]
S2. Please enter the age of your child(ren) below. If your child is under 1 years old, please enter 0
Child 1 [NUMERIC TEXT BOX, RANGE 0-6]
Child 2 [NUMERIC TEXT BOX, RANGE 0-6]
[INSERT CHILD BOXES FOR NUMBER OF CHILDREN AT S1, CHILD 1, CHILD 2, CHILD 3, ETC.]
SCREENER QUESTIONS TO IDENTIFY IF PREGNANT OR THINKING OF GETTING PREGNANT IN NEXT 12 MONTHS
[IF FEMALE AND S1=NO, ASK S3]
S3. Are you currently pregnant?
Yes
No
[ASK S4 IF S3=NO]
S4. Are you expecting to become pregnant within the next 12 months?
Yes
No
[IF NO TO BOTH S3 AND S4 TERMINATE]
S5. Which of the following statements most accurately reflects your views on vaccines for your child(ren)?
I (will) accept all recommended vaccines and have no
doubts or concerns [THANK AND TERMINATE]
I (will) accept all recommended vaccines but have doubts and concerns
I (will or have) refuse(d) or delay(ed) getting some
vaccines
I have not decided yet about plans for vaccinating my child
I will or have refuse(d) all recommended vaccines [THANK AND TERMINATE]
Don't know/ [THANK AND TERMINATE]
GROUPING
RESPONDENTS SHOULD ONLY COUNT IN ONE GROUP
[MQB]
Thank you for taking the time to complete this survey dealing with current issues of interest to Canadians.
Your participation is voluntary and your responses will be kept entirely confidential. The survey takes about 10 minutes to complete and is registered with the Marketing Research and Intelligence Association (MRIA).
CORE QUESTIONS
[ASK ALL]
Q1. Over the past three weeks, have you seen, read or heard any advertising from the Government of Canada?
Yes
No
[IF Q1=NO, SKIP TO T1A, IF YES, CONTINUE]
Q2. Think about the most recent ad from the Government of Canada that comes to mind. Where have you seen, read or heard this ad?
[RANDOMIZE]
Cinema
Facebook
Internet website
Magazines
Newspaper (daily)
Newspaper (weekly or community)
Outdoor billboards
Pamphlet or brochure in the mail
Public transit (bus or subway)
Radio
Television
Twitter
YouTube
Instagram
LinkedIn
Other [SPECIFY][ANCHOR]
Q3. What do you remember about this ad?
[OPEN END]
Q4. How did you know that it was an ad from the Government of Canada?
[OPEN END]
CAMPAIGN SPECIFIC QUESTIONS
[ASK ALL]
T1A. Over the past three weeks, have you seen, read or heard any Government of Canada advertising about routine childhood vaccinations?
Recommended childhood immunizations include routine vaccines to prevent tetanus, diphtheria, pertussis (whooping cough), polio and measles, mumps, rubella (MMR) as well as haemophilus influenza type B (Hib). They also include recommended vaccines to prevent chicken pox (varicella), but do not include vaccines you get to prevent flu or in advance of travelling.
Yes
No
[IF T1A= NO, SKIP TO T1D, IF YES, CONTINUE]
T1B. Where have you seen, read or heard this Government of Canada ad about childhood vaccination?
[RANDOMIZE]
Cinema
Facebook
Internet website
Magazines
Newspaper (daily)
Newspaper (weekly or community)
Outdoor billboards
Pamphlet or brochure in the mail
Public transit (bus or subway)
Radio
Television
Twitter
YouTube
Instagram
LinkedIn
Other [SPECIFY][ANCHOR]
T1C. What do you remember about this ad?
[OPEN END]
[ASK ALL]
T1D. On a scale of 1 to 5 where 1 is not at all, 5 is completely, and the midpoint 3 is moderately, how much do you trust recommended childhood vaccinations?
1 Not at all
2
3 Moderately
4
5 Completely
Don't know/no response
T1E. In the past 12 months, have you looked for information about childhood vaccinations?
Yes
No
Don't recall
Don't know
T1F. How much do you agree or disagree with each of the following statements, in general…
[COLUMNS]
Strongly agree
Somewhat agree
Somewhat disagree
Strongly disagree
Don't know
Prefer not to answer
[ROWS, RANDOMIZE]
Childhood vaccines are safe.
Childhood vaccines are effective.
Childhood vaccines are important for your child's health.
DEMOGRAPHICS
D1. Which of the following categories describes your employment status? Are
you…Select one only.
Working full-time (35 or more hours per week)
Working part-time (less than 35 hours per week)
Self-employed
Unemployed, but looking for work
A student attending school full-time
Retired
Stay at home (Full-time homemaker or unemployed but not looking for work)
Other employment status
D2. What is the highest level of formal education that you have completed? Select one only
Grade 8 or less
Some high school
High school diploma or equivalent
Registered Apprenticeship or other trades certificate or diploma
College, CEGEP or other non-university certificate or diploma
University certificate or diploma below bachelor's level
Bachelor's degree
Postgraduate degree above bachelor's level
D3. Which of the following categories best describes your total annual household income, including income from all household members, before taxes are deducted?
Under
$20,000
Between $20,000 and $40,000
Between $40,000 and $60,000
Between $60,000 and $80,000
Between $80,000 and $100,000
Between $100,000 and $150,000
$150,000 and above
Prefer not to say
D4. Were you born in Canada?
Born in Canada
Born outside of Canada, specify the country [SPECIFY]
[ASK D5 IF D4=BORN OUTSIDE]
D5. In what year did you first move to Canada?
[NUMERIC TEXT BOX, 4 DIGITS, "YYYY" UNDERNEATH, RANGE: 1900 - 2018]
D6. What is the language you first learned at home as a child and still understand?
English
French
Other language [SPECIFY]
That concludes the survey. This survey was conducted on behalf of the Public Health Agency of Canada. In the coming months the report will be available from Library and Archives Canada. We thank you very much for taking the time to answer this survey, it is greatly appreciated.
Post-Campaign Survey
YEAR/MONTH. What is your date of birth?
RESP_GENDER_ca. What is your sex?
Country10 (hidden question) . In which country do you live?
QMktSize_CA. REQUIRED What is your postal code? (example: A8A8A8)
SCREENER QUESTION TO IDENTIFY IF HAVE CHILDREN UNDER AGE 7 (SKEW FEMALE 70%)
S1. Are you the parent or legal guardian of a child age 6 years old or younger?
Yes (please enter number of children) [NUMERIC TEXT BOX]
No
[IF MALE AND S1=NO, TERMINATE]
S2. Please enter the age of your child(ren) below. If your child is under 1 years old, please enter 0
Child 1 [NUMERIC TEXT BOX, RANGE 0-6]
>Child 2 [NUMERIC TEXT BOX, RANGE 0-6]
[INSERT CHILD BOXES FOR NUMBER OF CHILDREN AT S1, CHILD 1, CHILD 2, CHILD 3, ETC.]
SCREENER QUESTIONS TO IDENTIFY IF PREGNANT OR THINKING OF GETTING PREGNANT IN NEXT 12 MONTHS
[IF FEMALE AND S1=NO, ASK S3]
S3. Are you currently pregnant?
Yes
No
[ASK S4 IF S3=NO]
S4. Are you expecting to become pregnant within the next 12 months?
Yes
No
[IF NO TO BOTH S3 AND S4 TERMINATE]
S5. Which of the following statements most accurately reflects your views on vaccines for your child(ren)?
I (will) accept all recommended vaccines and have no
doubts or concerns [THANK AND TERMINATE]
I (will) accept all recommended vaccines but have doubts and concerns
I (will or have) refuse(d) or delay(ed) getting some
vaccines
I have not decided yet about plans for vaccinating my child
I will or have refuse(d) all recommended vaccines [THANK AND TERMINATE]
Don’t know/ [THANK AND TERMINATE]
GROUPING
RESPONDENTS SHOULD ONLY COUNT IN ONE GROUP
Thank you for taking the time to complete this survey dealing with current issues of interest to Canadians.
Your participation is voluntary and your responses will be kept entirely confidential. The survey takes about 10 minutes to complete.
[MQB]
ASK ALL RESPONDENTS
Q1:
Over the past three weeks, have you seen, read or heard any advertising from the Government of Canada?
[IF YES, CONTINUE, IF NO, SKIP TO T1A]
Q2:
Think about the most recent ad from the Government of Canada that comes to mind. Where have you seen, read or heard this ad?
RANDOMIZE, ANCHOR OTHER, SELECT ALL THAT APPLY
Q3:
What do you remember about this ad?
Q4:
How did you know that it was an ad from the Government of Canada?
ASK ALL RESPONDENTS
T1A:
Over the past three weeks, have you seen, read or heard any Government of
Canada advertising about routine childhood vaccinations?
Recommended childhood immunizations include routine vaccines to prevent tetanus, diphtheria, pertussis (whooping cough), polio and measles, mumps, rubella (MMR) as well as haemophilus influenza type B (Hib). They also include recommended vaccines to prevent chicken pox (varicella), but do not include vaccines you get to prevent flu or in advance of travelling.
[IF YES, CONTINUE, IF NO, SKIP TO T1D]
T1B:
Where have you seen, read or heard this ad about routine childhood
vaccinations?
RANDOMIZE, ANCHOR OTHER, SELECT ALL THAT APPLY
T1C:
What do you remember about this ad?
ASK ALL RESPONDENTS
T1D:
On a scale of 1 to 5 where 1 is not at all, 5 is completely, and the midpoint 3
is moderately, how much do you trust recommended childhood vaccinations?
1 Not at all
2
3 Moderately
4
5 Completely
Don’t know
T1E:
In the past 12 months, have you looked for information about childhood
vaccinations?
Yes
No
Don’t recall
Don’t know
T1F:
How much do you agree or disagree with each of the following statements, in
general…
[COLUMNS]
Strongly agree
Somewhat agree
Somewhat disagree
Strongly disagree
Don’t know
Prefer not to answer
[ROWS, RANDOMIZE]
Childhood vaccines are safe.
Childhood vaccines are effective.
Childhood vaccines are important for your child's health.
T1H:
Here are some ads that have recently been broadcast on various media. Please make sure your sound is on and
click next to see the ads.
[INSERT VIDEO, OOH AND DIGITAL ADS]
[CLICK TO GO TO THE NEXT PAGE]
Over the past few months, have you seen, read or heard any of these ads?
[IF YES, CONTINUE, IF NO, SKIP TO T1J]
T1I:
Where have you seen, read or heard these ads?
RANDOMIZE, ANCHOR OTHER, SELECT ALL THAT APPLY
T1J:
What do you think is the main point these ads are
trying to get across?
T1K:
Please indicate your level of agreement with the following statements about
these ads?
RANDOMIZE STATEMENTS
|
|
1 |
2 |
3 |
4 |
5 |
|
These
ads catch my attention |
o |
o |
o |
o |
o |
|
These
ads are relevant to me |
o |
o |
o |
o |
o |
|
These
ads are difficult to follow |
o |
o |
o |
o |
o |
|
These
ads does not favour one political party over another |
o |
o |
o |
o |
o |
|
These
ads talk about an important topic |
o |
o |
o |
o |
o |
|
These
ads provide new information |
o |
o |
o |
o |
o |
|
These
ads clearly convey that the Government of Canada can provide answers to
parents with questions on vaccination. |
o |
o |
o |
o |
o |
|
These
ads made me more likely to think that childhood vaccination is important |
o |
o |
o |
o |
o |
|
These
ads made me feel more comfortable about childhood vaccination |
o |
o |
o |
o |
o |
T1L:
Did you do any of the following as a result of seeing the ad(s)? Please select
all that apply
Visited the Canada.ca/ChildhoodVaccines website
Discussed vaccination with my health professional
Discussed vaccination with a friend or family member
Shared the information
Decided to vaccinate my child
Looked for vaccination information online
Thought more about vaccinations for my child(ren)
Other, specify ___________________
I didn’t do anything as a result of seeing the ad
D1:
Which of the following categories best describes your current employment
status? Are you…
SELECT ONE ONLY
D2:
What is the highest level of formal education that you have completed?
SELECT ONE ONLY
D3:
Which of the following categories best describes your total annual household
income, including income from all household members, before taxes are deducted?
SELECT ONE ONLY
D4:
Where were you born?
ASK IF D4=BORN OUTSIDE CANADA
D5:
In what year did you first move to Canada?
YYYY
ADMISSIBLE RANGE: 1900-2016
D6:
What is the language you first learned at home as a child and still understand?
SELECT UP TO TWO
That concludes the survey. This survey was conducted on behalf of the Public Health Agency of Canada. In the coming months the report will be available from Library and Archives Canada. We thank you very much for taking the time to answer this survey, it is greatly appreciated.