Executive Summary
IMPLEMENTATION OF THE WORLD HEALTH ORGANIZATION’S BEHAVIOURAL INSIGHTS
COVID-19 SURVEY TOOL IN THE CANADIAN CONTEXT: RAPID AND COST-EFFECTIVE
MONITORING OF PUBLIC PERCEPTIONS, KNOWLEDGE AND BEHAVIOURS
2020
Submitted to
Privy Council
Office (PCO)
Prepared by
Leger
Ce rapport est aussi disponible en français
Contract
number: 35035-192504/001/CY
POR #001-20
Awarded April
1, 2020
Contract
Value: $248,342.93 (including HST)
Delivery
Date: October 2020
Leger
507 Place d’Armes, Suite 700
Montréal, Québec
G1R 2K2
Phone: 514-982-2464
Fax:
514-987-1960
For more information
on this report, please contact the Privy Council Office at: Por-rop@pco-bcp.gc.ca
Ce rapport est aussi disponible en français.
IMPLEMENTATION OF THE WORLD HEALTH ORGANIZATION’S BEHAVIOURAL INSIGHTS
COVID-19 SURVEY TOOL IN THE CANADIAN CONTEXT: RAPID AND COST-EFFECTIVE
MONITORING OF PUBLIC PERCEPTIONS, KNOWLEDGE AND BEHAVIOURS
Executive Summary
Prepared for Privy Council Office of Canada
Supplier Name:
Leger
October 2020
This public opinion
research methodological report presents the technical aspects of a web survey
conducted by Leger Marketing Inc. on behalf of the Privy Council Office of Canada.
The research was conducted with Canadians 18 and over who could understand and
express themselves in either French or English.
Cette publication est aussi disponible en français sous le titre :
Mise en śuvre de l’outil de sondage de l’Organisation mondiale de la
santé sur les comportements liés ŕ la COVID-19 dans le contexte canadien : une
surveillance rapide et économique des perceptions, des connaissances et des
comportements du public.
This publication may be reproduced for non-commercial
purposes only. Prior written permission must be obtained from the Privy Council
Office. For more information on this report, please contact the Privy Council
Office at Por-rop@pco-bcp.gc.ca or at:
Privy Council Office
Blackburn Bldg
85 Sparks St.
Ottawa, Ontario K1A 0A3
Catalogue Number: CP22-195/2021E-PDF
International Standard Book Number (ISBN): 978-0-660-37572-4
Français - Numéro
de catalogue : CP22-195/2021F-PDF
Français - Numéro
international normalisé du livre (ISBN) : 978-0-660-37573-1
Related publications (registration number: POR 001-20)
© Her Majesty the Queen in
Right of Canada, as represented by the Privy Council Office of Canada, 2021.
Table of
Contents
1.2 Notes on The Interpretation of The Findings
1.3 Declaration of Political Neutrality and Contact
Information
Leger is
pleased to present The Privy Council Office of
Canada, as well as the Government of
Canada, with this technical report describing the methodology used for the
longitudinal study on public perceptions, knowledge and behaviours during the
COVID-19 pandemic.
This report
was prepared by Leger who was contracted by The Privy Council Office (contract
number 35035-192504/001/CY awarded April 1st, 2020. This contract has a value of
$248,342.93 including HST).
In the context of the COVID-19 pandemic, the Privy
Council Office (PCO), the secretariat responsible for providing nonpartisan advice
to the Prime Minister and coordinating the work of federal government
departments and agencies, needed ongoing access to quantitative data describing
the evolution of Canadians' perceptions, attitudes, knowledge, and behaviours
in this uncertain context. The Behavioural Insight Tool (BI), developed by the
World Health Organization (WHO), was used as the data collection tool for this
study. At each survey wave, the collection tool was adjusted to reflect the
evolution of the pandemic and the primary data needs of PCO.
The main objectives of this study were to provide PCO
and other government departments (e.g., Health Canada & Public Health
Agency of Canada) with research-based information on Canadians’ perceptions,
knowledge and behaviours relating to COVID-19.
This includes:
·
trust in
health authorities, recommendations, and information;
·
risk perceptions;
·
acceptance
of recommended behaviours;
·
knowledge;
·
barriers/drivers
to recommended behaviours;
·
misperceptions;
·
and
stigma.
The results of this public opinion study has been
and will be put to various uses by PCO and other federal departments and
agencies:
·
Track Canadians’
evolving perceptions, knowledge and behaviours relating to COVID-19, to better
understand how public awareness campaigns and broader government response
efforts have been affecting the population more holistically and to inform
whole-of-government decision-making moving forward;
·
Compare data trends
cross-nationally (i.e., with other allied countries that adopt the WHO BI
protocol) in a standardized manner, facilitating a better understanding how the
Canadian populace fares relative to other nations on an ongoing basis in their
perceptions, attitudes, and knowledge; and
·
Enable the Government of Canada to further develop and refine COVID-19
response efforts to meet the specific needs of Canadians. This study was seen as an element of broader public engagement efforts
in response to COVID-19 that intend to proactively and iteratively educate and
encourage Canadians to play their part in sustained response efforts.
Since a sample drawn from an Internet panel is
not probabilistic in nature, the margin of error cannot be calculated for this
survey. Respondents for this survey were selected from among those who have
volunteered to participate/registered to participate in online surveys. The
results of such surveys cannot be described as statistically projectable to the
target population. The data have been weighted to reflect the demographic composition
of the target population. Because the sample is based on those who initially
self-selected for participation, no estimates of sampling error can be
calculated.
I hereby certify, as chief agent of
Leger, that the deliverables are in full compliance with the neutrality
requirements of the Policy
on Communications and Federal Identity and the Directive on the Management of
Communications—Appendix C (Appendix C: Mandatory Procedures for Public Opinion Research).
Specifically, the deliverables do
not include information on electoral voting intentions, political party
preferences, party positions, or the assessment of the performance of a
political party or its leaders.
Signed by:
Christian Bourque
Executive
Vice President and Associate
Leger
507 Place d’Armes,
Suite 700
Montréal, Quebec
H2Y 2W8
cbourque@leger360.com
This study was conducted through a web-based survey of the Canadian
population aged 18 and over who can speak English or French. The respondents
were randomly selected through the Leo panel, our panel of Canadian Internet
users comprising nearly half a million Internet users.
Eight waves of studies were
completed for this research project. A sample of 2,000 respondents was
collected each wave. Since this was a longitudinal study, the objective was to
re-invite the 2,000 respondents from wave 1 in subsequent survey waves. Our
goal was to maximize the number of respondents who respond to the maximum
number of survey waves. Respondents who cease to participate in subsequent
waves were replaced following the gender, age and regional quotas that was be
implemented in the project.
All respondents were
contacted via email by Leger. All invitations were bilingual to ensure that no
respondent gets a unilingual invitation in the wrong official language. Each
respondent was provided with an invitation that includes preapproved
information from PCO regarding the nature of the research (i.e. Government of
Canada) including the required information for consent and the rights and
obligations of respondents.
Fieldwork
for this survey took place from April 10, 2020 to September 16, 2020. Details
of the eight waves are provided in the following sections.
A series of quotas
were implemented for this project. Quotas were cross-referenced by gender and
age groups and were also imposed on the region of residence of respondents. The
first quota is 50% men and 50% women for the gender sample. These gender quotas
were also respected within the following age groups: 18-34, 35-54 and 55 and
over. Those gender and age quotas had to be
respected at the regional level. The Canadian regions were split as follows:
● Atlantic Canada (Newfoundland, Prince
Edward Island, Nova Scotia, New Brunswick);
● Quebec;
● Ontario;
● Manitoba/Saskatchewan/Nunavut;
● Alberta/Northwest Territories;
● British Columbia/Yukon.
The following table
details the expected distribution of the sample across the provinces and
territories for each wave.
The sample
distribution was planned as follows:
Provinces and Territories |
NL |
NS |
PE |
NB |
QC |
ON |
MB |
SK |
AB |
BC |
NU |
NT |
YT |
# of Respondents |
40 |
80 |
22 |
60 |
400 |
620 |
126 |
106 |
260 |
280 |
2 |
2 |
2 |
As there were no
respondents from Nunavut, the two planned respondents were replaced by
respondents from Manitoba and Saskatchewan. As for any general population
sample derived from a national survey, the final results were weighted by
region, age groups, gender, education and the presence of children in the
household to make the final samples representative of the actual population of
Canada. Details on the weighting factors are presented in a subsequent section
of this report.
To validate the programming of the
questionnaire, a pre-test was conducted in both languages before each wave of
the project. The following table shows the details of those pre-tests. A
validation of frequencies and databases was done after each pre-test to ensure
that the programming was accurate and functional.
Table 1. Pretest details for waves 1 through 8
Wave
1 |
|
Date of the pre-test |
April 10th, 2020 |
Number of completed questionnaires |
46 |
Average length during pre-test |
17.34 min. |
Wave
2 |
|
Date of the pre-test |
April 21st,2020 |
Number of completed questionnaires |
31 |
Average length during pre-test |
12.25 min. |
Wave
3 |
|
Date of the pre-test |
May 5th, 2020 |
Number of completed questionnaires |
24 |
Average length during pre-test |
17.47 min. |
Wave
4 |
|
Date of the pre-test |
May 27th, 2020 |
Number of completed questionnaires |
34 |
Average length during pre-test |
15.04 min |
Wave
5 |
|
Date of the pre-test |
June 23rd, 2020 |
Number of completed questionnaires |
30 |
Average length during pre-test |
18.32 min |
Wave
6 |
|
Date of the pre-test |
July 17th, 2020 |
Number of completed questionnaires |
55 |
Average length during pre-test |
18.02 |
Wave
7 |
|
Date of the pre-test |
August 13th, 2020 |
Number of completed questionnaires |
36 |
Average length during pre-test |
16.17 min. |
Wave
8 |
|
Date of the pre-test |
September 10th, 2020 |
Number of completed questionnaires |
53 |
Average length during pre-test |
20.42 min |
Data collection for this project began in
April 2020 and was carried out over several waves until September 2020. Each
wave lasted between four and six days in field. A minimum target of 2,000
respondents for each wave was established. Following the first wave, Léger
conducted recontacts to maximize the number of respondents who participated in
previous waves and replaced respondents who ceased to participate in subsequent
waves following the gender, age and regional quotas. The first days of data
collection were aimed at recontacting previous respondents, while the last days
of data collection were aimed at replacing the non-returning respondents, due
to attrition. The following table details the collection dates and the number
of respondents and recontacts for each wave.
Table 2. Data collection details for waves 1 through 8
Wave
1 |
|
Start of data collection |
April 10th, 2020 |
End of data collection |
April 14th, 2020 |
Invitations sent |
19,123 |
Number of completed interviews |
2,023 |
Survey Length (Average) |
26 minutes |
Survey Length (Median) |
22 minutes |
Wave
2 |
|
Start of data collection |
April 21st, 2020 |
End of data collection |
April 25th, 2020 |
Invitations sent |
16,514 |
Recontact |
1,703 |
New respondents |
14,811 |
Number of completed interviews |
2,098 |
Recontact |
1,489 |
New respondents |
609 |
Survey Length (Average) |
23 minutes |
Survey Length (Median) |
20 minutes |
Wave
3 |
|
Start of data collection |
May 5th, 2020 |
End of data collection |
May 10th, 2020 |
Invitations sent |
8,012 |
Recontact |
2,002 |
New respondents |
6,010 |
Number of completed interviews |
2,000 |
Recontact |
1,733 |
New respondents |
267 |
Survey Length (Average) |
23 minutes |
Survey Length (Median) |
20 minutes |
Wave
4 |
|
Start of data collection |
May 27th, 2020 |
End of data collection |
June 1st,2020 |
Invitations sent |
12,380 |
Recontact |
1,987 |
New respondents |
10,393 |
Number of completed interviews |
2,152 |
Recontact |
1,702 |
New respondents |
450 |
Survey Length (Average) |
25 minutes |
Survey Length (Median) |
21 minutes |
Wave
5 |
|
Start of data collection |
June 23rd, 2020 |
End of data collection |
June 28th, 2020 |
Invitations sent |
15,880 |
Recontact |
2,128 |
New respondents |
13,752 |
Number of completed interviews |
2,169 |
Recontact |
1,847 |
New respondents |
322 |
Survey Length (Average) |
23 minutes |
Survey Length (Median) |
20 minutes |
Wave
6 |
|
Start of data collection |
July 17th, 2020 |
End of data collection |
July 22nd, 2020 |
Invitations sent |
7,408 |
Recontact |
2,169 |
New respondents |
5,239 |
Number of completed interviews |
2,141 |
Recontact |
1,885 |
New respondents |
256 |
Survey Length (Average) |
23 minutes |
Survey Length (Median) |
20 minutes |
Wave
7 |
|
Start of data collection |
August 13th, 2020 |
End of data collection |
August 17th,2020 |
Invitations sent |
9,857 |
Recontact |
2,141 |
New respondents |
7,716 |
Number of completed interviews |
2,129 |
Recontact |
1,776 |
New respondents |
353 |
Survey Length (Average) |
24 minutes |
Survey Length (Median) |
21 minutes |
Wave
8 |
|
Start of data collection |
September 10th,2020 |
End of data collection |
September 16th,2020 |
Invitations sent |
9,442 |
Recontact |
2,129 |
New respondents |
7,313 |
Number of completed interviews |
2,117 |
Recontact |
1,822 |
New respondents |
295 |
Survey Length (Average) |
29 minutes |
Survey Length (Median) |
24 minutes |
Below is the
calculation of the participation rate to the web survey for all eight waves. The participation rate is
calculated using the following formula: Participation rate / response rate = R
÷ (U + IS + R). The table below provides details of the calculation. For all
waves, the participation rate ranges between 11,4% and 30%.
Table 3. Participation rate for wave 1
Base
Sample |
19,123 |
Invalid cases |
0 |
Invitations mistakenly sent to people who did not qualify for the
study |
0 |
Incomplete or missing email addresses |
0 |
Unresolved (U) |
16,255 |
Email invitations bounce back |
0 |
Email invitations unanswered |
16,255 |
EFFECTIVE SAMPLE* |
2,319 |
In-scope non-responding units (IS) |
211 |
Non-response from eligible respondents |
0 |
Respondent refusals |
0 |
Language problem |
0 |
Selected respondent not available (illness; leave of absence;
vacation; other) |
0 |
Early break-offs |
221 |
Responding units (R) |
2,108 |
Completed surveys disqualified – quota filled |
0 |
Completed surveys disqualified for other reasons |
85 |
COMPLETED INTERVIEWS |
2,023 |
Participation rate |
11.4% |
Table 4. Participation rate for wave 2
Base
Sample |
16,514 |
Invalid cases |
0 |
Invitations mistakenly sent to people who did not qualify for the
study |
0 |
Incomplete or missing email addresses |
0 |
Unresolved (U) |
13,907 |
Email invitations bounce back |
0 |
Email invitations unanswered |
13,907 |
EFFECTIVE SAMPLE* |
2,221 |
In-scope non-responding units (IS) |
78 |
Non-response from eligible respondents |
0 |
Respondent refusals |
0 |
Language problem |
0 |
Selected respondent not available (illness; leave of absence;
vacation; other) |
0 |
Early break-offs |
78 |
Responding units (R) |
2,143 |
Completed surveys disqualified – quota filled |
0 |
Completed surveys disqualified for other reasons |
45 |
COMPLETED INTERVIEWS |
2,098 |
Participation rate |
13.3% |
Table 5. Participation rate for wave 3
Base
Sample |
8,012 |
Invalid cases |
0 |
Invitations mistakenly sent to people who did not qualify for the
study |
0 |
Incomplete or missing email addresses |
0 |
Unresolved (U) |
5,551 |
Email invitations bounce back |
0 |
Email invitations unanswered |
5,551 |
EFFECTIVE SAMPLE* |
2,087 |
In-scope non-responding units (IS) |
57 |
Non-response from eligible respondents |
0 |
Respondent refusals |
0 |
Language problem |
0 |
Selected respondent not available (illness; leave of absence;
vacation; other) |
0 |
Early break-offs |
57 |
Responding units (R) |
2,030 |
Completed surveys disqualified – quota filled |
0 |
Completed surveys disqualified for other reasons |
30 |
COMPLETED INTERVIEWS |
2,000 |
Participation rate |
21% |
Table 6. Participation rate for wave 4
Base
Sample |
12,380 |
Invalid cases |
0 |
Invitations mistakenly sent to people who did not qualify for the
study |
0 |
Incomplete or missing email addresses |
0 |
Unresolved (U) |
9,580 |
Email invitations bounce back |
0 |
Email invitations unanswered |
9,580 |
EFFECTIVE SAMPLE* |
2,317 |
In-scope non-responding units (IS) |
127 |
Non-response from eligible respondents |
0 |
Respondent refusals |
0 |
Language problem |
0 |
Selected respondent not available (illness; leave of absence;
vacation; other) |
0 |
Early break-offs |
127 |
Responding units (R) |
2,190 |
Completed surveys disqualified – quota filled |
0 |
Completed surveys disqualified for other reasons |
38 |
COMPLETED INTERVIEWS |
2,152 |
Participation rate |
18.4% |
Table 7. Participation rate for wave 5
Base
Sample |
15,880 |
Invalid cases |
0 |
Invitations mistakenly sent to people who did not qualify for the
study |
0 |
Incomplete or missing email addresses |
0 |
Unresolved (U) |
12,409 |
Email invitations bounce back |
0 |
Email invitations unanswered |
12,409 |
EFFECTIVE SAMPLE* |
2,324 |
In-scope non-responding units (IS) |
130 |
Non-response from eligible respondents |
0 |
Respondent refusals |
0 |
Language problem |
0 |
Selected respondent not available (illness; leave of absence;
vacation; other) |
0 |
Early break-offs |
130 |
Responding units (R) |
2,194 |
Completed surveys disqualified – quota filled |
0 |
Completed surveys disqualified for other reasons |
25 |
COMPLETED INTERVIEWS |
2,169 |
Participation rate |
15% |
Table 8. Participation rate for wave 6
Base
Sample |
7,408 |
Invalid cases |
0 |
Invitations mistakenly sent to people who did not qualify for the
study |
0 |
Incomplete or missing email addresses |
0 |
Unresolved (U) |
4,983 |
Email invitations bounce back |
0 |
Email invitations unanswered |
4,983 |
EFFECTIVE SAMPLE* |
1,314 |
In-scope non-responding units (IS) |
113 |
Non-response from eligible respondents |
0 |
Respondent refusals |
0 |
Language problem |
0 |
Selected respondent not available (illness; leave of absence;
vacation; other) |
0 |
Early break-offs |
113 |
Responding units (R) |
2,201 |
Completed surveys disqualified – quota filled |
0 |
Completed surveys disqualified for other reasons |
60 |
COMPLETED INTERVIEWS |
2,141 |
Participation rate |
30% |
Table 9. Participation rate for wave 7
Base
Sample |
9,857 |
Invalid cases |
0 |
Invitations mistakenly sent to people who did not qualify for the
study |
0 |
Incomplete or missing email addresses |
0 |
Unresolved (U) |
7,309 |
Email invitations bounce back |
0 |
Email invitations unanswered |
7,309 |
EFFECTIVE SAMPLE* |
2,328 |
In-scope non-responding units (IS) |
165 |
Non-response from eligible respondents |
0 |
Respondent refusals |
0 |
Language problem |
0 |
Selected respondent not available (illness; leave of absence;
vacation; other) |
0 |
Early break-offs |
165 |
Responding units (R) |
2,163 |
Completed surveys disqualified – quota filled |
0 |
Completed surveys disqualified for other reasons |
34 |
COMPLETED INTERVIEWS |
2,129 |
Participation rate |
22% |
Table 10. Participation rate for wave 8
Base
Sample |
9,442 |
Invalid cases |
0 |
Invitations mistakenly sent to people who did not qualify for the
study |
0 |
Incomplete or missing email addresses |
0 |
Unresolved (U) |
7,007 |
Email invitations bounce back |
0 |
Email invitations unanswered |
7,007 |
EFFECTIVE SAMPLE* |
2,365 |
In-scope non-responding units (IS) |
2,252 |
Non-response from eligible respondents |
0 |
Respondent refusals |
0 |
Language problem |
0 |
Selected respondent not available (illness; leave of absence;
vacation; other) |
0 |
Early break-offs |
113 |
Responding units (R) |
2,139 |
Completed surveys disqualified – quota filled |
0 |
Completed surveys disqualified for other reasons |
22 |
COMPLETED INTERVIEWS |
2,117 |
Participation rate |
19% |