Executive Summary

IMPLEMENTATION OF THE WORLD HEALTH ORGANIZATION’S BEHAVIOURAL INSIGHTS COVID-19 SURVEY TOOL IN THE CANADIAN CONTEXT: RAPID AND COST-EFFECTIVE MONITORING OF PUBLIC PERCEPTIONS, KNOWLEDGE AND BEHAVIOURS

2020

Submitted to

Privy Council Office (PCO)

Prepared by

Leger

Ce rapport est aussi disponible en français

Contract number: 35035-192504/001/CY

POR #001-20

Awarded April 1, 2020

Contract Value: $248,342.93 (including HST)

Delivery Date: October 2020

Leger

507 Place d’Armes, Suite 700

Montréal, Québec

G1R 2K2

Phone: 514-982-2464

Fax: 514-987-1960

For more information on this report, please contact the Privy Council Office at: Por-rop@pco-bcp.gc.ca

Ce rapport est aussi disponible en français.

Executive Summary

Prepared for Privy Council Office of Canada

Supplier Name: Leger

October 2020

This public opinion research methodological report presents the technical aspects of a web survey conducted by Leger Marketing Inc. on behalf of the Privy Council Office of Canada. The research was conducted with Canadians 18 and over who could understand and express themselves in either French or English.

Cette publication est aussi disponible en français sous le titre :

Mise en œuvre de l’outil de sondage de l’Organisation mondiale de la santé sur les comportements liés à la COVID-19 dans le contexte canadien : une surveillance rapide et économique des perceptions, des connaissances et des comportements du public.

This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from the Privy Council Office. For more information on this report, please contact the Privy Council Office at Por-rop@pco-bcp.gc.ca or at:

Privy Council Office

Blackburn Bldg

85 Sparks St.

Ottawa, Ontario K1A 0A3

Catalogue Number: CP22-195/2021E-PDF

International Standard Book Number (ISBN): 978-0-660-37572-4
Français - Numéro de catalogue : CP22-195/2021F-PDF

Français - Numéro international normalisé du livre (ISBN) : 978-0-660-37573-1

Related publications (registration number: POR 001-20)

Table of Contents

1. Executive Summary 4

1.1 Background and Objectives 4

1.2 Notes on The Interpretation of The Findings 5

1.3 Declaration of Political Neutrality and Contact Information 5

1.4 Methodology 6

1.5 Quotas 6

2. Details for Each Wave 7

2.1 Pretest 7

2.2 Data collection 9

2.3. Participation rate 12

1. Executive Summary

Leger is pleased to present The Privy Council Office of Canada, as well as the Government of Canada, with this technical report describing the methodology used for the longitudinal study on public perceptions, knowledge and behaviours during the COVID-19 pandemic.

This report was prepared by Leger who was contracted by The Privy Council Office (contract number 35035-192504/001/CY awarded April 1^st, 2020. This contract has a value of $248,342.93 including HST).

1.1 Background and Objectives

In the context of the COVID-19 pandemic, the Privy Council Office (PCO), the secretariat responsible for providing nonpartisan advice to the Prime Minister and coordinating the work of federal government departments and agencies, needed ongoing access to quantitative data describing the evolution of Canadians' perceptions, attitudes, knowledge, and behaviours in this uncertain context. The Behavioural Insight Tool (BI), developed by the World Health Organization (WHO), was used as the data collection tool for this study. At each survey wave, the collection tool was adjusted to reflect the evolution of the pandemic and the primary data needs of PCO.

The main objectives of this study were to provide PCO and other government departments (e.g., Health Canada & Public Health Agency of Canada) with research-based information on Canadians’ perceptions, knowledge and behaviours relating to COVID-19.

This includes:

· trust in health authorities, recommendations, and information;

· risk perceptions;

· acceptance of recommended behaviours;

· knowledge;

· barriers/drivers to recommended behaviours;

· misperceptions;

· and stigma.

The results of this public opinion study has been and will be put to various uses by PCO and other federal departments and agencies:

· Track Canadians’ evolving perceptions, knowledge and behaviours relating to COVID-19, to better understand how public awareness campaigns and broader government response efforts have been affecting the population more holistically and to inform whole-of-government decision-making moving forward;

· Compare data trends cross-nationally (i.e., with other allied countries that adopt the WHO BI protocol) in a standardized manner, facilitating a better understanding how the Canadian populace fares relative to other nations on an ongoing basis in their perceptions, attitudes, and knowledge; and

· Enable the Government of Canada to further develop and refine COVID-19 response efforts to meet the specific needs of Canadians. This study was seen as an element of broader public engagement efforts in response to COVID-19 that intend to proactively and iteratively educate and encourage Canadians to play their part in sustained response efforts.

1.2 Notes on The Interpretation of The Findings

Since a sample drawn from an Internet panel is not probabilistic in nature, the margin of error cannot be calculated for this survey. Respondents for this survey were selected from among those who have volunteered to participate/registered to participate in online surveys. The results of such surveys cannot be described as statistically projectable to the target population. The data have been weighted to reflect the demographic composition of the target population. Because the sample is based on those who initially self-selected for participation, no estimates of sampling error can be calculated.

1.3 Declaration of Political Neutrality and Contact Information

I hereby certify, as chief agent of Leger, that the deliverables are in full compliance with the neutrality requirements of the Policy on Communications and Federal Identity and the Directive on the Management of Communications—Appendix C (Appendix C: Mandatory Procedures for Public Opinion Research).

Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, party positions, or the assessment of the performance of a political party or its leaders.

Signed by:

Christian Bourque

Executive Vice President and Associate

Leger

507 Place d’Armes, Suite 700

Montréal, Quebec

H2Y 2W8

cbourque@leger360.com

1.4 Methodology

This study was conducted through a web-based survey of the Canadian population aged 18 and over who can speak English or French. The respondents were randomly selected through the Leo panel, our panel of Canadian Internet users comprising nearly half a million Internet users.

Eight waves of studies were completed for this research project. A sample of 2,000 respondents was collected each wave. Since this was a longitudinal study, the objective was to re-invite the 2,000 respondents from wave 1 in subsequent survey waves. Our goal was to maximize the number of respondents who respond to the maximum number of survey waves. Respondents who cease to participate in subsequent waves were replaced following the gender, age and regional quotas that was be implemented in the project.

All respondents were contacted via email by Leger. All invitations were bilingual to ensure that no respondent gets a unilingual invitation in the wrong official language. Each respondent was provided with an invitation that includes preapproved information from PCO regarding the nature of the research (i.e. Government of Canada) including the required information for consent and the rights and obligations of respondents.

Fieldwork for this survey took place from April 10, 2020 to September 16, 2020. Details of the eight waves are provided in the following sections.

1.5 Quotas

A series of quotas were implemented for this project. Quotas were cross-referenced by gender and age groups and were also imposed on the region of residence of respondents. The first quota is 50% men and 50% women for the gender sample. These gender quotas were also respected within the following age groups: 18-34, 35-54 and 55 and over. Those gender and age quotas had to be respected at the regional level. The Canadian regions were split as follows:

● Atlantic Canada (Newfoundland, Prince Edward Island, Nova Scotia, New Brunswick);

● Quebec;

● Ontario;

● Manitoba/Saskatchewan/Nunavut;

● Alberta/Northwest Territories;

● British Columbia/Yukon.

The following table details the expected distribution of the sample across the provinces and territories for each wave.

The sample distribution was planned as follows:

Provinces and Territories	NL	NS	PE	NB	QC	ON	MB	SK	AB	BC	NU	NT	YT
# of Respondents	40	80	22	60	400	620	126	106	260	280	2	2	2

As there were no respondents from Nunavut, the two planned respondents were replaced by respondents from Manitoba and Saskatchewan. As for any general population sample derived from a national survey, the final results were weighted by region, age groups, gender, education and the presence of children in the household to make the final samples representative of the actual population of Canada. Details on the weighting factors are presented in a subsequent section of this report.

2. Details for Each Wave

2.1 Pretest

To validate the programming of the questionnaire, a pre-test was conducted in both languages before each wave of the project. The following table shows the details of those pre-tests. A validation of frequencies and databases was done after each pre-test to ensure that the programming was accurate and functional.

Table 1. Pretest details for waves 1 through 8

Wave 1
Date of the pre-test	April 10^th,2020
Number of completed questionnaires	46
Average length during pre-test	17.34 min.
Wave 2
Date of the pre-test	April 21^st,2020
Number of completed questionnaires	31
Average length during pre-test	12.25 min.
Wave 3
Date of the pre-test	May 5^th, 2020
Number of completed questionnaires	24
Average length during pre-test	17.47 min.
Wave 4
Date of the pre-test	May 27^th, 2020
Number of completed questionnaires	34
Average length during pre-test	15.04 min
Wave 5
Date of the pre-test	June 23^rd, 2020
Number of completed questionnaires	30
Average length during pre-test	18.32 min
Wave 6
Date of the pre-test	July 17^th, 2020
Number of completed questionnaires	55
Average length during pre-test	18.02
Wave 7
Date of the pre-test	August 13^th,2020
Number of completed questionnaires	36
Average length during pre-test	16.17 min.
Wave 8
Date of the pre-test	September 10^th,2020
Number of completed questionnaires	53
Average length during pre-test	20.42 min

2.2 Data collection

Data collection for this project began in April 2020 and was carried out over several waves until September 2020. Each wave lasted between four and six days in field. A minimum target of 2,000 respondents for each wave was established. Following the first wave, Léger conducted recontacts to maximize the number of respondents who participated in previous waves and replaced respondents who ceased to participate in subsequent waves following the gender, age and regional quotas. The first days of data collection were aimed at recontacting previous respondents, while the last days of data collection were aimed at replacing the non-returning respondents, due to attrition. The following table details the collection dates and the number of respondents and recontacts for each wave.

Table 2. Data collection details for waves 1 through 8

Wave 1
Start of data collection	April 10^th, 2020
End of data collection	April 14^th, 2020
Invitations sent	19,123
Number of completed interviews	2,023
Survey Length (Average)	26 minutes
Survey Length (Median)	22 minutes
Wave 2
Start of data collection	April 21^st, 2020
End of data collection	April 25^th, 2020
Invitations sent	16,514
Recontact	1,703
New respondents	14,811
Number of completed interviews	2,098
Recontact	1,489
New respondents	609
Survey Length (Average)	23 minutes
Survey Length (Median)	20 minutes
Wave 3
Start of data collection	May 5^th, 2020
End of data collection	May 10^th, 2020
Invitations sent	8,012
Recontact	2,002
New respondents	6,010
Number of completed interviews	2,000
Recontact	1,733
New respondents	267
Survey Length (Average)	23 minutes
Survey Length (Median)	20 minutes
Wave 4
Start of data collection	May 27^th, 2020
End of data collection	June 1^st,2020
Invitations sent	12,380
Recontact	1,987
New respondents	10,393
Number of completed interviews	2,152
Recontact	1,702
New respondents	450
Survey Length (Average)	25 minutes
Survey Length (Median)	21 minutes
Wave 5
Start of data collection	June 23^rd, 2020
End of data collection	June 28^th,2020
Invitations sent	15,880
Recontact	2,128
New respondents	13,752
Number of completed interviews	2,169
Recontact	1,847
New respondents	322
Survey Length (Average)	23 minutes
Survey Length (Median)	20 minutes
Wave 6
Start of data collection	July 17^th, 2020
End of data collection	July 22^nd, 2020
Invitations sent	7,408
Recontact	2,169
New respondents	5,239
Number of completed interviews	2,141
Recontact	1,885
New respondents	256
Survey Length (Average)	23 minutes
Survey Length (Median)	20 minutes
Wave 7
Start of data collection	August 13^th,2020
End of data collection	August 17^th,2020
Invitations sent	9,857
Recontact	2,141
New respondents	7,716
Number of completed interviews	2,129
Recontact	1,776
New respondents	353
Survey Length (Average)	24 minutes
Survey Length (Median)	21 minutes
Wave 8
Start of data collection	September 10^th,2020
End of data collection	September 16^th,2020
Invitations sent	9,442
Recontact	2,129
New respondents	7,313
Number of completed interviews	2,117
Recontact	1,822
New respondents	295
Survey Length (Average)	29 minutes
Survey Length (Median)	24 minutes

2.3. Participation rate

Below is the calculation of the participation rate to the web survey for all eight waves. The participation rate is calculated using the following formula: Participation rate / response rate = R ÷ (U + IS + R). The table below provides details of the calculation. For all waves, the participation rate ranges between 11,4% and 30%.

Table 3. Participation rate for wave 1

Base Sample	19,123
Invalid cases	0
Invitations mistakenly sent to people who did not qualify for the study	0
Incomplete or missing email addresses	0
Unresolved (U)	16,255
Email invitations bounce back	0
Email invitations unanswered	16,255
EFFECTIVE SAMPLE*	2,319
In-scope non-responding units (IS)	211
Non-response from eligible respondents	0
Respondent refusals	0
Language problem	0
Selected respondent not available (illness; leave of absence; vacation; other)	0
Early break-offs	221
Responding units (R)	2,108
Completed surveys disqualified – quota filled	0
Completed surveys disqualified for other reasons	85
COMPLETED INTERVIEWS	2,023
Participation rate	11.4%

Table 4. Participation rate for wave 2

Base Sample	16,514
Invalid cases	0
Invitations mistakenly sent to people who did not qualify for the study	0
Incomplete or missing email addresses	0
Unresolved (U)	13,907
Email invitations bounce back	0
Email invitations unanswered	13,907
EFFECTIVE SAMPLE*	2,221
In-scope non-responding units (IS)	78
Non-response from eligible respondents	0
Respondent refusals	0
Language problem	0
Selected respondent not available (illness; leave of absence; vacation; other)	0
Early break-offs	78
Responding units (R)	2,143
Completed surveys disqualified – quota filled	0
Completed surveys disqualified for other reasons	45
COMPLETED INTERVIEWS	2,098
Participation rate	13.3%

Table 5. Participation rate for wave 3

Base Sample	8,012
Invalid cases	0
Invitations mistakenly sent to people who did not qualify for the study	0
Incomplete or missing email addresses	0
Unresolved (U)	5,551
Email invitations bounce back	0
Email invitations unanswered	5,551
EFFECTIVE SAMPLE*	2,087
In-scope non-responding units (IS)	57
Non-response from eligible respondents	0
Respondent refusals	0
Language problem	0
Selected respondent not available (illness; leave of absence; vacation; other)	0
Early break-offs	57
Responding units (R)	2,030
Completed surveys disqualified – quota filled	0
Completed surveys disqualified for other reasons	30
COMPLETED INTERVIEWS	2,000
Participation rate	21%

Table 6. Participation rate for wave 4

Base Sample	12,380
Invalid cases	0
Invitations mistakenly sent to people who did not qualify for the study	0
Incomplete or missing email addresses	0
Unresolved (U)	9,580
Email invitations bounce back	0
Email invitations unanswered	9,580
EFFECTIVE SAMPLE*	2,317
In-scope non-responding units (IS)	127
Non-response from eligible respondents	0
Respondent refusals	0
Language problem	0
Selected respondent not available (illness; leave of absence; vacation; other)	0
Early break-offs	127
Responding units (R)	2,190
Completed surveys disqualified – quota filled	0
Completed surveys disqualified for other reasons	38
COMPLETED INTERVIEWS	2,152
Participation rate	18.4%

Table 7. Participation rate for wave 5

Base Sample	15,880
Invalid cases	0
Invitations mistakenly sent to people who did not qualify for the study	0
Incomplete or missing email addresses	0
Unresolved (U)	12,409
Email invitations bounce back	0
Email invitations unanswered	12,409
EFFECTIVE SAMPLE*	2,324
In-scope non-responding units (IS)	130
Non-response from eligible respondents	0
Respondent refusals	0
Language problem	0
Selected respondent not available (illness; leave of absence; vacation; other)	0
Early break-offs	130
Responding units (R)	2,194
Completed surveys disqualified – quota filled	0
Completed surveys disqualified for other reasons	25
COMPLETED INTERVIEWS	2,169
Participation rate	15%

Table 8. Participation rate for wave 6

Base Sample	7,408
Invalid cases	0
Invitations mistakenly sent to people who did not qualify for the study	0
Incomplete or missing email addresses	0
Unresolved (U)	4,983
Email invitations bounce back	0
Email invitations unanswered	4,983
EFFECTIVE SAMPLE*	1,314
In-scope non-responding units (IS)	113
Non-response from eligible respondents	0
Respondent refusals	0
Language problem	0
Selected respondent not available (illness; leave of absence; vacation; other)	0
Early break-offs	113
Responding units (R)	2,201
Completed surveys disqualified – quota filled	0
Completed surveys disqualified for other reasons	60
COMPLETED INTERVIEWS	2,141
Participation rate	30%

Table 9. Participation rate for wave 7

Base Sample	9,857
Invalid cases	0
Invitations mistakenly sent to people who did not qualify for the study	0
Incomplete or missing email addresses	0
Unresolved (U)	7,309
Email invitations bounce back	0
Email invitations unanswered	7,309
EFFECTIVE SAMPLE*	2,328
In-scope non-responding units (IS)	165
Non-response from eligible respondents	0
Respondent refusals	0
Language problem	0
Selected respondent not available (illness; leave of absence; vacation; other)	0
Early break-offs	165
Responding units (R)	2,163
Completed surveys disqualified – quota filled	0
Completed surveys disqualified for other reasons	34
COMPLETED INTERVIEWS	2,129
Participation rate	22%

Table 10. Participation rate for wave 8

Base Sample	9,442
Invalid cases	0
Invitations mistakenly sent to people who did not qualify for the study	0
Incomplete or missing email addresses	0
Unresolved (U)	7,007
Email invitations bounce back	0
Email invitations unanswered	7,007
EFFECTIVE SAMPLE*	2,365
In-scope non-responding units (IS)	2,252
Non-response from eligible respondents	0
Respondent refusals	0
Language problem	0
Selected respondent not available (illness; leave of absence; vacation; other)	0
Early break-offs	113
Responding units (R)	2,139
Completed surveys disqualified – quota filled	0
Completed surveys disqualified for other reasons	22
COMPLETED INTERVIEWS	2,117
Participation rate	19%