Trust, Information, and Digital Ecosystems Study (TIDES) - Methodological report

Prepared for the Privy Council Office (PCO)

Supplier:

Ekos Research Associates Inc.

Contract Number:

35035-221248

Contract Value:

$299,760.75 (tax included)

Award Date:

October 21, 2022

Delivery Date:

March 18, 2024

Registration Number:

POR 069-22

For more information on this report, please contact POR-ROP@pco-bcp.gc.ca

This public opinion research report presents the methodology for the conduct of four online surveys conducted by Ekos Research Associates Inc. on behalf of the Privy Council Office (PCO).

Cette publication est aussi disponible en français sous le titre:L'étude sur la confiance, l'information et l'écosystème numérique (ECIEN)

This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from PCO.

Catalogue Number:

CP22-214/2024E-PDF

International Standard Book Number (ISBN):

978-0-660-70892-8

Catalogue Number (French Report):

CP22-214/2024F-PDF

ISBN:

978-0-660-70893-5

© His Majesty the King in Right of Canada, 2024

Table of Contents

• Summary
  1. Background and Methodology
  2. Expenditure
  3. Political Neutrality Certification
• Appendices
  1. Detailed Methodology
  2. Survey Programming and Pre-Test
  3. Survey Administration/Quality Control
  4. Sample Source/Data Analysis
  5. Overview of Survey Waves
• Under Separate Cover: Questionnaires
• Under Separate Cover: Data Tables

Summary

A. Background and Methodology

The Privy Council Office (PCO) is the hub of non-partisan, public service support to the Prime Minister and Cabinet and its decision-making structures. Led by the Clerk of the Privy Council, PCO helps the Government implement its vision and respond effectively and quickly to issues facing the government and the country.

The proliferation of mis/disinformation is a growing challenge for Canadian society. Previous research has found that exposure to, and belief in, mis/disinformation can erode public trust in institutions and impact the Government’s ability to protect public health and deliver on broader national interests (e.g., responses to COVID-19, and climate adaptation/mitigation policy support). The federal government has prioritized combating mis/disinformation with an emphasis on developing and implementing government-wide counter-measures to protect our democracy.

To support the government’s agenda on these issues, the IIU - working with partners within PCO and at PHAC - is carrying out a series of research initiatives, expanding upon two multi-year programs of research related to COVID-19 response efforts and climate action already underway. Specifically, these series of initiatives aim to:

• Build a greater understanding of the factors that shape individuals’ relationship with information and institutions in the digital age, and test interventions that reduce the greatest harms of mis/disinformation – such as threats to public health, climate action, democratic and shared values, and diversity and inclusion.
• Deepen and broaden the Government’s understanding of challenges in maintaining and enhancing trust in the institutions people count on to support informed public discourse and debate, as well as the role that access to reliable information plays in those conversations.
• Develop a more robust understanding of the human factors that are associated with greater resilience to mis/disinformation for individuals, and the contextual and systemic factors that affect its spread.

In alignment with these broader objectives, Ekos conducted a multi-wave, cross-sectional study exploring the impact of public trust (in government, in institutions, and in each other) and susceptibility to mis/disinformation on outcomes of interest for the Government of Canada, assessing convergence and divergence within and across priority policy areas and aspects of our democratic society.

The goal of the study was to provide PCO and PHAC with research-based information on Canadians’ beliefs, attitudes, and behaviours relating to a variety of topics under the broader umbrellas of mis/disinformation, public trust, and democratic resilience. This report outlines the methodology used in undertaking this study.

The target audience for the study was adults living in Canada 18 years of age and older, aiming for appropriate representation of gender, age, education, and regional split (Canadian citizens as well as permanent residents).

The study consisted of four survey waves. A total of 3,375 respondents were surveyed in each wave. The sample included general population respondents in all survey waves, as well as an oversample of selected underrepresented populations in all waves.

B. Expenditure

The contract amount for this project was $299,760.75 (including HST).

C. Political Neutrality Certification

I hereby certify as Senior Officer of Ekos Research Associates Inc. that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research.

Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.

Signed by Derek Jansen (Vice President)

Appendices

A. Detailed Methodology

Data was collected through a series of four online survey waves with Canadians 18 years of age and older, randomly recruited from Ekos’ probability-based online panel. In each wave Ekos conducted an approximately 25-minute online survey with a sample of 3,375 respondents. The first wave consisted of approximately 2,600 general population respondents, and roughly 800 respondents from an oversample of underrepresented populations identified by the client. Each subsequent wave consisted of approximately 2,900 general public respondents, and roughly 450 from an oversample of underrepresented populations identified by the client. Section A4 provides more details about the sample sizes and sample composition in each of the four survey waves.

In each wave, efforts were made to include only panelists who did not participate in a previous wave, however some panelists in hard to reach groups (such as those under 35 years of age, those with high school education, etc) were surveyed more than once over the course of the study.

The general population sample frame for each survey wave consisted of roughly 50% women and 50% men for each age group (18 to 34; 35 to 54; 55+), and for each region:

• Atlantic Canada (Newfoundland, Prince Edward Island, Nova Scotia, New Brunswick);
• Quebec;
• Ontario;
• Manitoba/Saskatchewan/Nunavut;
• Alberta/Northwest Territories;
• British Columbia/Yukon.

The survey results for each wave (n=3,375) are valid within a margin of error of ± 1.7 percentage points, 19 times out of 20.

B. Survey Programming and Pre-Test

PCO provided Ekos with the final English questionnaire for each survey wave. Ekos translated, programmed and pre-tested the questionnaire for each survey wave with at least 20 respondents (a minimum of 10 English and 10 French pre-test completions each wave). Appended to the pre-test questionnaire was a diagnostic battery of questions which allowed respondents to provide an assessment of the clarity and flow of the questionnaire, and provide any final comments about the survey. The questionnaire for each survey wave was finalized in consultation with PCO.

C. Survey Administration/Quality Control

The final questionnaire in each survey wave was administered to the full sample using a bilingual e-mail invitation and a bilingual questionnaire, installed on a web-server controlled by Ekos. The e-mail invitation included a description of the survey (in both languages) along with a link to the survey website. When respondents clicked on the survey link, they were taken to a website containing the survey instrument. Once inside the survey, the respondent had the choice of completing the questionnaire in French or English (and they could change the survey language between French and English at any time). The survey database was mounted using a Personalized Identification Number (PIN), so only individuals with a PIN were allowed access to the survey (the PIN was included in the e-mail invitation). The PIN also allowed respondents to exit and re-enter the survey at any time to complete or change information before the questionnaire was completed/submitted.

Survey data collection adhered to the Standards for the Conduct of Government of Canada Public Opinion Research—Online Surveys, as well as all applicable industry standards. Ekos informed respondents of their rights under the Privacy Act and the Access to Information Act and ensured that those rights were protected throughout the research process. This included: informing respondents of the purpose of the research; identifying both the sponsoring department and the research supplier; informing respondents that their participation in the study is voluntary, and that the information provided would be administered according to the requirements of the Privacy Act.

Both the e-mail invitation and the survey instructions included an e-mail address and a 1-800 number that respondents could use in the event that there were questions about the study or completing the questionnaire.

In order to increase the response rate to the surveys, all non-respondents were sent up to five bilingual reminder e-mails, emphasizing the relevance and importance of the survey, and asking them to take the time needed to participate. Additionally, incentives of $10 or $15 (chosen randomly) were offered to everyone aged 18-24, as well as certain oversample groups, in an effort to increase response from these hard to reach groups. Some of our non-responding panelists (those aged 18-34, certain oversample groups, and other target groups who did not respond after the reminder emails) were also called by an interviewer from our call centre to remind them about the survey, and re-send the survey invitation if needed.

D. Sample Source/Data Analysis

The sample for the survey was drawn from Ekos’ in-house probability-based panel, Probit. Probit is a research panel that has been designed by Ekos to provide statistically representative data. Our online panel offers complete coverage of the Canadian population (i.e., Internet, phone, cell phone), random recruitment (in other words, participants are recruited randomly, they do not opt themselves into our panel), and equal probability sampling. All respondents to our panel are recruited by telephone using random digit dialling and are confirmed by live interviewers.

Our panel is considered to be probability based as it is recruited from random digit dial (RDD) sample, both landline and cell phone. The overall panel size is roughly 100,000 Canadian households. Random stratified samples are drawn from the panel database for individual research assignments.

Once the data for each survey wave was collected, the database was reviewed for data quality. Coding of open-ended questions was also completed. Survey results were weighted based on Statistics Canada data according to age, gender, region and education to ensure the sample was representative of the general public aged 18 years and older. Based on an analysis plan provided by PCO, Ekos provided results for each survey wave in the form of Excel data tables and SPSS files. A field summary was also provided at the end of each wave.

E. Overview of Survey Waves

Below we outline the field dates, sample sizes, sample composition (general public and underrepresented populations), and response rates^[1] for each of the four survey waves conducted as part of this study.