Executive Summary
Prepared for the Privy Council Office of Canada
Supplier name: Advanis Inc.
Contract Number: 35035-200902/001/CY
Contract Value: $129,853.54 (including HST)
Award Date: 2020-11-30
Delivery Date: 2021-02-12
Registration Number: POR 071-20
For more information on this report, please contact the Privy Council Office at:por-rop@pco-bcp.gc.ca
Ce rapport est aussi disponible en français
Canada-US Relations Survey
Executive Summary
Prepared for the Privy Council Office of Canada
Supplier name: Advanis Inc.
February 2021
This public opinion research report presents the methodology of a telephone survey conducted by Advanis Inc. on behalf of the Privy Council Office. The research was conducted with 2,000 adult Canadians, and 1,107 adult Americans between December 8th, 2020 and February 3rd, 2021.
Cette publication est aussi disponible en français sous le titre : Les relations Canado-Américaines – rapport méthodologique.
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from the Privy Council Office. For more information on this report, please contact the Privy Council Office at:
por-rop@pco-bcp.gc.ca
Privy Council Office
85 Sparks Street
Room 228
Ottawa, Ontario K1A 0A3
Catalogue number:
CP22-194/2021E-PDF
International Standard Book Number (ISBN):
978-0-660-37513-7
Related publications (registration number: POR 071-20):
Catalogue number CP22-194/2021F-PDF (Executive Summary, French)
International Standard Book Number (ISBN): 978-0-660-37514-4
© Her Majesty the Queen in Right of Canada, as represented by the Privy Council Office of Canada, 2021.
The Privy Council Office (PCO) is the hub of non-partisan, public service support to the Prime Minister and Cabinet and its decision-making structures. Led by the Clerk of the Privy Council, PCO helps the Government implement its vision and respond effectively and quickly to issues facing the government and the country.
As an advisor to the Prime Minister, PCO brings together quality, objective policy advice and information to support the Prime Minister and Cabinet, including (1) information on the priorities of Canadians and (2) expertise on outcomes-based policy and program approaches.
The Communications and Consultation Secretariat within PCO supports the Prime Minister’s Office in coordinating government communications and setting broad government communications themes and messages, in accordance with government priorities, as determined by the Prime Minister, Cabinet, Cabinet committees and the Clerk of the Privy Council. The Secretariat also works with PCO policy secretariats to advise and support Cabinet and its committees.
Through the use of the Canada-US Relations Survey, PCO aimed to explore the perceptions of Canadians and Americans on various bilateral issues, such as trade, security and the environment. By carrying out this research, PCO will be able to ensure a better understanding of public perceptions and develop communications strategies and products that are effective.
The study was conducted, in both English and French, using a 12-minute computer-assisted telephone interviewing (CATI) survey. Numbers were generated using RDD (random digit dialing) systems using both landline and wireless phone numbers across Canada and the United States. Overall, 2,000 Canadians completed the survey between December 8th, 2020 and January 16th, 2021 with a response rate of 11.9%, and a margin of error of 2.2% (19 times out of 20 at a 95% confidence interval). 1107 Americans completed the survey between December 8th, 2020 and February 3, 2021 with a response rate of 2.7%, and a margin of error of 2.95% (19 times out of 20 at a 95% confidence interval). Given the timing of the research calling was suspended for the last 2 weeks of December. In the United States, calling was also paused after the January 6th, 2021 attack on the US Capitol building, and did not resume until after the January 20th inauguration. The data was weighted based on gender, age, region and country.
The survey was conducted via telephone, using computer-assisted telephone interviewing (CATI) technology, with a representative random sample of 2,000 Canadians and 1,107 Americans. The average survey length was estimated to be 12 minutes, with the average for Canada ended up at just over 11.5 minutes, and the United States at just over 13 minutes. Advanis’ approach to CATI is to apply the best technology, interviewers, and quality assurance to each project to achieve the highest quality data in the most efficient manner that meets the project and client needs.
An overlapping dual-frame sample was proposed in order to reach those in both the younger age categories, and those considered to be “cord-cutters”. “Cord-cutting”, or using only a wireless phone, is becoming more prevalent, particularly in the United States, so increasing the proportion of wireless sample was required during field.
Data collection in Canada versus the United States was very different in terms of participation and challenges. In Canada, a total of 29,025 phone numbers were dialed, and 2,000 people completed in the survey. Within Canada, in order to better represent the population age and gender distributions, an additional 758 people qualified for the survey but were unable to participate because enough data with their demographic profile had already been collected. Of the calls conducted in Canada, 16 percent were conducted in French and 84 percent were conducted in English. This resulted in 372 (19%) of the 2000 completes being French-speaking interviews. Canada achieved a response rate of 11.9%, and a completion rate of 7.9%. The response rate calculation is presented in Appendix A.
In the United States, a total of 52,962 phone numbers were dialed, and 1,107 people completed the survey. The response rate in the United States was only 2.4% (calculation is presented in Appendix A). Calling in the United States was difficult for a number of reasons, in particular, getting respondents to answer the phone was quite difficult. The amount of no answer/answering machine type calls was 20% higher in the US than in Canada. Among US numbers dialed, 30,500 were wireless numbers. When comparing landline to wireless although the refusal rate was higher on wireless (30% wireless vs. 21% landline), the proportion of those willing to answer the phone was also higher, resulting in a 3.2% response rate among wireless numbers, compared to 1.9% on landline.
In addition to lower response rates, data collection in the United States also encountered cases of individuals making unsolicited comments to interviewers during calls.
It should be noted that the timing of data collection for the study may not have been ideal. Response rates prior to the holiday season in December are typically lower in general. US calling was also suspended after the attack on the US capital on January 6th, 2021, and paused until after the inauguration on January 20th, 2021. The fact that data collection was done at a moment of high political turmoil in the US probably impacted the participation rate of certain segments of the population this could have introduced some bias.
A pre-test was conducted for the study on Tuesday December 8th, 2020. A total of 33 completes was collected, including 10 English and 10 French in Canada, and 13 English completes from the United States. During the pre-test the average survey length was 18 minutes for the United States and 16 minutes for Canada, so it was recommended that some questions be removed to decrease the survey length.
The total cost of this research was $129,853.54 (including HST).
Political neutrality certification
I hereby certify as Senior Officer of Advanis that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Policy on Communications and Federal Identity and the Directive on the Management of Communications.
Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leaders.
Signed: Nicolas Toutant, Vice-President, Research and Evaluation