2019-20 Annual Tracking Study

Final report

Prepared for: Immigration, Refugees and Citizenship Canada

Supplier Name: Ipsos
Contract Number: B8815-20-0083/001/CY
Contract Value: $166,839.12 (including HST)
Award Date: 2019-12-11
Delivery Date: 2020-03-31

Registration Number: POR 051-19

For more information on this report, please contact IRCC at:
IRCC.COMMPOR-ROPCOMM.IRCC@cic.gc.ca

Ce rapport est aussi disponible en français.

Copyright

This public opinion research report presents the results of a survey conducted by Ipsos on behalf of Immigration, Refugees and Citizenship Canada. The research was conducted with 2,000 Canadians by telephone, 2,361 Canadians online, and 1,320 surveys in 14 specific municipalities by telephone, between January 30^th, 2020 and March 4^th, 2020.

Cette publication est aussi disponible en français sous le titre : Étude de suivi annuelle de 2019-20.

This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Immigration, Refugees and Citizenship Canada. For more information on this report, please contact Immigration, Refugees and Citizenship Canada at IRCC.COMMPOR-ROPCOMM.IRCC@cic.gc.ca or at:

Communications Branch
Immigration, Refugees and Citizenship Canada
Jean Edmonds Tower South
365 Laurier Ave W
Ottawa ON K1A 1L1

Catalogue Number: Ci4-183/1-2020E-PDF
International Standard Book Number (ISBN): 978-0-660-35523-8

Related publications (registration number: POR 051-19):
Catalogue Number: Ci4-183/1-2020F-PDF (Final Report, French)
ISBN: 978-0-660-35524-5

© Her Majesty the Queen in Right of Canada, as represented by the Minister of Immigration, Refugees and Citizenship Canada, 2020

Political neutrality statement

I hereby certify as Senior Officer of Ipsos that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leaders.

Mike Colledge
President
Ipsos Public Affairs

Executive summary

Ipsos Public Affairs is pleased to present this report to Immigration, Refugees and Citizenship Canada.

Background

Since 1994, when it was established as a new department bringing together immigration services and citizenship registration, Immigration, Refugee and Citizenship Canada (IRCC) has played several key roles: admitting immigrants, foreign students, visitors and temporary workers; resettling refugees; helping immigrants adapt to Canadian society and become Canadian citizens; and managing access to Canada.

IRCC conducts an ongoing research program to help the Department develop a better understanding of Canadian attitudes toward the issues surrounding citizenship and immigration. By gauging and analyzing the opinions of newcomers and immigrants, the Department gains insights into important policy areas related to the mandate of the department and related services.

IRCC identified a need to conduct quantitative research (telephone and online surveys) among the general public, newcomers and settled immigrants.

Research objectives

The qualitative and quantitative research was intended to explore the views of members of the Canadian general population and attitudes of newcomers and settled immigrants on issues such as immigration, integration, settlement, multiculturalism and citizenship as well as IRCC services that are of key importance to IRCC’s policies and programs. Research was intended to support IRCC in ensuring high quality policy options, program design and advice to ministers; encouraging and effectively managing citizen-focused services; managing organizational and strategic risks proactively; and gathering and using relevant information on program results. The value of this contract, including HST, is $166,839.12.

Methodology

To meet the research objectives, Ipsos conducted a national telephone survey and a national online survey. The 15-minute national telephone survey was conducted among a nationwide sample of n=2,000 Canadian adults between January 30^th and February 23^rd, 2020. The telephone survey sample was a probability sample generated through random digit dialing obtaining an overall margin of error of +/-2.1 percentage points (calculated at a 95% confidence interval). The 14-minute online survey was conducted among 2,361 respondents between February 11^th and 25^th, 2020, drawn entirely from Ipsos’ proprietary panel, iSay. As the online survey used non-probability sampling, a margin of error cannot be calculated. Respondents were offered the survey in the official language of their choice.

Ipsos also conducted an 11-minute telephone survey with a random sample of 1,320 individuals from 14 specific municipalities between February 10th and March 4th, 2020. The sample of members of the general population aged 18+ was a probability sample generated through random digit dialing obtaining an overall margin of error of +/-2.9 percentage points (calculated at a 95% confidence interval).

A full quantitative methodology report, including all information about the execution of the fieldwork that is needed to replicate the research initiative, may be found in Appendix 1. The quantitative survey research instruments in English and French may be found in Appendix 2. A set of tabulated results from the quantitative surveys are provided under a separate cover.

Should you have any questions or comments, please contact:

Jennifer Macey
Vice President
Ipsos Public Affairs
Telephone: 416-324-2900
Email: Jennifer.macey@ipsos.com

Appendix 1 – Quantitative methodology

Telephone survey (National)

Ipsos conducted a 15-minute telephone survey among a nationwide sample of n=2,000 Canadian adults between January 30^th and February 23^rd, 2020. The sample is a probability sample generated through random digit dialing. For respondents contacted on a land line, respondents within households were selected at random, by using the “birthday method” of identifying and interviewing the member of the household (aged 18+) who had their birthday last.

Respondents contacted on a cellular phone were also random digit dialed, and needed to be 18+ to participate. Wireless samples were selected on a provincial level (as it is not practical to accurately select by market given the mobile nature of the technology) from a database containing all possible numbers in 1000-blocks of area codes and exchanges dedicated to wireless numbers.

Within the total sample of 2,000 Canadians for this survey, 700 respondents were contacted on their landlines, while the other 1,300 respondents were contacted on their cellphones. The margin of error for a telephone survey of 2,000 respondents is ±2.1%, using a confidence interval of 95% (19 times out of 20). The final questionnaire used was provided by IRCC to ensure adequate tracking of previous research results conducted by the department.

Telephone sample weighting

The tables below indicate the unweighted and weighted distributions of the telephone sample. The sample was stratified by region, with soft quotas also set for gender and age to ensure appropriate representation across categories. Weighting was applied to the sample to ensure that the final data reflects the adult population of Canada by region, age and gender according to the 2016 Census.

Please note, the totals below may not add up to 2,000 due to some respondents’ refusal to provide socioeconomic information.

Statistics presented in the table above show minimal differences between the final unweighted and weighted samples. However, the youngest age group (18 to 24 years old) is underrepresented, resulting in a higher unweighted to weighted ratio.

Call dispositions

The following table provides the call dispositions and response rate calculation, as per the former MRIA’s empirical method of calculating response rates for telephone surveys.

The response rate, calculated as the number of in-scope – responding units divided by the sum of unresolved units, in-scope – non-responding units, and in-scope – responding units, was 7.46% for landline numbers, 8.39% for cellphone numbers, and 8.07% for all telephone numbers. The total response rate of 8.07% for a telephone survey of the Canadian general population with up to 8 call-backs per household is typical.

Non-response analysis

As with any probability sample, there exists within the current sample the possibility of non-response bias. In particular, this survey would not include members of the population who do not have access to a telephone (either landline or cell phone) or who are not capable of responding to a survey in either English or French. In addition, some groups within the population are systemically less likely to answer surveys.

The table below compares the unweighted sample to the 2016 Census results by region, age, gender, education, income and country of birth. The comparison between the two samples for the three variables used in the weighting scheme (using interlocking weights for region with age and region with gender) shows a slight underrepresentation of younger Canadians (18 to 24 years of age). However, this discrepancy is small enough that it can be corrected through weighting without affecting the quality of the final results. As the regional distribution was set through hard quotas, the weighting had virtually no impact on final numbers. Males were slightly overrepresented in the unweighted sample, at 52% compared to 49% in the actual population.

This comparison between the unweighted sample distribution and the actual population figures for variables not included in the weighting scheme shows that the final sample obtained was mostly representative of the general population for this survey. However, there are noticeable differences in education levels between the sample and the Canadian adult population, with the telephone sample being more educated than Census figures show. The largest gap was for the university educated stratum, with 47% of the sample having obtained a university degree, compared to 22% among Canadian adults. Education is a variable that could be considered in future weighting schemes for national surveys to correct for this imbalance. Income distributions for the sample are very close to those measured in the 2016 Census. The country of birth distribution matches the 2016 Census.

Online sample (National)

The online sample of 2,361 respondents was drawn entirely from Ipsos’ proprietary panel, iSay, consisting of approximately 185,000 Canadians recruited in all provinces and territories. As this is a non-probability sample, a margin of error cannot be calculated. Respondents to the online survey were invited to participate via email, with a unique URL link to the survey provided to them. This link could only be used once, with respondents being allowed to take pause during completion and return to complete it at a later time. Survey questionnaires took 14 minutes to complete on average. All surveys were completed between February 11^th and 25^th, 2020.

Incentives and quality control measures

Respondents to Ipsos’ online surveys are offered a number of innovative incentive programs in the forms of a point-based system where participants can redeem points for various items. We do not reward our panelists using cash payments.

Extensive quality-control procedures are in place within IIS (Ipsos Interactive Services, who manage our panel) to ensure that the survey inputs (sample and questionnaire design) allow for high-quality survey outputs (survey data). These processes span the life cycle of a panelist and are in place for all Ipsos online surveys. IIS experts are constantly monitoring and reviewing the performance of our quality measures and updating and integrating new ones as respondents’ behaviors and the online landscape evolve.

Panelists are who they say they are

• Double Opt-In approach to confirm identity
• Country validation via Geo-IP
• Mismatch between device settings and geolocation
• Anonymous proxy detection
• Detection of robots via Captcha code
• Detection of “5 minutes” emails (temporary email addresses)
• Detection of data anomalies and patterns
• Maintenance of Ipsos blacklist
• RealAnswer™- detection of pasted and robot answers

They have not participated recently in similar surveys

• Strict panel usage rules to avoid interviewing the same people too often and prevent them from becoming too used to a type of survey or product category
• Duplicate devices identification through digital Fingerprinting (RelevantID^®) and web/flashcookie

They complete surveys seriously

• Survey taking behavior: speeding, straight lining, open-ends quality evaluation
• Panelists’ history monitored across surveys and used for panel purge removing “bad” or inactive respondents

They can only take the survey once

• Duplicate emails identification
• Duplicate devices identification through digital fingerprinting (RelevantID^®) and web/flashcookie
• Duplicate contact details identification

Online sample weighting

The tables below indicate the unweighted and weighted distributions of the online sample. The sample was stratified by region, with soft quotas also set for gender and age to ensure appropriate representation across categories. Weighting was applied to the sample to ensure that the final data reflects the adult population of Canada by region, age and gender according to the 2016 Census.

Please note, the totals below may not add up to 2,361 due to some respondents’ refusal to provide socioeconomic information.

Statistics presented in the tables above show minimal differences between the final unweighted and weighted samples. However, the youngest age group (18 to 24 years old) is underrepresented, resulting in a higher weight ratio of 1.22:1, which remains well within acceptable ranges for a survey of the general population and fares favourably compared to the gap observed in the telephone sample.

Email statistics

The table below presents general statistics regarding the response rate for the email phase of research.

The response rate, calculated as the number of in-scope – responding units divided by the sum of unresolved units, in-scope – non-responding units, and in-scope – responding units, was 10.66%. This response rate is within normal ranges for a survey of the Canadian adult population.

Non-response analysis

As with any non-probability sample there exists within the current sample the possibility of non-response bias. In particular, this survey would not include members of the population who do not have access to a computer with an Internet connection (either at home or at work) or who are not capable of responding to a survey in either English or French. In addition, some groups within the population are systemically less likely to answer surveys.

The tables below compare the unweighted sample to the 2016 Census results by region, age, gender, education, income and country of birth. Overall, the sample is highly representative of the national adult population, except for a few gaps which are described below.

The comparison for the variables used in the weighting scheme are minimal, except for a small gap for the youngest age group, which is slightly underrepresented in the unweighted sample. However, as discussed above, the age distribution in the online sample remains solid. The largest observable gaps between the unweighted sample and Census 2016 data relates to education levels, as observed with the phone sample as well. We find that the unweighted sample tends to overrepresented more educated Canadians, although the gaps are not as important as for the telephone sample. The online sample also contains a smaller proportion of Canadians who have a household income of $100,000 and above, as well as a slightly smaller proportion of immigrants.

Comparison of phone and online samples (National)

The tables below present a comparison of the telephone and online samples across the variables used in the weighting scheme for samples. Totals for each variable may not add up to total sample size due to some respondents’ refusal to provide socioeconomic information.

Comparison by socioeconomic information not used in weighting

The next tables show the comparison between both samples for education and income levels, as well as country of birth, which were not included in the weighting scheme. Considering that no quotas were set for any of these three variables during fieldwork, the similarities between the two samples are remarkable, though there is a 9-point gap in those who have obtained a university degree. As discussed above, both samples over represent university-educated Canadians and underrepresent those with a completed high school degree or less, with the online sample being slightly closer to Census 2016 figures. Some differences can also be seen with regards to income levels when comparing the online and phone respondents. More specifically, these gaps are visible in the under $40,000 bracket (8-point gap) and the $100,000 or more bracket (15-point gap).

Finally, both samples show a difference in the proportion of respondents born in Canada. While 24% of phone respondents report being born outside of the country, only 17% of online respondents report the same. According to Census 2016 figures, the actual proportion in the Canadian population is 22%. Therefore, the telephone sample very closely represents the immigrant population, while the online sample slightly underrepresents it.

The differences noted above in the proportion of immigrants sampled via each mode of interviewing do not explain some of the statistical differences noted in survey responses from one data collection mode to the other. Respondents who are not born in Canada are more positive than those born in the country on most questions included in this study, but the size of those gaps in attitudes are too small to make the kind of difference that would be needed to explain the differences in survey responses between online and phone respondents. At most, this smaller proportion of immigrants in the online sample could explain a difference of 1 percentage point on some questions.

Municipal telephone survey

Ipsos conducted an 11-minute telephone survey targeting 14 specific municipalities, with a random sample of 1,320 individuals selected from the Canadian general population (aged 18 or older) between February 10^th and March 4^th, 2020. The sample is a probability sample generated through random digit dialing.

For respondents contacted on a land line, respondents within households were selected at random, by using the “birthday method” of identifying and interviewing the member of the household (aged 18+) who had their birthday last.

Respondents contacted on a cellular phone were also random digit dialed and needed to be 18+ to participate. Wireless samples were selected on a municipality level from a database containing all possible numbers in 1000-blocks of area codes and exchanges dedicated to wireless numbers. Out of 14 municipalities, three municipalities (Altona/Rhineland, Claresholm and West Kootenay) were contacted via land line only, as these are small areas without unique dedicated wireless exchanges, meaning it is not possible to accurately select wireless phone numbers for these municipalities given the mobile nature of the technology.

Within the total sample of 1,320 Canadians for this survey, 635 respondents were contacted on their landlines, while the other 685 respondents were contacted on their cellphones. The margin of error for a telephone survey of 1,320 respondents is ±2.9%, using a confidence interval of 95% (19 times out of 20). The final questionnaire used was provided by IRCC to ensure adequate tracking of previous research results conducted by the department.

Call dispositions

The following table provides the call dispositions and response rate calculation, as per the former MRIA’s empirical method of calculating response rates for telephone surveys.

The response rate, calculated as the number of in-scope – responding units divided by the sum of unresolved units, in-scope – non-responding units, and in-scope – responding units, was 9.76% for landline numbers, 7.08% for cellphone numbers, and 7.92% for all telephone numbers. The total response rate of 7.92% for a telephone survey of the Canadian general population with up to 8 call-backs per household is typical.

Municipal sample weighting

The tables below indicate the unweighted and weighted distributions of the municipal sample. The sample was stratified by municipality. Weighting was applied to the samples from 11 of the 14 municipalities to ensure that the final data reflects their adult population by age according to 2016 Census profiles. Weighting was not applied to the samples from the three municipalities that were contacted via land line only (Altona/Rhineland, Claresholm and West Kootenay).

Appendix 2 – Quantitative instruments

English and French quantitative instruments are provided under separate cover.