National Justice Survey 2023

Methodological Report

Prepared For:
Department of Justice Canada (JUS)

Contract Information

  • Supplier Name: Advanis Inc.
  • Contract number: 19040-230021
  • Contract Value: $102,183.23 (including HST)
  • Award date: February 16, 2023
  • Delivery date: May 31, 2023

For more information on this report, please contact Department of Justice Canada at: rsd.drs@justice.gc.ca

Ce rapport est aussi disponible en français.

Canada Logo

This public opinion research report presents the methodological details for the 2023 National Justice Survey conducted by Advanis Inc. on behalf of Department of Justice Canada (“Justice Canada”). The survey was administered with 4,487 Canadians between March 9th and May 18th, 2023.

Cette publication est aussi disponible en français sous le titre : Sondage national sur la justice 2023

This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from the Department of Justice Canada. For more information on this report, please contact the Department of Justice Canada at rsd.drs@justice.gc.ca or at:

Department of Justice Canada
284 Wellington Street
Ottawa ON K1A 0H8

1. Executive Summary

1.1 Background and Objectives

The National Justice Survey (NJS) is an annual national public opinion research study conducted by the Department of Justice Canada ("Justice Canada"). The survey collects information on the views, knowledge, concerns, and priorities of Canadians on important justice-related issues. Results of the study are used to inform ongoing and future policy development, departmental reporting, communications, and public engagement on important justice-related issues.

The objective of the 2023 NJS was to collect information on the following:

  1. public awareness of restorative justice (RJ) processes, as well as opinions regarding its use;
  2. public awareness of and views on the Youth Criminal Justice Act (YCJA);
  3. public perceptions about Indigenous justice systems and processes; and,
  4. public awareness of and confidence in the criminal justice system, and civil justice system (with a focus on the family justice system).

In addition to questions on the topic areas outlined above, the questionnaire included questions on key demographic characteristics of respondents (e.g., gender, age, ethno-cultural group, Indigenous identity, income, employment).

1.2 Methodology

The 2023 National Justice Survey was conducted online with 4,487 people aged 18 years and older living in each of Canada’s provinces and territories from March 9 to May 18, 2023. Respondents from a pre-profiled database recruited randomly via telephone were sent SMS or email invites to participate in the study. The average survey length was 18 minutes. The survey data were weighed to replicate, as closely as possible, the Canadian population, aged 18 years or older by age, gender, region, highest level of education, Indigenous identity and ethno-cultural identity[1] using data from the 2021 Census of Population. Respondents were offered the survey in the official language of their choice.

An oversampling of First Nations, Inuit, and Métis, as well as those from racialized ethno-cultural groups was employed and important in the context of the growing calls for public sector organizations to efforts to collect disaggregated data, and to understand justice issues through an intersectional lens.

The quantitative survey instrument, all methodical information that would be required to replicate the research initiative, can be found in Appendix A. Tabulated results from the quantitative survey are provided in English and French under separate covers.

1.3 Contract value

The value of this contract, including HST, is $102,183.23.

1.4 Political neutrality requirement

Political neutrality certification

I hereby certify as Senior Officer of Advanis that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Policy on Communications and Federal Identity and the Directive on the Management of Communications.

Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leaders.

Signed
Signature
Nicolas Toutant
Vice-President, Research and Evaluation
Advanis

Sampling plan

Advanis used its General Population Random Sample (GPRS), which is a pre-profiled database of approximately 522,000 Canadians to find potential respondents. Advanis’ GPRS recruits people living in Canada using random digit telephone dialing (including both landline and cell phone numbers) to collect a random sample of potential respondents. From this randomly generated population sample, two more random samples were taken to populate the study sample: the first from the general population, and the second from those who have been identified as belonging to Indigenous or racialized ethno-cultural groups. The initial random samples were generated in February 2023. In May 2023, the data collection period was extended to address a lower response rate among younger Canadians. An additional randomly generated population sample was created from the pre-profiled database for those who have been identified as under 35 years of age. Questions within the survey further filtered out responses from anyone under the age of 18.

An oversampling of Indigenous respondents and respondents from racialized ethno-cultural groups was required to compare responses of Indigenous and racialized people, to those of White people. A minimum of 300 completed surveys was targeted from each of: Black, East/Southeast Asian, South/West Asian, Middle Eastern or North African, Latinx, First Nations, Inuk (Inuit) and Métis respondents. The final sample was weighted to known population proportions, resulting in a sample that is reasonably generalizable to the Canadian population. Data were collected through an online survey of approximately 18 minutes.

Table 1: Sample Frame by Indigenous and ethno-cultural identity
Indigenous identity and ethno-cultural identity Sample Targeted surveys
First Nations 2,445 300
Métis 1,924 300
Inuk (Inuit) 193 300 *
Black 1,932 300
East or Southeast Asian 2,407 300
Latinx 2,732 300
Middle Eastern or North African 1,712 300
South Asian 2,732 300
White 7,768 1,600
Multiple Indigenous/Ethno-cultural identities and prefer to self identify 256 -
Unknown 1,284 -
Total 25,385 4,000
* Best effort, based on low incidence in general population

3. Pretest and invitations

3.1 CATI recruit

Advanis used a CATI (Computer Assisted Telephone Interviewing) survey to recruit potential participants to participate in the online survey. Potential participants pulled from Advanis’ GPRS sample were randomly called and were asked if they agreed to participate in an online study. Those who agreed were sent an invitation via SMS and, in a few cases, via email. Overall, 17,698 people agreed to participate in the online survey. Invitations were also sent if the interviewer reached a potential participant’s voicemail.

3.2 Email and SMS invites/reminders

Invitations were sent by SMS and email. After sending the initial invitation, a reminder message was sent every three days to potential respondents who had not completed the survey and who had not previously been screened out of the survey. A maximum of five reminder messages were sent.

3.3 Survey pre-test and Data Collection

A test pre-launch was conducted on March 10, 2023. The pre-test was completed by 42 respondents in English, and 21 respondents in French. Respondents were asked to provide feedback on the survey instrument, including any suggestions that would improve the survey for others. Based on feedback received from 2 respondents participating in the pre-test, a small adjustment was made to one question to further clarify that, in the context of restorative justice, the victim(s) of homicide may include family and others directly affected by the offence.

A support email address and phone number were provided in the survey itself for any technical issues related to the survey. Surveys had a unique number embedded in the hyperlink to eliminate the possibility of duplicate responses from one participant.

4. Completed interviews

4.1 Results

The table below shows the details of initial invitations sent, unopened surveys, screened out applicants, as well as complete and incomplete surveys. The response rate, calculated as the number of in-scope responding units divided by the sum of unresolved units, in-scope non-responding units, and in-scope responding units, was 26.6%. Response rates for this type of survey and methodology generally range between 20% and 40%, depending on the subject matter and survey length. The overall response rate was lower relative to the 2022 National Justice Survey (32.9%), particularly among younger Canadians. In response to the lower initial response rate, the data collection period was extended, and an additional random sample was generated among those identified as under 35 years of age from the pre-profiled database of Canadians to increase the representativeness of the surveyed population.

Table 2: Results
Disposition Count
Total Email Invitations Issued (SMS/Email) 17,698
Invalid (incomplete/incorrect email address, email invitation bounce backs) 334
Total in-scope units 17,364
Total in-scope - non-responding units 12,750
Total unresolved units (no response at all) 11,745
Qualified respondent break-off (incomplete) 1,005
Total in-scope - responding units 4,614
Over quota 0
Other disqualified 127
Completed surveys 4,487
Response Rate (in-scope responding / total in-scope units) 26.6%

4.2 Margin of Error

Recruitment using Advanis’ General Population Random Sample panel was probabilistic as such, the margin of error can be reported. The margin of error for a sample of 4,487 is +/- 1.5 at the 95% Confidence Interval. The data reported was weighed to replicate, as closely as possible, the Canadian population, aged 18 years or older by age, gender, region, highest level of education, and Indigenous identity and ethno-cultural identity[2] using data from the 2021 Census of Population.

4.3 Completes and Targets

Overall, 17,698 people were invited to participate in the online survey. The average survey length was 18 minutes. In total, 4,487 surveys were completed. The number of completed surveys per ethno-cultural and Indigenous identity group matched approximately the number of completed surveys targeted by Justice Canada, with the exception of respondents identifying as Inuk (Inuit). This is due the smaller size of the Inuit population, as well as more difficulty in reaching this group.

Table 3: Number of Completes and Targeted Number of Completes
Indigenous identity and ethno-cultural identity Completed surveys Target
First Nations 308 300
Métis 337 300
Inuk (Inuit) 17 300 *
Multiple Indigenous identities 124
Black 327 300
East or Southeast Asian 336 300
Latinx 328 300
Middle Eastern or North African 333 300
South Asian 318 300
White 1,755 1,600
More than one of the groups above 147
Prefer to self-identify as: 28 -
Prefer not to answer 129 -
Total 4,487 4,000
* Best effort, based on low incidence in general population

A total of 91 respondents under the age of 18 were screened out of the online survey. Respondents who identified as more than one of the ethno-cultural groups listed, ‘Prefer to self-identify as:’, or chose not to indicate their ethno-cultural group were allowed to continue. ‘Prefer to self-identify as:’ mentions were later evaluated by analysts at Justice Canada and coded to existing categories, where applicable.

5. Weighting

Overall, 4,487 online surveys were completed during data collection. The final data were weighted by age, gender, region, highest level of education completed, and Indigenous identity and ethno-cultural identity to align with 2021 Census of Population estimates as closely as possible. The population proportions were sourced from Statistics Canada. Census Profile. 2021 Census. Statistics Canada Catalogue no. 98-401-X2021001 and Advanis applied the weights to the survey database. Detailed weights are available in the data set.

5.1 Weighting approach and methodology

Though a rake weighting scheme had been applied to the 2022 National Justice Survey data, it was determined that a cell weighting scheme, where the weights are computed so the sample totals conform to the population totals on a cell-by-cell basis, would be a more reliable way to weight the 2023 data, given the need to analyze several different sub-segments of the population. Additionally, Justice Canada requested a single weighting variable (rather than multiple weighting variables, specific to different segments of the data) to simplify the analysis process.

A pre-weighting analysis evaluated the representativeness of the sample population by age, gender, region, highest level of education completed, Indigenous identity and ethno-cultural identity. Further analysis was conducted to understand which respondent characteristics had a greater impact in overall perceptions of the justice system. These characteristics were given a more important consideration (relative to other weighting factors) in the final weighting approach.

Based on this analysis, the following factors were identified to prioritize in the weighting scheme:

  1. Age: 18-34, 35-54, 55+
  2. Gender: Men+, Women+
    1. After a review and subsequent back coding of self-identify responses by Justice Canada, the remaining 116 respondents who identified as a non-binary person, a two-spirit person, or who preferred to self-identify or chose not to answer were randomly assigned to Men+ or Women+ categories for the purposes of weighting and crosstabulation. Data for these populations could not be weighted due to the absence of reference data.
  3. Region: British Columbia, Ontario, Quebec, and the Rest of Canada (i.e., all other provinces and territories combined, including those who chose not to answer).
  4. Indigenous identity and ethno-cultural identity:
    1. First Nations, Métis, and Inuk;
    2. Black (e.g., African/African descent, Afro-Caribbean, Afro-Latino/Afro-Latina/Afro-Latinx);
    3. White (e.g., European/European descent);
    4. Latino/Latina/Latinx (e.g., non-white Latin American descent)
    5. South Asian/South Asian descent (e.g., Indian, Pakistani, Sri Lankan)
    6. Middle Eastern or North African/Middle Eastern or North African descent (e.g., Afghan, Iranian, Lebanese, Egyptian, Algerian, Saudi Arabian, Turkish, Iraqi, Yemeni, Jordanian);
    7. East Asian/ East Asian descent (e.g., Chinese, Japanese, Korean) or Southeast Asian/Southeast Asian descent (e.g., Filipino, Vietnamese, Cambodian, Laotian, Thai);
    8. More than one of the groups above, prefer to self-identify and those who chose not to answer.
  5. Highest level of education completed: University degree or higher, less than university degree or those who chose not to answer.

Analysis by other respondent characteristics (or by other groupings of the above weighting factors) should be done with caution and may not be representative of the analyzed population.

5.2 Estimating population proportions

The population proportions were sourced from Statistics Canada. Census Profile. 2021 Census. Statistics Canada Catalogue no. 98-401-X2021001. Since census data does not include counts that align with the proposed weighting scheme, the following process was used to develop reasonable estimates of the population by age, gender, region, highest level of education completed, Indigenous identity and ethno-cultural identity:

  1. Population counts for age, gender, and region were used to create 24 unique segments of the population. [Example: Age 18-34 | Male | British Columbia]
  2. Population counts for Indigenous identity and ethno-cultural identity by region were then used to estimate population counts for each of the 192 unique segments of the population. [Example: Age 18-34 | Male | British Columbia | First Nations, Métis, or Inuk (Inuit)]
  3. Finally, national statistics for the highest level of education completed were used to further sub-divide the population segments by education, resulting in 384 segments in total. [Example: Age 18-34 | Male | British Columbia | First Nations, Métis, or Inuk (Inuit) | University Degree or higher]

Population estimates for these 384 segments of the population were then compared to the surveyed population. When sample counts were insufficient to accurately weight the data, one or more elements of the group were aggregated to increase the size of the segment. For example, the following segments were combined into one segment:

Decisions regarding which elements to combine were based on the factors previously determined as having less impact on overall perceptions of the justice system. In the example above, it was determined that removing the regional weighting factor between British Columbia and the Rest of Canada (excluding Ontario and Quebec) would be less likely to affect results than combining these segments across age or gender categories.

The result of this approach is that data may not align exactly with all marginal populations, but the data will be more reliable when comparing sub-segments of the population. The final weighting scheme included 329 unique segments, with weights in the range of 0.04 to 4.67.

5.3 Comparability of 2022 and 2023 results

Both the 2022 and 2023 National Justice Survey data are weighted by the same factors (age, gender, region, highest level of education completed, Indigenous identity and ethno-cultural identity) so comparisons can be made between the two. As is the case with any random samples of the population, analysts should demonstrate caution when analyzing and comparing smaller sub-segments of the population, in consideration of the smaller base size and the process used for estimating population proportions that are not provided in the national census data (e.g., ethno-cultural identity by age and gender).

When comparing results relating to confidence and other perceptions of the Canadian justice system, it should also be noted that the 2023 National Justice survey included a labeled midpoint on rating scales.

Example:

Q2 (2023 Survey)

How confident are you that the Canadian criminal justice system is...

[Levels marked with * are randomized]

Q2 (2022 Survey)

How confident are you that the Canadian criminal justice system is...

[Levels marked with * are randomized]

Across most questions using rating scales, selection of the labeled midpoint is higher compared to the 2022 surveys where it was not labeled.

6. Appendix A: Quantitative Instruments

Introduction

Thank you for taking the time to complete this survey about justice issues of interest to Canadians.

Si vous préférez répondre au sondage en français, veuillez cliquer sur français.

This survey is voluntary but your input is valuable to us, and the survey will take about 15 minutes to complete. If you get interrupted while doing the survey, you can click on the same link to pick up right where you left off.

The Government of Canada has contracted an independent public opinion research company, Advanis , to conduct the research on their behalf. This survey uses Advanis' TellCityHall data collection methodology. Your personal information is treated with the highest standards of confidentiality by Advanis. We will not use this information for any purposes other than those consistent with market research requirements and guidelines. This survey is registered with the Canadian Research Insights Council's (CRIC) Research Verification Service. The project verification number is: 20230302-AD244 . Verify the legitimacy of this survey.

© 2023 Privacy Policy CRIC Pledge

AGE1

In what year were you born?

____________________

AGE2

In which of the following age categories do you belong?

Term2

Those are all the questions we have for you today. Thank you for your interest in this survey, but you must be 18 or older to participate. This research study is being conducted on behalf of the Department of Justice Canada. We thank you very much for taking the time to participate, it is greatly appreciated.

This study was hosted by market and social research firm: http://www.advanis.ca

using our data collection method: http://tellcityhall.ca/

If you'd like to see results from other studies conducted by Advanis via TellCityHall, please visit tellcityhall.ca/surveys .

© 2023 Privacy Policy CRIC Pledge

(Status code 501)

GENDR

I identify as: (Please only select one)

INDIG

I identify as: (Please only select one)

ETHN

I identify as: (Please only select one)

1 - Public Confidence in the Criminal Justice System

Q1

How would you describe your level of awareness of the role of the following area in the criminal justice system?

Q2

How confident are you that the Canadian criminal justice system is...

2 - Youth and Criminal justice System

Q3int

In the previous section, we asked you to tell us about your awareness of and confidence in the Canadian criminal justice system as a whole. This next section asks similar questions about the youth criminal justice system, specifically.

Press the right arrow to continue.

Q3

How would you describe your level of awareness of the Youth Criminal Justice Act ?

Q4

How confident are you that the Canadian youth criminal justice system is...

Q5int

In Canada, the Youth Criminal Justice Act (YCJA) is the law that guides the youth criminal justice system. The YCJA applies to youth aged 12 to 17 years old who are accused of committing criminal offences.

Youth who are found guilty of an offence are sentenced according to a number of principles that differ from adults (i.e., take into account the maturity of the young person; consider all available options other than custody that are reasonable in the circumstances). However, in exceptional cases, youth found guilty of certain types of offences could receive an adult sentence if a Youth Court Judge deems it appropriate under the circumstances. This means that the youth, for example, can be subject to mandatory minimum penalties, receive longer custody sentences, including life sentences, and receive an adult criminal record (permanent) rather than a youth criminal record (only accessible for up to five years after sentence is completed).

Note: youth who receive an adult sentence are still tried in youth court and are held in youth custody facilities until at least the age of 18. Press the right arrow to continue.

Q5

Under which conditions, if any, should the courts consider giving an adult sentence to a youth found guilty of an offence? (Select all that apply)

Q6

To what extent do you agree or disagree with the following statement:

Q7

To what extent do you agree or disagree with the following statement:

3 - Restorative Justice

Q8int

Now, turning to the use of restorative justice in the criminal justice system, please read the following information. A restorative justice (RJ) process provides a supportive and safe opportunity for communication between those who are affected by an offence (victims, offenders, and community). The goal of RJ is to repair harm, including by emphasizing victims’ healing and promoting meaningful accountability of offenders. The process is only used in cases where an offender has accepted responsibility for the offence and both the victim and the offender have voluntarily agreed to participate. A RJ process can be offered at any stage of the criminal justice process (pre-charge, post-charge, pre-sentence, during incarceration and post-sentence). There are many different types of RJ processes. Skilled facilitators guide these processes. Press the right arrow to continue.

Q8

How familiar are you with restorative justice?

Q9

Should a victim of crime be given the opportunity to communicate with an offender to tell them about the impact of the crime? This could be done through a representative, via a letter or video or an in-person meeting.

Q10

Provided that both the victim and the offender want to take part in the process, which of the following types of offences should be eligible for a restorative justice process?

Q11

How likely is it that restorative justice would:

Q12

To what extent do you agree or disagree with the following statement:

Q13

Do you believe that restorative justice would provide an experience for victims of crime that is:

Q14

Do you believe that restorative justice would provide an experience for offenders that is:

4 - The Canadian Family Justice System

Q15int

Now, turning to the family justice system. The family justice system in Canada extends beyond the courts and includes family justice services such as parent education sessions, supervised access, and family dispute resolution services. Issues such as divorce, parenting (custody and access), child and spousal support, as well as division of property are all handled by the family justice system.

Press the right arrow to continue.

Q15

How would you rate your knowledge of the family justice system in Canada?

Q16

How confident are you that the family justice system in Canada is...

Q17

Have you been involved with the family justice system in Canada within the past two years?

Q18

How would you describe your ability to access the family justice system in Canada?

Q19

What is your main source of information (that is, the source you would be most likely to use and access most often) about the Canadian family justice system? (Please only select one)

Q20

How comfortable would you be accessing the family justice system in the following scenario?

5 - Civil Justice System

Q21int

The family justice system is part of the civil justice system in Canada.

We would now like to get your views on the other aspects of the civil justice system.

General civil law includes, for example, disputes about contracts, wills, property, personal injury or employment issues. These disputes can occur between individuals or between individuals and organizations, including governments and businesses.

Press the right arrow to continue.

Q21

How confident are you that the civil justice system (excluding the family justice elements) in Canada is...

6 - Indigenous Justice

Q22int

Now, turning to Indigenous justice.

Indigenous communities have indicated that the revitalization of Indigenous customs, culture, language and an effort to support the rebuilding of Indigenous legal systems and processes is a priority.

Press the right arrow to continue.

Q22

To what extent do you agree or disagree with the following statement? Indigenous peoples in Canada should be able to administer their own systems of justice, including enforcing Indigenous laws and by-laws on their lands.

Demographics

DemoInt

There are just a few questions left in the survey. Your personal information is treated with the highest standards of confidentiality by Advanis.

Press the right arrow to continue.

CJSINVOL2

Have you been involved with the criminal justice system in the past two years? (Select all that apply)

EDU

What is the highest level of formal education that you have completed to date?

INC

Which of the following categories best describes your total household income? That is, the total income of all persons in your household, before taxes?

BORN

Were you born in Canada?

EMPL

Which of the following categories best describes your current employment status?

TYPE

In what type of community do you live?

LOC

What are the first three characters of your postal code?

____________________

PROVINCE

In which province or territory do you live?

EndTCH

That concludes the survey. This research study is being conducted on behalf of the Department of Justice Canada. We thank you very much for taking the time to participate, it is greatly appreciated.

This study was hosted by market and social research firm: http://www.advanis.ca

using our data collection method: http://tellcityhall.ca/

If you'd like to see results from other studies conducted by Advanis via TellCityHall, please visit tellcityhall.ca/surveys .

© 2023 Privacy Policy CRIC Pledge

(Status code -1)

Footnotes

  • 1 Based on Visible Minority population, Statistics Canada. 2022. Canada [Country] (table). Census Profile. 2021 Census. Statistics Canada Catalogue no. 98-401-X2021001
  • 2 Based on Visible Minority population, Statistics Canada. 2022. Canada [Country] (table). Census Profile. 2021 Census. Statistics Canada Catalogue no. 98-401-X2021001