National Justice Survey 2022

Methodological Report

Prepared For:
Department of Justice Canada (JUS)

Contract Information

  • Supplier Name: Advanis Inc.
  • Contract number: 19040-220002/001/CY
  • Contract Value: $68,440.21 (including HST)
  • Award date: January 19, 2022
  • Delivery date: May 31, 2022
  • Registration number: POR 100-21

For more information on this report, please contact Department of Justice Canada at: rsd.drs@justice.gc.ca

Ce rapport est aussi disponible en français.

Canada Logo

This public opinion research report presents the methodological details for the 2022 National Justice Survey conducted by Advanis Inc. on behalf of Department of Justice Canada (“Justice Canada”). The survey was administered with 4,949 Canadians between February 15th and March 10th, 2022.

Cette publication est aussi disponible en français sous le titre : Sondage national sur la justice 2022

This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Department of Justice Canada. For more information on this report, please contact the Department of Justice Canada at rsd.drs@justice.gc.ca or at:

Department of Justice
284 Wellington Street
Ottawa ON K1A 0H8

1. Executive Summary

1.1 Background and Objectives

The National Justice Survey (NJS) is an annual national public opinion research study conducted by the Department of Justice Canada (JUS). The survey collects information on the views, knowledge, concerns and priorities of Canadians on important justice-related issues. Results of the study are used to inform ongoing and future policy development, departmental reporting, communications, and public engagement on important justice-related issues.

The objective of the 2022 NJS was to collect information on the following:

  1. confidence in the fairness and accessibility of Canada’s criminal justice system;
  2. public awareness and understanding of the Canadian Victims Bill of Rights;
  3. the impact of COVID-19 on perceptions and experiences of access to Canada’s family justice system;
  4. perceptions of Canada’s youth criminal justice system, including the Youth Criminal Justice Act; and
  5. perceptions and concerns about cybercrime in Canada.

In addition to questions on the topic areas outlined above, the questionnaire included questions on key demographics (gender, age, ethno-cultural group, Indigenous identity, income, employment).

1.2 Methodology

The 2022 survey was conducted online with 4,949 people aged 18 years and older living in each of Canada’s provinces and territories from February 15 to March 10, 2022. Respondents from a pre-profiled database recruited randomly via telephone were sent SMS or email invites to participate in the study. The average survey length was 18 minutes. The survey data were weighed to replicate, as closely as possible, the Canadian population, aged 18 years or older by age, gender, region, highest level of education, Indigenous identity and ethno-cultural identity[1] using data from the 2016 Census of Population. Respondents were offered the survey in the official language of their choice.

An oversampling of First Nations, Inuit and Métis, as well as those from racialized ethno-cultural groups was employed and important in the context of the growing calls for public sector organizations to improve efforts to collect data to better understand and address issues of systemic racism in the justice system.

A full quantitative methodology report, including all information about the execution of the fieldwork that is needed to replicate the research initiative, can be found in Appendix A. The quantitative survey research instruments and a set of tabulated results from the quantitative surveys are provided in English and French under separate covers.

1.3 Contract value

The value of this contract, including HST, is $68,440.21.

1.4 Political neutrality requirement

Political neutrality certification

I hereby certify as Senior Officer of Advanis that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Policy on Communications and Federal Identity and the Directive on the Management of Communications.

Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leaders.

Signed
Signature
Nicolas Toutant
Vice-President, Research and Evaluation
Advanis

Sampling plan

Advanis used its General Population Random Sample (GPRS), which is a pre-profiled database of approximately 480,000 Canadians to find potential respondents. Advanis’ GPRS recruits people living in Canada using random digit telephone dialing (including both landline and cell phone numbers) to collect a random sample of potential respondents. From this randomly generated population sample, two more random samples were taken to populate the study sample: the first from the general population, and the second from those who have been identified as belonging to Indigenous or racialized ethno-cultural groups. Questions within the survey further filtered out responses from anyone under the age of 18.

An oversampling of Indigenous respondents and respondents from racialized ethno-cultural groups was required in order to compare responses of Indigenous and racialized people, to those of White people. A minimum of 300 completed surveys was targeted from each of: Black, East/Southeast Asian, South/West Asian, Middle Eastern or North African, Latinx, First Nations, Inuit and Métis respondents. The final sample was weighted to known population proportions, resulting in a sample that is reasonably generalizable to the Canadian population. Data was collected through a web survey of approximately 18 minutes.

Table 1: Sample Frame by Indigenous and ethno-cultural identity
Indigenous identity and ethno-cultural identity Sample Targeted surveys
First Nations 1,981 300
Métis 1,722 300
Inuk (Inuit) 210 300 *
Black 2,070 300
East or Southeast Asian 1,981 300
Latinx 2,003 300
Middle Eastern or North African 1,321 300
South Asian 2,305 300
White 5,905 1,600
Other 461 -
Unknown 1,062 -
Total 21,021 4,000

3. Pretest and invitations

3.1 CATI recruit

Advanis used a CATI (Computer Assisted Telephone Interviewing) survey to recruit potential participants to participate in the web survey. Potential participants pulled from Advanis’ GPRS sample were randomly called and were asked if they agreed to participate in an online study. Those who agreed were sent an invitation via SMS and via email in a few cases. Overall, 15,514 people agreed to receive an invitation to participate in the web survey. Invitations were also sent if the interviewer reached a potential participant’s voicemail.

3.2 Email and SMS invites/reminders

Invitations were mainly sent by SMS. Emails were sent only if it was requested specifically by the potential participant. After sending the initial invitation, a reminder message was sent every three days to potential respondents who had not completed the survey and who had not previously been screened out of the survey. A maximum of three reminder messages were sent.

3.3 Survey pre-test and Data Collection

A test pre-launch was conducted on February 15, 2022. The pre-test was completed by 23 respondents in English, and 15 respondents in French. Respondents were asked to provide feedback on the survey instrument, including any suggestions that would improve the survey for others. Minor question modifications were implemented based on the pre-test to address questions that were not understood by all respondents, and to reduce the overall survey length from an average of approximately 20 minutes to approximately 18 minutes.

A support email address and phone number were provided in the survey itself for any technical issues related to the survey. Surveys had a unique number embedded in the hyperlink to eliminate the possibility of duplicate responses from one participant.

4. Completed interviews

4.1 Results

The table below shows the details of initial invitations sent, unopened surveys, screened out applicants, as well as complete and incomplete surveys. The response rate, calculated as the number of in-scope responding units divided by the sum of unresolved units, in-scope non-responding units, and in-scope responding units, was 32.9%. Response rates for this type of survey and methodology generally range between 20% and 40%, depending on the subject matter and quenstionnaire length.

Table 2: Results
Disposition Count
Total Email Invitations Issued (SMS/Email) 15,514
Invalid (incomplete/incorrect email address, email invitation bounce backs) 10
Total in-scope - non-responding units 10,406
Total unresolved units (no response at all) 9,037
Qualified respondent break-off (incomplete) 1,369
Total in-scope - responding units 5,098
Over quota -
Other disqualified 149
Completed questionnaires 4,949
Response Rate (in-scope responding / (unresolved + in-scope non-responding + in-scope responding) 32.9%

4.2 Margin of Error

Although recruitment to Advanis’ General Population Random Sample panel was probabilistic in nature, the sample drawn from it for our survey cannot be considered probabilistic. As such, this would beconsidered a panel project and no margin of error can be reported. The data reported was weighed to replicate, as closely as possible, the Canadian population, aged 18 years or older by age, gender, region, highest level of education, and Indigenous identity and ethno-cultural identity[2] using data from the 2016 Census of Population.

4.3 Completes and Targets

Overall, 15,514 people were invited to participate to the web survey. The average survey length was 18 minutes. In total, 4,949 web questionnaires were completed. The number of completed surveys per Indigenous and ethno-cultural identity group matched approximately the number of completed surveys targeted by JUS.

Table 3: Number of Completes and Targeted Number of Completes
Indigenous identity and ethno-cultural identity Completed the web survey Target
First Nations 320 300
Métis 389 300
Inuk (Inuit) 28 300 *
Multiple Indigenous identities 52 -
Black 408 300
East or Southeast Asian 324 300
Latinx 316 300
Middle Eastern or North African 324 300
South Asian 368 300
White 2,110 1,600
More than one of the groups above 158 -
Other 35 -
Prefer not to answer 117 -
Total 4,949 4,000

A total of 149 respondents under the age of 18 were screened out of the web survey. Respondents who identified as more than one of the ethno-cultural groups listed, ‘other’, or chose not to indicate their ethno-cultural group were allowed to continue. ‘Other’ mentions were later evaluated by analysts in the Justice department and coded to existing categories, where applicable.

5. Weighting

Overall, 4,949 web interviews were conducted during data collection. The final data were weighted by age, gender, region, highest level of education, and Indigenous identity and ethno-cultural identity to align with 2016 Census of Population estimates as closely as possible. The population proportions were sourced from Statistics Canada. Census Profile. 2016 Census. Statistics Canada Catalogue no. 98-316-X2016001 and Advanis applied the weights to the survey database. Detailed weights are available in the data set. A rake weighting approach was used, resulting in hundreds of weights.

6. Appendix: Quantitative Instruments

Login

Thank you for taking the time to complete this survey dealing with justice issues of interest to Canadians.

Si vous préférez répondre au sondage en français, veuillez cliquer sur français.

This survey is voluntary but your input is valuable to us, and the survey will take about 15 minutes to complete. If you get interrupted while doing the survey, you can click on the same link to pick up right where you left off.

The Government of Canada has contracted an independent public opinion research company, Advanis (http://advanis.net), to conduct the research on their behalf. This survey uses Advanis' TellCityHall (http://www.tellcityhall.ca) data collection methodology. Your personal information is treated with the highest standards of confidentiality by Advanis. Survey answers will be grouped with other participant responses, and your personal information will only be used for the purposes stated in the survey. We will not use this information for any purposes other than those consistent with market research requirements and guidelines. This survey is registered with the Canadian Research Insights Council's (CRIC) Research Verification Service. The project verification number is: 20220208-AD296 Click here (https://www.canadianresearchinsightscouncil.ca/rvs/home/) to verify the legitimacy of this survey.

Intro

AGE1

In what year were you born?

Minimum: 1900, Maximum: 2022

__________

AGE2 [Show if 2004 or refused]

In which of the following age categories do you belong?

Term2 [Show if Born after 2004 or under18 or prefer not to say]

Those are all the questions we have for you today. Thank you for your interest in this survey, but you must be 18 or older to participate. This research study is being conducted on behalf of the Department of Justice Canada. We thank you very much for taking the time to participate, it is greatly appreciated.

This study was hosted by market and social research firm: (http://www.advanis.ca) using our data collection method: (http://tellcityhall.ca/)

If you'd like to see results from other studies conducted by Advanis via TellCityHall, please visit tellcityhall.ca/surveys (http://www.tellcityhall.ca/surveys.html).

GENDR

What is your gender?

Gender refers to current gender which may be different from what is indicated on your legal documents

Int1

In this survey, we are looking to understand opinions from diverse groups of Canadians. The next few questions will be used for statistical purposes only. All responses are strictly confidential.

Press the right arrow to continue.

INDIG

Are you First Nations, Inuk (Inuit), or Métis?

ETHN [Show if Do not identify as FNMI]

Do you identify as...?

Please select the category that you identify with most.

Public Confidence in the Criminal Justice System

Q1

How would you describe your level of awareness when it comes to the role of the following areas in the criminal justice system?

[Levels marked with * are randomized]

Q2

How confident are you that the Canadian criminal justice system is...

[Levels marked with * are randomized]

Confidence in the Family Justice System

Q3

Now, turning to the family justice system. The family justice system in Canada extends beyond the courts and includes family justice services such as parent education sessions, supervised access, and family dispute resolution services. Issues such as divorce, parenting (custody and access), child and spousal support, as well as division of property are all handled by the family justice system.

How would you rate your knowledge of the family justice system in Canada?

Q4

How confident are you that the family justice system in Canada is...

[Levels marked with * are randomized]

Q4a

Have you been involved with the family justice system in Canada within the past two years?

Q5 [Show if Q4a Involved in FJS past 2years]

How would you describe your ability to access the family justice system in Canada?

Q6

What is your main source of information (that is, the source you would be most likely to use and access most often) about the Canadian family justice system?

Choose one

[Levels marked with * are randomized]

Q7

How comfortable would you be accessing the family justice system in the following scenarios?

[Levels marked with * are randomized]

Victims of Crime

Q8

In July 2015, the Canadian Victims Bill of Rights came into force to ensure that rights of victims are considered during every stage of the criminal justice process. Victims have the right to information, the right to participation, the right to seek restitution, and the right to protection.

Prior to taking this survey, how would you describe your level of awareness of the Canadian Victims Bill of Rights?

Q9

How important do you think it is for Canadians to know about the Canadian Victims Bill of Rights?

Q10aint

The Criminal Code contains a number of provisions that make it easier for victims and witnesses to provide their testimony in court proceedings. With this in mind, we are interested to know what you think about animals, such as trained dogs, supporting witnesses when they testify.

Press the right arrow to continue.

Q10a

Please indicate if you disagree or agree with the following statements:

[Levels marked with * are randomized]

Q10bint

Other sections of the Criminal Code allow victims and witnesses to testify outside the courthouse (such as from a police station or their own home) using technology. With this in mind, we are interested to know what you think about this approach during and beyond COVID-19.

Press the right arrow to continue.

Q10b

Please indicate if you disagree or agree with the following statements:

[Levels marked with * are randomized]

Youth and Criminal justice System

Q11int

This section includes a series of questions on the youth criminal justice system. In Canada, the Youth Criminal Justice Act (YCJA) is the law that guides the youth criminal justice system. The YCJA applies to youth aged 12 to 17 years old who are accused of committing criminal offences.

Press the right arrow to continue.

Q11

Prior to this survey, how would you describe your level of awareness of the Youth Criminal Justice Act?

Q12

How confident are you that the Canadian youth criminal justice system is...

[Levels marked with * are randomized]

Q13

What is your main source of information (that is, the source you would be most likely to use and access most often) about the youth criminal justice system?

Choose one

[Levels marked with * are randomized]

Q14int

The Youth Criminal Justice Act (YCJA) focuses on resolving matters outside the courts because it recognizes that youth are not as emotionally and cognitively mature as adults. The act provides ways to deal with some of the issues faced by youth that may have led them to committing crimes, like poverty, mental illness or cognitive disabilities, and substance use and addiction. The YCJA also helps young people take responsibility and be accountable for their actions when they are accused of a crime. Custody sentences are only given for the most serious crimes. With this in mind, we are interested to know what you think about Canada’s approach to youth crime.

Press the right arrow to continue.

Q14

Please indicate if you disagree or agree with the following statements:

Q14b

When dealing with a youth involved in the criminal justice system, the following groups should be given the opportunity to be involved (e.g., input into what accountability measures are taken):

[Levels marked with * are randomized]

Q14a

Under which conditions, if any, should youth be sentenced to custody?

Select all that apply

Q15int

Indigenous and racialized youth are overrepresented in the youth criminal justice system and are more likely to be sentenced to custody than white youth. Policy makers are working to reduce this overrepresentation and advance reconciliation with Indigenous peoples by eliminating systemic discrimination from the justice system.

Press the right arrow to continue.

Q15

Please indicate to what extent you disagree or agree with the following statements:

Q16int

Compared with the youth system, the adult system has limited ability to deal with issues outside the formal criminal justice system, and relies much more on custody sentences. That said, recent research suggests that some people may not reach brain maturity until about the age of 25. This may make young adults (aged 18-24) less emotionally and cognitively mature, less capable of anticipating the consequences of their actions and at greater need for social supports, than older adults.

Press the right arrow to continue.

Q16a

Please indicate to what extent you disagree or agree with the following statements.

Young adults in the following age range should be held accountable differently than older adults, for instance by receiving services and supports that are tailored to their level of maturity:

Q16b

Please indicate to what extent you disagree or agree with the following statements.

Young adults in the following age range who have been professionally assessed as not having the required level of maturity to be held accountable using the adult system, should be held accountable using the youth justice system:

Q16c

Please indicate to what extent you disagree or agree with the following statements.

All young adults in the following age range should be held accountable using the youth justice system instead of the adult justice system:

Q17

Under the Youth Criminal Justice Act, the privacy of young persons who are accused or found guilty of a crime is highly protected and access to their youth records is very restricted, moreso than for adults. Currently, access to adult records can be granted for research purposes, however youth records cannot be accessed unless a judge approves.

To what extent do you disagree or agree that access to youth records should be allowed for research purposes? Please assume that privacy measures would be in place to ensure that youth are not identifiable.

Perceptions of Cybercrime

Q18

We are now interested to know your perceptions of cybercrime in Canada. The RCMP defines cybercrime as a crime where the Internet, and computers, tablets or mobile devices have a substantial role in the commission of an offence. This is the last issue we will ask you about in this survey.

In the past two years, how concerned have you been about the threat of cybercrime (such as cyberbullying, ransomware)?

Q19

Have work from home policies during the COVID-19 pandemic impacted your level of concern about the threat of cybercrime?

Q20

Who do you think is mostly responsible for cybercrime against people residing in Canada?

Demographics

Demographics

There are just a few questions left in the survey. The following questions will be used for statistical purposes only. All responses are strictly confidential.

Press the right arrow to continue.

CJSINVOL2

Have you been involved with the criminal justice system in the past two years?

Select all that apply

EDU

What is the highest level of formal education that you have completed to date?

INC

Which of the following categories best describes your total household income? That is, the total income of all persons in your household, before taxes?

BORN

Were you born in Canada?

EMPL

Which of the following categories best describes your current employment status?

TYPE

In what type of community do you live?

LOC

What are the first three characters of your postal code?

__________________________________________________

PROVINCE [Show if LOC Not answered]

In which province or territory do you live?

End

That concludes the survey. Those are all the questions we have for you today. This research study is being conducted on behalf of the Department of Justice Canada. We thank you very much for taking the time to participate, it is greatly appreciated.

This study was hosted by market and social research firm: (http://www.advanis.ca) using our data collection method: (http://tellcityhall.ca/)

If you'd like to see results from other studies conducted by Advanis via TellCityHall, please visit tellcityhall.ca/surveys (http://www.tellcityhall.ca/surveys.html).

Footnotes

  • * Best effort, based on low incidence in general population
  • * Best effort, based on low incidence in general population
  • 1 Based on Visible Minority population, Statistics Canada. 2017. Canada [Country] (table). Census Profile. 2016 Census. Statistics Canada Catalogue no. 98-316-X2016001
  • 2 Based on Visible Minority population, Statistics Canada. 2017. Canada [Country] (table). Census Profile. 2016 Census. Statistics Canada Catalogue no. 98-316-X2016001