Prepared for Public Safety and Emergency Preparedness Canada
For more information on this report, please contact ps.communications-communications.sp@canada.ca
This public opinion research report presents the methodology for two online surveys conducted by Ekos Research Associates Inc. on behalf of Public Safety Canada. The research was conducted in two phases, the first with 2,118 Canadian adults and was collected between June 17 and 29, 2021. The second phase included 2,019 Canadian adults and was collected between March 7 and 18, 2022.
Cette publication est aussi disponible en français sous le titre : Évaluation de la campagne de sensibilisation du public aux armes à feu – Rapport méthodologique de l'OÉCP.
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Public Safety Canada. For more information on this report, please contact ps.communications-communications.sp@canada.ca.
Related publications registration number: POR-010-21
© Her Majesty the Queen in Right of Canada, as represented by the Minister of Public Works and Government Services, 2022.
Ekos Research Associates Inc. is pleased to submit this report to Public Safety Canada detailing the methodology of two online surveys assessing the ad recall from the Firearms Public Awareness Campaign. The goal of this campaign was to raise awareness about the rise of gun violence in Canada and the actions being taken by the Government of Canada to address it.
The Minister of Public Safety's mandate letter outlined the Government's commitment to take action to further reduce firearms-related violence, including by strengthening gun control laws, targeting gang violence and illegal firearms trafficking, and banning assault-style firearms. On May 1, 2020, the Government of Canada announced the prohibition of over 1,500 models of assault-style firearms and certain components of some newly prohibited firearms. These models represent nine categories of firearms and two types identified by characteristic.
Public Safety Canada has developed a national campaign to raise awareness and educate Canadians and firearms owners of the need to reduce violent crime by limiting illicit access to firearms. The campaign focused on raising awareness and understanding among Canadians 18+ (slight skew to men) to raise awareness and understanding of the actions being taken by the Government of Canada to address gun violence.
The research included two components:
The results of this research will allow Public Safety Canada to assess the recall of the ad campaign based on feedback from the survey data and analysis.
Two bilingual, national online surveys were conducted: one as a pre-campaign baseline, in advance of the campaign launch, and one at the end of the campaign to measure change in awareness, and to assess the impact of the campaign. The baseline survey included 2,118 Canadian adults, and was collected between June 17 and 29, 2021. The post campaign sample included 2,019 Canadians adults, and was collected between March 7 and 18, 2022. Each sample strived for a 60% representation among men as previous research conducted on behalf of Public Safety Canada showed that men are slightly more represented in the firearms owners audience and this demographic information should be taken into account due to the subject matter of the campaign. Each survey instrument included the Advertising Campaign Evaluation Tool (ACET) questions used by the Government of Canada in all of its advertising evaluation research, along with additional questions testing impact on perceptions targeted through the campaign. Each sample included randomly selected respondents from all provinces and territories.
Each survey sample relied on Ekos' Probit panel, which is assembled using a random digit dial process for sampling from a blended land-line cell-phone frame, and provides full coverage of Canadians with telephone access. The distribution of the recruitment process is meant to mirror the actual population in Canada (as defined by Statistics Canada). As such, our more than 120,000 member panel can be considered representative of the general public in Canada (meaning that the incidence of a given target population within our panel very closely resembles the public at large) and margins of error can be applied.
Appendix A provides the details of the methodology for these pre- and post-campaign surveys. The pre- and post-campaign questionnaires can be found in Appendix B and C.
The opinions and observations expressed in this document do not reflect those of Public Safety Canada. This report was compiled by Ekos Research Associates Inc. based on research conducted specifically for this project.
I hereby certify as Senior Officer of Ekos Research Associates Inc. that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the on Communications and Federal Identity and the Directive on the Management of Communications. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leaders.
Signed by: Susan Galley (Vice President)
The Advertising Campaign Evaluation Tool (ACET) consisted of a baseline and post-campaign survey, featuring questions used by the Government of Canada in all of its advertising evaluation research. The ACET focused on recall of the campaign, along with questions testing the impact of the ads on perceptions and behaviour. The average time it took respondents to complete the survey was seven minutes for the baseline, and 10 minutes for the post-campaign survey. The baseline survey was pre-tested with 30 respondents (20 in English, 10 in French), and the post-campaign survey was pre-tested with 27 respondents (17 in English, 10 in French). Since no significant changes were made these cases were retained in the final sample.
In each case, respondents to the online survey were 18 years of age and older and were randomly selected, striving for a mix of 60% men and 40% women. The sample included all provinces and territories, and the survey was administered in English and French, as well as with an accessible link for those using a mobile phone or screen reading technology. The survey sample relied on an Ekos' Probit panel, which is assembled using a random digit dial process for sampling from a blended land-line cell-phone frame and provides full coverage of Canadians with telephone access. The distribution of the recruitment process is meant to mirror the actual population in Canada (as defined by Statistics Canada). As such, our more than 120,000-member panel can be considered representative of the general public in Canada (meaning that the incidence of a given target population within our panel very closely resembles the public at large) and margins of error can be applied. All households/individuals in the Probit panel are contacted by telephone and the nature of the panel is explained in greater detail (as are Ekos' privacy policies) and demographic information is collected. At this time, the online/off-line as well as landline/cell phone status is ascertained to determine the method of completing surveys (i.e., online, telephone, or mail). This variable of ‘type of telephone service' (cell phone only, landline only or both) collected at the time of screening is used to determine cell phone only sample. As with any random digit dialing sample, Probit panel cases are considered to be a probability-based sample.
In the baseline survey, the total number of completed cases in the sample was 2,118 and was collected between June 17 and 29, 2021. The post-campaign survey was collected between March 7 and 18, 2022 including 2,019 cases. The associated margin of error is up to plus or minus 2.2%, at a .95 confidence interval (i.e., 19 times out of 20) in each sample. The regional distribution is as follows:
| Province/Region | Baseline | Post-Campaign |
|---|---|---|
| British Columbia and Territories | 303 | 280 |
| Alberta | 243 | 248 |
| Manitoba & Saskatchewan | 142 | 139 |
| Ontario | 836 | 814 |
| Quebec | 452 | 428 |
| Atlantic | 142 | 110 |
| Total | 2,118 | 2,019 |
Survey data collection adhered to the Government of Canada standard for public opinion research as well as all applicable industry standards. Ekos informed respondents of their rights under the Privacy Act and the Access to Information Act and ensured that those rights were protected throughout the research process. This included: informing respondents of the purpose of the research; identifying both the sponsoring department and the research supplier; informing respondents that their participation in the study is voluntary, and that the information provided would be administered according to the requirements of the Privacy Act.
Once the survey data was collected, each database was reviewed for data quality. Coding was also completed. Survey results were weighted based on Statistics Canada data according to age, gender and region to ensure the sample was representative of the general public aged 18 years and older.
Data tables were created for each survey to isolate results for major subgroups to be used in the analysis (e.g., results for each age segment, gender, region, education and household income). Other subgroups include mother tongue, respondents born in Canada, family with children, employment, and Government of Canada ad recall (in general and for firearms).
The response rate for the online pre-campaign baseline survey was 18.7%, using a formula developed by the Marketing Research Intelligence Agency in conjunction with the Government of Canada. A total of 13,237 email invitations were sent, of which 58 were returned as undeliverable, for a resulting valid sample of 13,179. A total of 2,118 valid cases were completed and combined with another 356 respondents who were screened out of the survey as ineligible[1], as the responding numerator in the calculation of the response rate. In the post-campaign, 9,673 invitations were sent, of which 495 were undeliverable for a valid sample of 9,178. In addition to the 2,019 valid cases that completed the survey, 293 respondents were screened out of the survey. The resulting response rate was 25.1%.
| Col1 | Pre-campaign | Post-Campaign |
|---|---|---|
| Total Invited | 13,237 | 9,673 |
| Invalid | 0 | 0 |
| Unresolved | 58 | 495 |
| In-scope Responding | 10,705 | 6,866 |
| Responding Units | 2,474 | 2,312 |
| Response Rate (R/(U+IS+R) | 18.7% | 25.1% |
A comparison of each unweighted sample with 2016 Census figures from Statistics Canada suggests that there are similar sources of systematic sample bias in the survey, following patterns typically found in most general public surveys. There is a somewhat more educated sample in each survey than found in the population with 31% reporting university degrees in the baseline and 29% in the post-campaign survey, compared with 23% in the population. There is also an under representation of Canadians born outside of Canada in each survey (14% in the baseline and 12% in the post-campaign survey versus 22%). As previously described, each sample was weighted by age, gender and region.
Thank you for taking the time to complete this survey dealing with current issues of interest to Canadians.
Si vous préférez répondre au sondage en français, veuillez cliquer sur français.
Your participation is voluntary and your responses will be kept entirely confidential. The survey takes about 5 minutes to complete.
The survey is registered with the Research Verification Services operated by the Canadian Research Insights Council (CRIC). Click here if you wish to verify its authenticity (project code 20210614-EK988053-21Quantitative2021-06-142021-07-10130002000). To view our privacy policy, click here.
If you require any technical assistance, please contact online@ekos.com.
Do you, or does anyone in your household, work for any of the following organizations?
Select all that apply
Are you...
In what year were you born?
Hesitant, QAGEX; <18 or NR, terminate
In which of the following age categories do you belong?
In which province or territory do you live?
Core Questions
Over the past three weeks, have you seen, read or heard any advertising from the Government of Canada?
Think about the most recent ad from the Government of Canada that comes to mind. Where have you seen, read or heard this ad?
Select all that apply
What do you remember about this ad?
How did you know that it was an ad from the Government of Canada?
Campaign Specific Questions
Over the past three weeks, have you seen, read or heard any Government of Canada advertising about firearms or gun violence?
Where have you seen, read or heard this advertising about firearms or gun violence?
Select all that apply
What do you remember about this ad?
Have you heard anything about reducing gun violence in Canada?
Where have you seen, read or heard about reducing gun violence?
Select all that apply
Have you heard of any new measures the Government of Canada has announced lately to address gun-related violence?
What new measures have you heard about or seen?
How much of a threat to public safety do you think gun violence is in your community?
Would you say that the regulation of firearms in Canada is too strict, not strict enough, or strikes the right balance?
To what extent do you support or oppose the ban of assault-style firearms in Canada?
Demographic Questions
Finally, a few questions for statistical purposes only. Please be assured that all of your answers will remain completely confidential.
Which of the following categories best describes your current employment status? Are you...
Select only one
What is the highest level of formal education that you have completed?
Select only one
Are there any children under the age of 18 currently living in your household?
Which of the following categories best describes your total annual household income, including income from all household members, before taxes are deducted?
Where were you born?
Born outside Canada, D5
In what year did you first move to Canada?
What is the language you first learned at home as a child and still understand?
Select up to two
Do you live in a household where there are firearms present?
That concludes the survey. This survey was conducted on behalf of Public Safety Canada. We thank you very much for taking the time to answer this survey, it is greatly appreciated.
THNK2
We regret that your responses have shown that you are ineligible to participate in this survey. Thank you for your time.
Thank you for taking the time to complete this survey dealing with current issues of interest to Canadians.
Si vous préférez répondre au sondage en français, veuillez cliquer sur français.
Your participation is voluntary and your responses will be kept entirely confidential. The survey takes about 5 minutes to complete.
The survey is registered with the Research Verification Services operated by the Canadian Research Insights Council (CRIC). Click here if you wish to verify its authenticity (project code 20210614-EK988053-21Quantitative2021-06-142021-07-10130002000). To view our privacy policy, click here.
If you require any technical assistance, please contact online@ekos.com.
Do you, or does anyone in your household, work for any of the following organizations?
Select all that apply
Are you...
In what year were you born?
Hesitant, QAGEX; <18 or NR, terminate
In which of the following age categories do you belong?
In which province or territory do you live?
Core Questions
Over the past three weeks, have you seen, read or heard any advertising from the Government of Canada?
Yes, Q1
Think about the most recent ad from the Government of Canada that comes to mind. Where have you seen, read or heard this ad?
Select all that apply
Yes, Q1
What do you remember about this ad?
Yes, Q1
How did you know that it was an ad from the Government of Canada?
Campaign Specific Questions
Over the past three weeks, have you seen, read or heard any Government of Canada advertising about firearms or gun violence?
Yes, T1A
Where have you seen, read or heard this advertising about firearms or gun violence?
Select all that apply
Yes, T1A
What do you remember about this ad?
Have you heard anything about reducing gun violence in Canada?
Yes, T1DA
Where have you seen, read or heard about reducing gun violence?
Select all that apply
Have you heard of any new measures the Government of Canada has announced lately to address gun-related violence?
What new measures have you heard about or seen?
How much of a threat to public safety do you think gun violence is in your community?
Would you say that the regulation of firearms in Canada is too strict, not strict enough, or strikes the right balance?
To what extent do you support or oppose the ban of assault-style firearms in Canada?
Here are some ads that have recently been broadcast on various media.
[Displays a YouTube 30 second video ad]
[Displays a Youtube 6 second video ad, randomized among 3]
Over the past two months, have you seen these ads?
Here are some ads that have recently been broadcast on various media.
[Displays a set of 3 banner ads, randomized among 3 set]
Over the past two months, have you seen this ad?
Here are some ads that have recently been broadcast on various media.
[Displays a set of 3 banner ads, randomized among 2 set]
Over the past two months, have you seen this ad?
Yes, T1IA, T1IB or T1IC
Where have you seen these [T1IA = 1 and T1IB = 2,99 and T1IC = 2,99] video ads? [Else] ads?
Click here to review the ads you've seen.
Select all that apply
What do you think is the main point these ads are trying to get across?
Please indicate your level of agreement with the following statements about these ads.
Click here to review the ads.
These ads catch my attention
These ads are relevant to me
These ads are difficult to follow
These ads do not favour one political party over another
These ads talk about an important topic
These ads provide new information
These ads clearly convey that the Government of Canada has taken measures recently to address gun-related violence
Demographic Questions
Finally, a few questions for statistical purposes only. Please be assured that all of your answers will remain completely confidential.
Which of the following categories best describes your current employment status? Are you...
Select only one
What is the highest level of formal education that you have completed?
Select only one
Are there any children under the age of 18 currently living in your household?
Which of the following categories best describes your total annual household income, including income from all household members, before taxes are deducted?
Where were you born?
Born outside Canada, D5
In what year did you first move to Canada?
What is the language you first learned at home as a child and still understand?
Select up to two
Do you live in a household where there are firearms present?
That concludes the survey. This survey was conducted on behalf of Public Safety Canada. We thank you very much for taking the time to answer this survey, it is greatly appreciated.
We regret that your responses have shown that you are ineligible to participate in this survey. Thank you for your time.