Image 1 - Text Description
A still representation of video where a man struggles to carry a large cinderblock on his shoulder while a woman looks at him. The text reads, "Addiction can be a heavy load to carry."
Methodological Report
Prepared for Health Canada
For more information on this report, please contact Health Canada at: hc.cpab.por-rop.dgcap.sc@canada.ca.
Ce rapport est aussi disponible en français
I hereby certify as Senior Officer of Ipsos that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.
Mike Colledge
President
Ipsos Public Affairs
This public opinion research report presents the methodology of Pre- and Post-Evaluation of Stigma and Opioid Harm Reduction Advertising Campaign online survey conducted by Ipsos Public Affairs on behalf of Health Canada. The pre-campaign survey was conducted with a sample of n=2319 Canadians ages 18+ between July 27th and August 17th, 2022, and the post-campaign survey was conducted with a sample of n=2,327 Canadians ages 18+ between April 17th and 30th, 2023.
Cette publication est aussi disponible en français sous le titre : OECP avant et après la campagne publicitaire sur la réduction de la stigmatisation et des méfaits liés aux opioïdes
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Health Canada. For more information on this report, please contact Health Canada at hc.cpab.por-rop.dgcap.sc@canada.ca or at:
Communications Branch
Health Canada
200 Eglantine Driveway
AL 1915C, Tunney's Pasture
Ottawa, ON K1A 0K9
Catalogue Number: H14-447/2023E-PDF
International Standard Book Number (ISBN): 978-0-660-49593-4
Related publications (registration number: POR 012-22):
©His Majesty the King in Right of Canada, as represented by the Minister of Health, 2023
The growing number of overdoses and deaths related to the use of opioids is a national public health crisis that has been exacerbated by the COVID-19 pandemic. It is estimated that 26,690 people in Canada have died from apparent opioid-related overdoses between January 2016 and September 2021. Because of the COVID-19 pandemic, the years 2020 and 2021 have been the most fatal to date for the opioid overdose crisis with a 95% increase in apparent opioid toxicity deaths occurring (April 2020 to March 2021) from the same time period prior to the pandemic. Since then, deaths have remained high, with 5,368 apparent opioid toxicity deaths having occurred between January and September 2021. This is approximately 20 deaths per day. For a similar timeframe in the years prior to the pandemic, there were between seven (in 2016) and 12 (in 2018) deaths per day.
Responding to the opioid overdose crisis is a 2021 mandate commitment for the Minister of Health, who has been tasked with advancing a comprehensive strategy to address problematic substance use in Canada and support efforts to improve public education to reduce stigma. As part of that commitment, a campaign was launched that focused on reducing the barriers created by stigma such as seeking treatment.
Two separate online surveys were conducted to assess the impact of the campaign. The pre-campaign survey was conducted between July 27th and August 17th, 2022, and the post-campaign was conducted between April 17th and 30th, 2023. Both surveys were conducted in English and French.
Ipsos programmed, hosted, and provided sample management services, while Health Canada provided the online questionnaires. Ipsos was responsible for data collection and data storage in Canada, data processing, and data weighting. The total contract value of this research was $85,433.65, including HST.
The opioid crisis is complex, and it affects all communities, all age groups, and all socioeconomic groups. However, men working in physically demanding jobs are at increased risk from opioid-related harms, as young to middle-aged men account for more than three-quarters of apparent opioid overdose-related deaths. Disproportionate amounts of these men are employed in physically demanding, male-dominated occupations, such as construction. There is an urgent need to address the overrepresentation of this group in the opioid overdose crisis.
In an effort to target and create messaging that resonates with men in physically demanding jobs, a new creative concept with a new visual identity has been developed as part of Health Canada's multi-year marketing campaign to address the growing opioid overdose crisis. The campaign aims to:
The Government's Policy on Communications and Federal Identity requires the evaluation of advertising campaigns exceeding $1 million in media buy using the Advertising Campaign Evaluation Tool (ACET). The ACET was created in 2002 following a Cabinet directive identifying the need for a standard advertising evaluation approach across departments. The main objectives were to bring rigor and consistency to ad campaign evaluation and to develop norms on metrics against which campaigns could be evaluated.
Recent changes to reporting now require both pre- and post-campaign ACET surveys that use an online panel and similar questions.
The objectives of the research are as follows:
The target audience of the research was as follows:
This project involved two (2) waves of surveying – a pre-campaign survey and a post-campaign survey. Both surveys were executed online using a non-probability online panel. This is the standard approach for all Government of Canada advertising evaluation surveys. The initial baseline survey was conducted before the campaign launched and the recall survey was conducted following the completion of the campaign.
Respondents for the survey samples were drawn from a trusted partner panel vendor, Canadian Viewpoint Inc. The pre-campaign survey was conducted with a sample of n=2,319 Canadians ages 18+ and the post-campaign survey was conducted with a sample of n=2,327 Canadians ages 18+. Each phase included around n=2,000 members of the general population, including oversamples of males aged 20 to 59 who work in physically demanding professions, so that the total sample size for this target group is n=500. Interviews were conducted in English and French, with respondents being able to complete the survey in either language of their choice.
For the pre-campaign survey, a pre-test was conducted on July 27th, with n=20 completes (n=11 English / n=9 French), to confirm survey length before fully deploying the questionnaire. No issues with the survey or collected data were flagged. The survey was fully launched and ran between July 27th and August 17th, 2022.
For the post-campaign survey, a pre-test was conducted on April 17th, 2023, with n=155 completes (n=133 English / n=22 French). Again, no issues were flagged. The survey was fully launched and ran between April 18th and 30th, 2023.
For both surveys, quotas were set to ensure representation by region, age, and gender, according to the latest Census information. Sampling targets were obtained through natural fallout. An oversample of males aged 20 to 59 who work in physically demanding professions was conducted per survey, n=500 in the pre-survey and n=502 in the post-survey. The unweighted oversample counts are shown below.
| Target audience | Pre-campaign | Post-campaign |
|---|---|---|
| Males aged 20 to 59 who work in physically demanding professions | 500 (421 wtd) | 502 (414 wtd) |
The sampling methodology utilized email invitations and router technology to invite participants. Each participant received a unique URL link. This link could only be used once, with respondents being allowed to pause during completion and return to complete. On average, the pre-campaign survey was approximately 3 minutes in length (LOI 2:58 min), while the post-campaign ACET was approximately 6 minutes in length (LOI: 6:13 min).
Ipsos partnered with sub-contractor Canadian Viewpoint Inc. Canadian Viewpoint has one of the largest consumer panels with ~300,000 active panellists. Like Ipsos, Canadian Viewpoint uses mixed-medium strategies (phone/online/in-person) to build its panel to allow more representative samples. Canadian Viewpoint's online consumer panel is also recruited using post screeners during telephone surveys, Facebook, online vetted lists and in-person studies. Incentives were not used for recruitment purposes to ensure quality, but respondents were incentivized for completing the survey, which is directly proportionate to the amount of time taken to complete the survey and with comparable incentives offered by other online panel sources.
The comprehensive background profiling data were gathered when respondents joined the panel through screener questions, which allowed for the targeting of respondents based on key criteria, such as region, age, gender, education and income level, profession, and other characteristics.
The online survey was conducted using Computer Assisted Web Interviewing (CAWI). Every panellist opted-in or chose to participate in research surveys. Panellists that meet the criteria we were looking for were randomly selected and sent an email invitation to complete the survey. Participants were offered a choice to complete the survey in either English or French.
The survey platform was Accessibility for Ontarians with Disabilities Act (AODA) compliant, according to Web Content Accessibility Guidelines (WCAG 2.0AA).
It is important to note that any panel – no matter how well recruited – does not enable random selection in the sense that not all members of the public are likely to join an online survey panel. This applies to panels recruited through online approaches as well as panels recruited via traditional telephone Random Digit Dialing (RDD) in that the sampling naturally skews towards those who are amenable to answering a survey.
Canadian Viewpoint uses a multi-faceted approach to quality, integrating controls and best practices into every phase of the process. Panel members come from diverse fully certified sources. A dedicated team monitors performance by source using feedback loops and continuous reporting to prevent fraud. Our dedicated team monitors the panels for potentially fraudulent behaviour and blocks these people from ever entering a survey via safeguards that includes:
The table below shows the unweighted and weighted distributions of the online sample. Weighting was applied to the both the pre-campaign and post-campaign samples to ensure that the final data reflects the general adult population by region, age, and gender according to the 2021 Census. A Random Iterative Method (RIM) technique was applied for weighting.
| Pre-campaign Survey | Post-campaign Survey | |||
|---|---|---|---|---|
| Unweighted Sample Size | Weighted Sample Size | Unweighted Sample Size | Weighted Sample Size | |
| Canada | 2319 | 2319 | 2327 | 2327 |
| Region | ||||
| British Columbia/Yukon | 292 | 323 | 313 | 324 |
| Alberta/Northwest Territories | 273 | 258 | 246 | 259 |
| Prairies (MB/SK) / Nunavut | 155 | 147 | 172 | 148 |
| Ontario | 953 | 900 | 949 | 903 |
| Quebec | 475 | 535 | 484 | 537 |
| Atlantic Canada | 171 | 155 | 163 | 148 |
| Gender | ||||
| Male | 1271 | 1123 | 1289 | 1127 |
| Female | 1033 | 1181 | 1023 | 1185 |
| Diverse / Prefer not to Answer | 15 | 12 | 15 | 9 |
| Age | ||||
| 18-34 | 638 | 619 | 663 | 621 |
| 35-54 | 845 | 747 | 837 | 750 |
| 55+ | 836 | 953 | 827 | 957 |
The figures presented in the table above show minimal differences between the unweighted and weighted samples. As a result of the strong representativeness of the unweighted data, the largest weight factor that was applied for any respondent is 1.16, which is well within acceptable ranges for a survey of the general population.
For this survey, a sample router was used. Therefore, a response rate cannot be calculated. However, the participation rate for the pre-campaign survey was 96% and the post-campaign survey was 97%. Participation rate is calculated as follows: (qualified completes + over quota + terminates)/click-through).
| Completions | Pre-campaign Survey | Post-campaign Survey |
|---|---|---|
| Click-Through | 5192 | 6425 |
| - Partial Completes | 223 | 188 |
| - Terminates | 1482 | 2538 |
| - Over quota | 1168 | 1372 |
| Qualified Completes | 2319 | 2327 |
| Participation Rate | 96% | 97% |
The results of this survey are not statistically projectable to the target population because the sampling method used does not ensure that the sample represents the target population with a known margin of sampling error. Reported percentages are not generalizable to any group other than the sample studied, and therefore no formal statistical inferences can be drawn between the sample results and the broader target population it may be intended to reflect. The data have been weighted to reflect the demographic composition of Canadian adults aged 18 years and over.
The table below compares the unweighted survey samples to the 2021 Census results by region, age, and gender. Overall, the sample is highly representative of the national adult population.
| Pre-campaign Survey | Post-campaign Survey | |||
|---|---|---|---|---|
| Unweighted Percentage | Census 2021 Proportions | Unweighted Percentage | Census 2021 Proportions | |
| Region* | ||||
| British Columbia/Yukon | 13% | 14% | 13% | 14% |
| Alberta/Northwest Territories | 12% | 11% | 11% | 11% |
| Prairies (MB/SK) / Nunavut | 6% | 6% | 7% | 6% |
| Ontario | 41% | 39% | 41% | 39% |
| Quebec | 20% | 23% | 21% | 23% |
| Atlantic | 7% | 7% | 7% | 7% |
| Gender* | ||||
| Male | 55% | 49% | 55% | 49% |
| Female | 45% | 51% | 44% | 51% |
| Diverse / Prefer not to Answer | <1% | <1% | <1% | <1% |
| Age* | ||||
| 18-34 | 28% | 27% | 28% | 27% |
| 35-54 | 36% | 32% | 36% | 32% |
| 55+ | 36% | 41% | 36% | 41% |
| * Denotes variables included in the weighting scheme. | ||||
Differences among the variables used in the weighting scheme are minimal. The distribution in the sample is consistent, with only slight differences observed between the unweighted percentages and the 2021 Census data.
Thank you for taking a few minutes to complete this survey on current issues that matter to Canadians. Si vous préférez répondre à ce sondage en français, veuillez cliquer sur français [Switch to french version].
Your participation is voluntary and your responses will be kept entirely confidential. The survey takes about 5 minutes to complete.
START SURVEY
Click here if you wish to verify the authenticity of this survey. To view our privacy policy, click here.
If you require any technical assistance, please contact XXX.
a) Does anyone in your household work for any of the following organizations?
Select all that apply
If "none of these organizations" continue, otherwise thank and terminate.
b) What is your gender?
c) In what year were you born?
YYYY
If male at B and range 2002-1962 at C ask question E
Admissible range 1900-2004
If > 2004, thank and terminate
Ask D if question C is left blank
d) In which of the following age categories do you belong?
Select one only
If "less than 18 years old" or "blank", thank and terminate
If male at B and 20 to 59 at D ask question E
e) Are you currently or have you within the past six months worked at some point in construction trades, long-haul trucking, or heavy machinery operation?
If yes categorize as target audience
In which province or territory do you live?
Select one only
If no province or territory is selected, thank and terminate
Ask all respondents
Over the past three weeks, have you seen, read or heard any advertising from the Government of Canada?
Think about the most recent Government of Canada ad that comes to mind. What do you remember about this ad?
Ask all respondents
Over the past three weeks, have you seen, read or heard any Government of Canada advertising about opioids?
Where have you seen, read or heard this ad about opioids?
[Note: select from list all media used in the campaign. you may also include other media of your choice. headings are for guidance only and not to be used in the final version of the questionnaire]
Select all that apply
What do you remember about this ad?
Ask all respondents
Campaign specific attitudinal and behavioural questions may be added here
To what extent do you agree with the following statements…
1 – Strongly disagree
2
3
4
5 – Strongly agree
How would you describe your level of knowledge about opioids?
1 – Not at all knowledgeable
2
3
4
5 – Very knowledgeable
Which of the following categories best describes your current employment status? Are you…
Select one only
What is the highest level of formal education that you have completed?
Select one only
Are there any children under the age of 18 currently living in your household?
Which of the following categories best describes your total annual household income, including income from all household members, before taxes are deducted?
Select one only
Where were you born?
Ask if D5=born outside Canada
In what year did you first move to Canada?
YYYY
Admissible range: 1900-2022
What is the language you first learned at home as a child and still understand?
Select up to two
That concludes the survey. This survey was conducted on behalf of Health Canada. In the coming months the report will be available from Library and Archives Canada. We thank you very much for taking the time to answer this survey, it is greatly appreciated.
Thank you for taking a few minutes to complete this survey on current issues that matter to Canadians. Si vous préférez répondre à ce sondage en français, veuillez cliquer sur français [Switch to french version].
Your participation is voluntary and your responses will be kept entirely confidential. The survey takes about 7 minutes to complete.
START SURVEY
Click here if you wish to verify the authenticity of this survey. View our privacy policy.
If you require any technical assistance, please contact xx
f) Does anyone in your household work for any of the following organizations?
Select all that apply
If "none of these organizations" continue, otherwise thank and terminate.
g) What is your gender?
h) In what year were you born?
YYYY
If male at B and range 2002-1962 at C ask question E
Admissible range 1900-2023
If > 2004, thank and terminate
Ask D if question C is left blank
i) In which of the following age categories do you belong?
Select one only
If "less than 18 years old" or "blank", thank and terminate
If male at B and D is 20 to 59 ask question E
j) Are you currently or have you within the past six months worked at some point in construction trades, long-haul trucking, or heavy machinery operation?
If yes categorize as target audience
f) In which province or territory do you live?
Select one only
If no province or territory is selected, thank and terminate
Ask all respondents
Over the past three weeks, have you seen, read or heard any advertising from the Government of Canada?
Think about the most recent Government of Canada ad that comes to mind. What do you remember about this ad?
Ask all respondents
Over the past six months, have you seen, read or heard any Government of Canada advertising about opioids?
Where have you seen, read or heard this ad about opioids?
Select all that apply
What do you remember about this ad?
Ask all respondents
Campaign specific attitudinal and behavioural questions may be added here
To what extent do you agree with the following statements…
1 – Strongly disagree
2
3
4
5 – Strongly agree
How would you describe your level of knowledge about opioids?
1 – Not at all knowledgeable
2
3
4
5 – Very knowledgeable
Which term do you think is the clearest way to describe the current crisis of approximately 20 opioid-related deaths per day:
(select one only)
Here are some ads that have recently been broadcast on various media. Please make sure your sound is on and click next to see the ads.
[Click to go to the next page]
[Randomize order of ads]
A still representation of video where a man struggles to carry a large cinderblock on his shoulder while a woman looks at him. The text reads, "Addiction can be a heavy load to carry."
Description: An image used during the campaign displaying a man wearing a yellow hardhat that carries a cinderblock on his shoulder. The text reads “Addiction can be a heavy load to carry.”
An image used during the campaign displaying a tradesman looks off to the distance. The text reads "Addiction impacts men in trades more than others."
Over the past six months, have you seen any of these ads?
If no, skip to T1J
Where have you seen, read or heard these ads?
Select all that apply
What do you think is the main point these ads are trying to get across?
Please indicate your level of agreement with the following statements about these ads?
Randomize statements
| 1 Strongly Disagree |
2 | 3 | 4 | 5 Strongly Agree |
|
|---|---|---|---|---|---|
| These ads catch my attention | |||||
| These ads are relevant to me | |||||
| These ads are difficult to follow | |||||
| These ads do not favour one political party over another | |||||
| These ads talk about an important topic | |||||
| These ads provide new information | |||||
| These ads clearly convey that the Government of Canada is taking action on the opioid crisis | |||||
| These ads gave me an understanding that ending stigma related to opioid use can help save lives. | |||||
| These ads gave me an understanding that support is available to end opioid addiction. |
[Ask if T1H=yes to any ad]
Did you do any of the following as a result of seeing these ads?
Select all that apply
Which of the following categories best describes your current employment status? Are you…
Select one only
What is the highest level of formal education that you have completed?
Select one only
Are there any children under the age of 18 currently living in your household?
Which of the following categories best describes your total annual household income, including income from all household members, before taxes are deducted?
Select one only
Where were you born?
Ask if D5=born outside Canada
In what year did you first move to Canada?
YYYY
Admissible range: 1900-2023
What is (are) the language(s) you first learned at home as a child and still understand?
Select up to two
That concludes the survey. This survey was conducted on behalf of Health Canada In the coming months the report will be available from Library and Archives Canada. We thank you very much for taking the time to answer this survey, it is greatly appreciated.