Post-Opioid Campaign Evaluation - Methodological Report

Prepared for Health Canada
Supplier name: Ipsos Public Affairs
Contract Number: HT372-192411/001/CY
Contract value: $29,659.06
Award Date: May 27, 2019
Delivery Date: July 8, 2019

Registration number: POR 010-19

For more information on this report, please contact Health Canada at: hc.cpab.por-rop.dgcap.sc@canada.ca

Ce rapport est aussi disponible en français.

POLITICAL NEUTRALITY STATEMENT

I hereby certify as Senior Officer of Ipsos that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.

Mike Colledge
President
Ipsos Public Affairs

Post-Opioid Campaign Evaluation
Methodological Report

Prepared for Health Canada
Supplier name: Ipsos Public Affairs
July 2019

This public opinion research report discusses the methodology of a post-campaign ACET online survey conducted by Ipsos Public Affairs on behalf of Health Canada. The research study was conducted with 2,345 Canadians aged 18 and over, including 447 males aged 20 to 39, who were the target audience. The survey was conducted between June 4 and 30, 2019. This survey is an extension of baseline and post-campaign surveys conducted in 2018 and early 2019. The baseline survey was conducted between October 19 and November 16, 2018, the first post-campaign survey was conducted between December 11 and 21, 2018 and the second post-campaign survey, which evaluated a different ad aimed at the stigma around opioid use, was conducted between February 18 and March 4, 2019.

Cette publication est aussi disponible en français sous le titre : Évaluation après de la campagne sur les opioïdes

This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Health Canada. For more information on this report, please contact Health Canada at hc.cpab.por-rop.dgcap.sc@canada.ca or at:

Communications Branch
Health Canada
200 Eglantine Driveway
AL 1915C, Tunney's Pasture
Ottawa, ON K1A 0K9

Catalogue Number:
H14-330/2019E-PDF

International Standard Book Number (ISBN):
978-0-660-31874-5

Related publications (registration number: POR 010-19):
Catalogue Number H14-330/2019F-PDF (Final Report, French)
978-0-660-31875-2

© Her Majesty the Queen in Right of Canada, as represented by the Minister of Health, 2019

TABLE OF CONTENTS

Executive Summary

1. Background

1.1 Summary Statement

The research consisted of a quantitative online survey with a national sample of Canadians aged 18 and over and oversamples of males aged 20-39. In total, n=2,345 (including n=447 males aged 20-39) online surveys were conducted.

The survey was conducted in English and French between June 4 and 30, 2019.

For this survey, RIM weighting1 was employed to balance demographics by three variables (gender, age and region) to ensure that the survey sample's composition reflects that of the adult population according to 2016 Census data and to provide results intended to approximate the sample universe.

Ipsos programmed, hosted and provided sample management services only, while Health Canada provided the online surveys. Ipsos was responsible for data collection and data storage in Canada, data processing, and data weighting. The total contract value of this research was $ 29,659.06 including HST.

2. Purpose of the Research

2.1 Research Objectives

Health Canada has launched a multi-year advertising campaign on the subject of opioids to address stigma as a barrier to treatment, to raise awareness and understanding of the risks associated with problematic use of opioids (legal and illegal) and to help reduce problematic use, dependence and overdoses.

The purpose of this research was to assess the effectiveness of the advertising campaign. Three ads were tested, each having a different focus:

  1. Awareness of the opioid crisis in Canada and what the Government of Canada is doing
  2. Addressing stigma related to opioid use
  3. Explaining the Good Samaritan Law

Specific research objectives included:

Appendices

Participant Recruitment

Sample Source

The Ipsos i-Say panel is the primary source for recruiting respondents. Where necessary, a supplemental sample from third party vendor panels is also used as a source. The Ipsos i-Say online panel is a diversely-sourced and actively maintained panel of over 200,000 Canadian adults. Utilizing this panel allows us to confidently build online samples based on known population characteristics like the ones available through Census data. Invitations to participate in each survey are adjusted to reflect the most recent Current Population Data provided by Statistics Canada.

Online panel participants are recruited and screened using rigorous double and triple opt-in checks2 to ensure that respondents want to receive contact such as invitations to participate in research surveys. Ipsos i-Say panel Terms and Conditions of Membership details the respondents' rights to, at any time, for any reason ask to be removed from the panel and stop receiving emails from Ipsos and its clients.

Panel recruitment consists of the following steps:

It is important to note that any panel – no matter how well recruited – does not enable random selection in the sense that not all members of the public are apt to join a panel. This applies to panels recruited through online approaches as well as panels recruited via traditional telephone Random Digit Dialing.

Rigorous panel maintenance procedures mean only those participants who actively participate in online surveys, at least once every six months, are included. Participants can choose whether to enter a survey or not. While participating in a survey, the respondent can simply exit the survey by closing the survey Internet browser window at any time without submitting their responses if they no longer want to participate.

Incentives

To ensure a high participation rate from our panelists, the i-Say panel provides a number of innovative incentive programs. Ipsos' i-Say online panel participants are incentivized through a points program whereby respondents collect points that can be redeemed for rewards of their choice from a collection of products from online retailers such as gift cards and prizes. Each incentive is tailored to the specific requirements of each survey, depending on the length of the survey, the subject matter of the study, and the time required to complete a minimum number of interviews.

Sample Details

Details Related to Target Population

A sampling frame was devised to outline a quota structure representative of Canadian adults 18 years and older, including oversamples of males aged 20-39 years old. All Canadian adults aged 18 years and older were eligible to participate.

Description of Sample Frame

The Ipsos i-Say online panel consists of over 200,000 Canadian adults recruited through diverse sources and is constantly refreshed. Background data gathered when respondents join the panel allows us to finely target respondents on the basis of key criteria such as region (national samples are typically divided into over 40 submarkets nationwide ensuring accurate distributions within provinces), age, and gender. The results of quota based sampling against these criteria allow us to construct samples that are reflective of Canadian population characteristics. In this instance, sampling quotas were set according to the respondents' region, gender, and age to match a target profile based on Census 2016 data. This serves to underpin the representativeness of the survey results.

All sample surveys and polls may be subject to sources of error, including, but not limited to coverage error and measurement error. Traditional margins of error don't apply to online panel-based samples of general public audiences as the traditional margin of error calculation assumes that the sample measured is a probability sample. As there is no way to measure the probability that any given member of the public may join or be a part of an online panel, these samples are inherently non-probabilistic.

Non-Response Analysis: There exists within the current sample the possibility of non-response bias. In particular, this survey would not include members of the population who do not have access to the Internet or who are not capable of responding to a survey in either English or French.

Extrapolation: The table below compares the unweighted sample to the 2016 Census results by region, age, and gender, as well as the unweighted geographical and demographic distribution of the sample.

  Unweighted
Sample Size		 Sample Proportions 2016 Census Proportions
Canada 2,345 100% -
Region
British Columbia 298 13% 13.57%
Alberta 254 11% 11.21%
Prairies (SK/MB) 153 7% 6.51%
Ontario 839 36% 38.41%
Quebec 685 29% 23.47%
Atlantic Canada 116 5% 6.83%
Gender
Male 1263 54% 48.57%
Female 1082 46% 51.43%
Age
18-24 103 4% 10.49%
25-34 429 18% 16.40%
35-44 377 16% 16.14%
45-54 435 19% 17.91%
55-64 360 15% 17.47%
65+ 641 27% 21.14%
Males Aged 20-39 447 19% 16.70%

Case Dispositions

The following table provides the response dispositions and response rate calculation, as per the former Marketing Research and Intelligence Association's (MRIA) empirical method of calculating response rates for online surveys.


Calculation for Data Collection3		 Post-Campaign Survey
Total Email Invitations Issued 28,539
Unresolved (U) (no response) 25,433
In-scope - non-responding (IS) - Qualified respondent break-off (incomplete) 181
In-scope - Responding units (R) 2,925
Over quota 394
Other disqualified 186
Completed Interviews 2,345
Response Rate = R/(U+IS+R) 10%

Sampling is carried out via a proprietary sampling application that allows us to construct complex samples based on the target and screening requirements. To develop the survey sample and minimize non-response, we use a large and diversely sourced panel. In addition, we have the capability of broadening the sample frame further through the inclusion of other actively managed panel resources as needed to ensure the sample requirements are met among the hardest-to-reach audiences (such as younger males).

Survey Instrument

ADVERTISING CAMPAIGN EVALUATION TOOL

POST-CAMPAIGN SURVEY

To be conducted after the ads have been run in the media.

INTRODUCTION

Thank you for taking the time to complete this survey dealing with current issues of interest to Canadians. Si vous préférez répondre au sondage en français, veuillez cliquer sur français [SWITCH TO FRENCH VERSION].

Your participation is voluntary and your responses will be kept entirely confidential. The survey takes about 5 minutes to complete.

START SURVEY

Click here if you wish to verify the authenticity of this survey. To view our privacy policy, click here.

a) Does anyone in your household work for any of the following organizations?

SELECT ALL THAT APPLY

IF "NONE OF THESE ORGANIZATIONS" CONTINUE, OTHERWISE THANK AND TERMINATE.

b) Are you…

c) In what year were you born?

YYYY

ADMISSIBLE RANGE 1900-2001

IF > 2001, THANK AND TERMINATE

ASK D IF QUESTION C IS LEFT BLANK

d) In which of the following age categories do you belong?

SELECT ONE ONLY

IF "LESS THAN 18 YEARS OLD" OR "BLANK", THANK AND TERMINATE

In which province or territory do you live?

SELECT ONE ONLY

IF NO PROVINCE OR TERRITORY IS SELECTED, THANK AND TERMINATE

CORE QUESTIONS

ASK ALL RESPONDENTS

Q1:
 Over the past six weeks, have you seen, read or heard any advertising from the Government of Canada?

Q2:
Think about the most recent ad from the Government of Canada that comes to mind. Where have you seen, read or heard this ad?

SELECT ALL THAT APPLY

Q3:
What do you remember about this ad?

Q4:
 How did you know that it was an ad from the Government of Canada?

CAMPAIGN SPECIFIC QUESTIONS

ASK ALL RESPONDENTS

T1A:
 Over the past six weeks, have you seen, read or heard any Government of Canada advertising about opioids, stigma or what to do if you witness an opioid overdose?

T1B:
Where have you seen, read or heard this Government of Canada ad about opioids, stigma or what to do if you witness an opioid overdose?

SELECT ALL THAT APPLY

T1C:
What do you remember about this ad?

ASK ALL RESPONDENTS

T1D:
How would you describe your level of knowledge about opioids?

T1E: To what extent do you agree or disagree with the following statements…

Here are ads that have recently been broadcast on various media. Please make sure your sound is on and click next to see the ads.

[CLICK TO GO TO THE NEXT PAGE]

[RANDOMIZE ORDER OF ADS]

T1F_Ad1_Awareness:
Over the past six weeks, have you seen this ad?

T1F_Ad2_GoodSam:
Over the past six weeks, have you seen this ad?

T1F_Ad3_Stigma:
Over the past six weeks, have you seen this ad?

IF NO TO ALL 3 ADS, SKIP TO T1H

T1G:
Where have you seen these ads?

SELECT ALL THAT APPLY

T1H:
What do you think is the main point these ads are trying to get across?

T1I:
Please indicate your level of agreement with the following statements about these ads?

RANDOMIZE STATEMENTS

  Strongly Disagree       Strongly Agree
These ads caught my attention 1 2 3 4 5
These ads are relevant to me 1 2 3 4 5
These ads are difficult to follow 1 2 3 4 5
These ads do not favour one political party over another 1 2 3 4 5
These ads talk about an important topic 1 2 3 4 5
These ads provide new information 1 2 3 4 5
These ads clearly convey that the Government of Canada is taking action on the opioid crisis. 1 2 3 4 5
These ads gave me an understanding that ending stigma related to opioid use can help save lives. 1 2 3 4 5

[ASK IF T1F=YES TO ANY AD]

T1J:
Did you do any of the following as a result of seeing these ads?

SELECT ALL THAT APPLY

DEMOGRAPHIC QUESTIONS

D1:
 Which of the following categories best describes your current employment status? Are you…

SELECT ONE ONLY

D2:
 What is the highest level of formal education that you have completed?

SELECT ONE ONLY

D3:
Are there any children under the age of 18 currently living in your household?

D4:
Which of the following categories best describes your total annual household income, including income from all household members, before taxes are deducted?

SELECT ONE ONLY

D5:
Where were you born?

ASK IF D5=BORN OUTSIDE CANADA

D6:
In what year did you first move to Canada?

YYYY

ADMISSIBLE RANGE: 1900-2019

D7:
What is the language you first learned at home as a child and still understand?

SELECT UP TO TWO

That concludes the survey. This survey was conducted on behalf of Health Canada. In the coming months the report will be available from Library and Archives Canada. We thank you very much for taking the time to answer this survey, it is greatly appreciated.