Mental Health Advertising Campaign 2022 Post Advertising Campaign Evaluation Tool (ACET)

Methodological Report

Prepared for Health Canada

Supplier name: Ipsos Public Affairs
Contract Number: HT372-224355
Contract value: $51,453.87 (including applicable taxes)
Award Date: July 13^th, 2022
Delivery Date: March 27^th, 2023
Registration number: POR 014-22

For more information on this report, please contact Health Canada at: hc.cpab.por-rop.dgcap.sc@canada.ca

Ce rapport est aussi disponible en français

Political neutrality statement

I hereby certify as Senior Officer of Ipsos that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.

Mike Colledge
President
Ipsos Public Affairs

Mental Health Advertising Campaign 2022 – Post Advertising Campaign Evaluation Tool (ACET)

Methodological Report

Prepared for Health Canada
Supplier name: Ipsos Public Affairs
March 2023

This public opinion research report presents the methodology of the Mental Health Advertising Campaign 2022 – Post Advertising Campaign Evaluation Tool (ACET) online survey conducted by Ipsos Public Affairs on behalf of Health Canada and the Public Health Agency of Canada. The survey was conducted with a sample of n=2243 Canadians ages 18+ between March 6^th and March 21^st, 2023.

Cette publication est aussi disponible en français sous le titre : Campagne publicitaire sur la santé mentale 2022 - Outil d'évaluation post-campagne publicitaire.

This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Health Canada. For more information on this report, please contact Health Canada at hc.cpab.por-rop.dgcap.sc@canada.ca or at:

Communications Branch
Health Canada
200 Eglantine Driveway
AL 1915C, Tunney's Pasture
Ottawa, ON K1A 0K9

Catalogue Number:
H14-437/2023E-PDF

International Standard Book Number (ISBN):
978-0-660-41522-2

Related publications
(registration number: POR 014-22)

Catalogue Number:
H14-437/2023F-PDF (Final Report, French)

ISBN:
978-0-660-41523-9

^©His Majesty the King in Right of Canada, as represented by the Minister of Health, 2023

Contents

• Executive Summary
  • 1. Background
  • 2. Research Objectives
  • 3. Methodology
    • 3.1 Online Sample
    • 3.2 Participant Recruitment
  • 4. Appendix – Survey Questionnaire
    • 4.1 English Questionnaire

Executive Summary

1. Background

The COVID-19 pandemic has had significant adverse impacts on the health and well-being of Canadians. Many Canadians have reported increased feelings of stress, anxiety, depression and loneliness, and national surveillance data has shown a rise in drug toxicity deaths and other substance-related harms. To this end, it is critical that Canadians continue to have access to effective resources and tools to support their mental health and wellbeing and are able to obtain reliable information about mental health and substance use. As reflected in Budget 2022, the Government of Canada (GC) is committed to improving the delivery of mental health services and supports to ensure Canadians are able to receive high quality, culturally appropriate care. This includes funding for Health Canada to continue to improve mental health and access to substance use services:

• $140 million over two years, for the Wellness Together Canada portal to continue to provide Canadians with tools and services to support their mental health and well-being.
• $100 million over three years for the Substance Use and Addictions Program to support harm reduction, treatment, and prevention at the community level. By providing appropriate information and resources, the GC can help empower Canadians to take care of their own mental health, and to support others, thereby helping contribute to the well-being of communities.

In a continuous effort to support the mental health of Canadians, Health Canada (HC) and the Public Health Agency of Canada (PHAC) conducted the 2022-23 national COVID-19 Mental Health campaign. The campaign aims to drive audiences to Canad.ca/mental health to motivate interest in and engagement with free resources, tools, and professional support services – including the Wellness Together Canada portal.

The Government of Canada's Policy on Communications and Federal Identity requires the evaluation of advertising campaigns exceeding $1 million in media buy using the Advertising Campaign Evaluation Tool (ACET). This study was conducted to meet this requirement.

The Mental Health advertising campaign ran from December 19, 2022, until March 31, 2023, and targeted gen pop, with a skew to six at-risk and vulnerable groups of Canadians.

2. Research Objectives

HC and PHAC are looking to assess the impact of its COVID-19 Mental Health Advertising campaign in raising awareness of the tools and professional support services available through the Canada.ca/mental health. This would be done with a post-campaign survey to see the effect of the communications intervention.

The objectives of the research are as follows:

• Determine if people have seen the advertisements associated with the campaign
• Determine where the ads have been seen
• Measure recall of specific elements of the campaign
• Identify attitudinal and behavioural changes as a result of the advertising campaign

The results of this research will allow HC and PHAC to evaluate the performance of this campaign.

Target Audience

The primary target audience is Canadians 18 years and older. The target audience for the campaign includes six (6) at-risk groups of Canadians:

1. Youth and young adults (a18-34)
2. Parents with school aged children (a5-17)
3. Seniors (a65+)
4. Indigenous Peoples
5. Ethnic and Racialized Populations
6. LGBTQ2+ communitywho experience a disproportionally higher rate of mental health and substance use issues. According to recent reports, 42 percent of Wellness Together Canada users self-identified as LGBTQ2S+.

3. Methodology

3.1 Online Sample

Respondents for the survey samples were drawn from a trusted Ipsos partner panel vendor, Canadian Viewpoint Inc. The survey was conducted with a sample of n=2,243 Canadians ages 18+. Interviews were conducted in the language of their choice, either English or French.

A pre-test was launched on March 6, 2023, with 48 completes (25 English / 23 French). An open-ended question was asked at the end of the survey where any problems, questions, or unclear questions could be brought to our attention; no issues were flagged. The survey was fully launched and ran between March 7 and March 21, 2023.

Quotas were set to ensure representation by region, age, and gender, according to the latest Census information.

The sampling methodology utilized email invitations and router technology to invite participants. Each participant received a unique URL link. This link could only be used once, with respondents being allowed to pause during completion and return to complete. On average, the online interviews took 10 minutes to complete.

For this study quotas were set to ensure representation by region, age, and gender, according to the latest Census information. Sampling targets were obtained through natural fallout. An oversample of Indigenous (including 140) and 2SLGBTQI+ (including 115) was administered. The unweighted counts from Indigenous and 2SLGBTQI+ are shown below.

3.2 Participant Recruitment

Sample Source

Ipsos partnered with sub-contractor Canadian Viewpoint Inc. Canadian Viewpoint has one of the largest consumer panels with ~300,000 active panelists. Like Ipsos, Canadian Viewpoint uses mixed-medium strategies (phone/online/in-person) to build its panel to allow more representative samples. Canadian Viewpoint's online consumer panel is also recruited using post screeners during telephone surveys, Facebook, online vetted lists and in-person studies. Incentives were not used for recruitment purposes to ensure quality, but respondents were incentivized for completing the survey, which is directly proportionate to the amount of time taken to complete the survey and with comparable incentives offered by other online panel sources. We informed Health Canada and PHAC before we engaged with other approved panel vendors.

The comprehensive background profiling data were gathered when respondents joined the panel through screener questions, which allowed for the targeting of respondents based on key criteria, such as region, age, gender, education and income level, children and age of children in the household, ethnic background, indigenous identity, and other characteristics.

The online survey was conducted using Computer Assisted Web Interviewing (CAWI). Every panelist opted-in or chose to participate in research surveys. Panelists that meet the criteria we were looking for were randomly selected and sent an email invitation to complete the survey. Participants were offered a choice to complete the survey in either English or French.

The survey platform was Accessibility for Ontarians with Disabilities Act (AODA) compliant, according to Web Content Accessibility Guidelines (WCAG2.0AA).

Incentives and Quality Control Measures

Canadian Viewpoint Inc (CVI) use a multi-faceted approach to quality, integrating controls and best practices into every phase of the process. Panel members come from diverse fully certified sources. A dedicated team monitors performance by source using feedback loops and continuous reporting to prevent fraud. Our dedicated team monitors the panels for potentially fraudulent behavior, and blocks these people from ever entering a survey via safeguards that includes:

• A double opt-in authentication process
• Cash incentives, in the form of cheques to ensure confirmation of our panelist identity
• Respondents provide us with legitimate information about their identity and address and allow them to receive their incentive cheque
• Immediate examination of suspicious email addresses, suspicious behavior, or unrealistic completion
• Fast and permanent removal of offenders
• CVI works with our clients to set up customized data quality checks for the surveys that we program.
• We limit invitations to 3 x a week. The average respondent receives 1.38 surveys a month from Ipsos and completes 5 - 6 surveys per year
• CVI keeps the participation history, join date and source of the respondent on file. Project analysis of such data is available to our clients upon request
• Proprietary Verification is a suite of quality control procedures, including digital fingerprinting, address verification against USPS databases (standard in the US and Canada,) and third-party verification upon request

Sample Weighting

The table below shows the unweighted and weighted distributions of the online sample. Weighting was applied to the sample to ensure that the final data reflects the general adult population by region, age, and gender according to the 2021 Census. A Random Iterative Method (RIM) technique was applied for weighing.

The figures presented in the table above show minimal differences between the unweighted and weighted samples. As a result of the strong representativeness of the unweighted data the largest weight factor that was applied for any respondent is 1.06, which is well within acceptable ranges for a survey of the general population.

Email Statistics

For this survey, a sample router was used. Therefore, a response rate cannot be calculated. However, the participation rate for the post-wave survey was 98%. The participation rate is calculated as follows: (qualified completes + over quota + terminates)/click-through).

Non-Response Analysis

The results of this survey are not statistically projectable to the target population because the sampling method used does not ensure that the sample represents the target population with a known margin of sampling error. Reported percentages are not generalizable to any group other than the sample studied, and therefore no formal statistical inferences can be drawn between the sample results and the broader target population it may be intended to reflect. The data have been weighted to reflect the demographic composition of Canadian adults aged 18 years and over.

The table below compares the unweighted survey samples to the 2021 Census results by region, age, and gender. Overall, the sample is highly representative of the national adult population.

Differences among the variables used in the weighting scheme are minimal. The distribution in the sample is consistent, with only slight differences observed between the unweighted percentages and the 2021 Census data.

4. Appendix – Survey Questionnaire

4.1 English Questionnaire

Thank you for taking the time to complete this survey dealing with current issues of interest to Canadians. Do you prefer to continue in English or French? / Préférez-vous continuer en français ou en anglais ?

• English / Anglais
• French / Français

Your participation is voluntary, and your responses will be kept entirely confidential. The survey takes about 10 minutes to complete.

Start survey

To view our privacy policy, click here.

Privacy policy :

English: Privacy & Data Protection | Ipsos

French: Confidentialité et protection des données personnelles | Ipsos

If you require any technical assistance, please contact daniel.kunasingam@ipsos.com

a) Does anyone in your household work for any of the following organizations?

Select all that apply

• A marketing research firm
• A magazine or newspaper
• An advertising agency or graphic design firm
• A political party
• A radio or television station
• A public relations company
• The federal or provincial government
• None of these organizations [Exclusive]

If "none of these organizations" continue, otherwise thank and terminate.

b) Are you…?

• Male gender
• Female gender
• Gender diverse

c) In what year were you born?

YYYY

Admissible range 1900-2004

If >2004, thank and terminate

Ask d if question c is left blank

d) In which of the following age categories do you belong?

Select one only

• Less than 18 years old
• 18 to 24
• 25 to 34
• 35 to 44
• 45 to 54
• 55 to 64
• 65 or older

If "less than 18 years old" or "blank", thank and terminate

E) In which province or territory do you live?

Select one only

• Alberta
• British Columbia
• Manitoba
• New Brunswick
• Newfoundland and Labrador
• Northwest Territories
• Nova Scotia
• Nunavut
• Ontario
• Prince Edward Island
• Quebec
• Saskatchewan
• Yukon
• I do not live in Canada [thank and terminate]

If no province or territory is selected, thank and terminate

e) Are you the parent or legal guardian of a child aged 5 years old or younger?

1. Yes
2. No
3. Prefer not to answer

[If yes at 'e' ask 'f']

How old is/are the child/children? Select all that apply

• Under 5
• 5 to 11
• 12 to 17
• 99- Prefer not to answer [Exclusive]

Screener questions to identify if indigenous populations or ethnic/racialized populations

g. Do you identify as any of the following? Select all that apply

• White
• Indigenous person (First Nations, Inuit or Métis)
• South Asian
• Chinese
• Filipino
• Southeast Asian
• West Asian
• Korean
• Japanese
• Black
• Latin American
• Arab
• Other
• Prefer not to answer

h. And do you identify as any of the following? Select all that apply

• A member of the 2SLGBTQI+ community
• A person with a disability
• None of the above [Exclusive]

Core questions

Ask all respondents

Q1: Over the past three weeks, have you seen, read or heard any advertising from the Government of Canada?

• Yes
• No => Go to T1A

Q3: Think about the most recent Government of Canada ad that comes to mind. What do you remember about this ad?

Campaign-specific questions

Ask all respondents

T1A: Over the past two months, have you seen, read or heard any Government of Canada advertising about getting your flu vaccine?

• Yes
• No => GO TO T1A

T1B: Where have you seen, read or heard this ad about mental health and substance use supports? Select all that apply.

Select all that apply

T1C: What do you remember about this ad?

Ask all respondents

Campaign specific attitudinal and behavioural questions may be added here

Q1A. Generally speaking, how would you rate your overall mental health?

• Excellent (Do not show – code 5)
• Somewhat good (Do not show – code 4)
• Average (Do not show – code 3)
• Somewhat poor (Do not show – code 2)
• Poor (Do not show – code 1)
• Not sure (Do not show – code 9)

Q2. Has your mental health gotten worse, better or stayed the same in the last year?

• It has gotten worse
• It has gotten better
• It has stayed the same

Q3A. For each of the items listed below, please indicate whether you have experienced any changes in the past year by selecting increased, decreased, stayed the same/no change or not applicable.

Progressive grid -set up or otherwise recommended [Randomize]

• Stress
• Anxiety
• Fear
• Sadness
• Loneliness
• Financial worry
• Suicidal thoughts
• Self harm

• Increased
• Decreased
• Stayed the same/No change
• Not applicable
• Don't know / Prefer not to answer

Q4 Please indicate if the situations listed below changed for you in the last year by selecting increased, decreased or the same/no change.

Progressive grid -set up or otherwise recommended [Randomize]

• My consumption of alcohol like beer, wine or spirits.
• My consumption of cannabis.
• My consumption of tobacco/vaping.
• My consumption of illegal/illicit drugs.

• Increased
• Decreased
• Same/No change
• I don't use this substance
• Don't know / Prefer not to answer

Q5 Have you received help for mental health in the last year? (select one only)

• Yes
• No
• Don't know

[If "yes" in Q5]

Q6 Are you currently receiving help? (select one only)

• Yes
• No

Q7. What type of support have you received?

[If "yes" in Q5]

Select all that apply.

• Online/virtual support from support workers, social workers, psychologists or other professionals
• In person support from support workers, social workers, psychologists or other professionals
• Phone (including text) support
• Resources and tips for taking care of yourself
• Your primary health care provider
• Resources and/or services through your local government
• Resources and/or services through your provincial/territorial government
• Resources and/or services through the Government of Canada
• Support from local, or a non-for profit organization
• Through digital apps or platforms, like Wellness Together Canada
• Other, please specify:

[If "No" in Q6 AND rate average/poor mental health: Q1 = 1, 2 or 3ASK Q8]

Q8 Why are you not currently receiving help? (Select all that apply)

• My mental health is fine (Do not show code 1)
• I don't need help (Do not show code 2)
• I don't have the time (Do not show code 3)
• I don't know where to go (Do not show code 4)
• I can't access help / help not readily available
• It is too expensive (Do not show code (5)
• Other, please specify: (Do not show code 77)
• Don't know (99) [Exclusive]

Q9 How familiar would you say you are with the different mental health services available?

• Very familiar
• Somewhat familiar
• Not very familiar
• Not at all familiar

Q10 Did you know the Government of Canada offers free mental health and substance use help? (select one only)

• Yes, mental health only
• Yes, substance use only
• Yes, both
• No

Ad recall questions [ask in post-campaign survey only]

T1H: Here are some ads that have recently been broadcast on various media. Click next to watch.

[Video]

[Banner to be added]

[Click to go to the next page]

Over the past six weeks, have you seen these ads?

• Yes
• No => Go to T1J

T1I: Where have you seen, these ads? Select all that apply.

Select all that apply

T1J: What do you think is the main point these ads are trying to get across?

T1K: Please indicate your level of agreement with the following statements about these ads?

Randomize statements

[Ask all]

T1L. Did you [If NO in T1H: Will you] do any of the following as a result of seeing this ad? Please select all that apply [Randomize]

• Visited[Visit] Canada.ca/covid‐mental‐health
• Discussed[Discuss] my challenges with mental health with a healthcare professional/doctor, friend or family member
• Shared[Share] the ad with someone you know
• Shared [Share] the website with someone you know
• Looked [Look]for information for free mental health and substance use support and resources online
• Other, specify
• I didn't[won't] do anything as a result of seeing the ad

Demographic questions

D1: Which of the following categories best describes your current employment status? Are you…

SELECT ONE ONLY

• Working full-time (30 or more hours per week)
• Working part-time (less than 30 hours per week)
• Self-employed
• Unemployed, but looking for work
• A student attending school full-time
• Retired
• Not in the workforce (Full-time homemaker or unemployed but not looking for work)
• Other employment status

D2: What is the highest level of formal education that you have completed?

Select one only

• Less than a High School diploma or equivalent
• High school diploma or equivalent
• Registered Apprenticeship or other trades certificate or diploma
• College, CEGEP or other non-university certificate or diploma
• University certificate or diploma below bachelor's level
• Bachelor's degree
• Postgraduate degree above bachelor's level

D4: Which of the following categories best describes your total annual household income, including income from all household members, before taxes are deducted?

Select one only

• Under $20,000
• Between $20,000 and $40,000
• Between $40,000 and $60,000
• Between $60,000 and $80,000
• Between $80,000 and $100,000
• Between $100,000 and $150,000
• $150,000 and above
• Prefer not to say

D5: Where were you born?

• Born in Canada
• Born outside Canada
• Specify the country:

Ask if d5=born outside canada

D6: In what year did you first move to Canada?

YYYY

Admissible range: 1900-2021

D7: What is the language you first learned at home as a child and still understand?

Select up to two

• English
• French
• Other language, specify ___________________

That concludes the survey. This survey was conducted on behalf of Health Canada. In the coming months the report will be available from Library and Archives Canada. We thank you very much for taking the time to answer this survey, it is greatly appreciated.

If you or someone you know is having challenges with mental health or substance use, please visit Canada.ca/mental-health for free support.

Mental health support: Get help