2022-2023 CRA Benefits and Credits Campaign

Advertising Campaign Evaluation Tool (ACET) –

Executive Summary

Prepared for Canadian Revenue Agency (CRA)

Supplier name: Ipsos

Contract Number: CW2299077

Contract value: $30,284.00

Award Date: March 30th, 2023

Delivery Date: June 30th, 2023

Registration number: POR 150-22

For more information on this report, please contact Canadian Revenue Agency at:

cra-arc.media@cra-arc.gc.ca

Ce rapport est aussi disponible en français

2022-2023 CRA Benefits and Credits Campaign

Advertising Campaign Evaluation Tool (ACET) – Executive Summary

Prepared for Canadian Revenue Agency (CRA)

Supplier name: Ipsos

June 2023

This public opinion research report presents the methodology of the 2022-2023 CRA Benefits and Credits Campaign - ACET online survey conducted by Ipsos on behalf of the Canadian Revenue Agency (CRA).

The online survey was conducted with n=2,010 respondents between May 19th and 29th, 2023.

Cette publication est aussi disponible en français sous le titre : Campagne publicitaire sur les prestations et les crédits de l’Agence de 2022-2023 – Outil d’évaluation des campagnes publicitaires (OECP) – Sommaire exécutif.

This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Canada Revenue Agency. For more information on this report, please contact Canada Revenue Agency at: cra-arc.media@cra-arc.gc.ca.

Public Affairs Branch

555 MacKenzie Ave.

Ottawa, Ontario

K1A 0L5 Canada

Catalogue Number: Rv4-137/2023E-PDF

International Standard Book Number (ISBN): 978-0-660-68738-4

Related publications (registration number: POR 150-22):

Catalogue Number: Rv4-137/2023F-PDF (Methodological Report, French)

ISBN: 978-0-660-68739-1

© His Majesty the King in Right of Canada, as represented by the Minister of the Canada Revenue Agency, 2023

POLITICAL NEUTRALITY STATEMENT

I hereby certify as Senior Officer of Ipsos that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.

Mike Colledge

President

Ipsos Public Affairs

1. Background

The Canada Revenue Agency (CRA) administers taxes, benefits, and related programs for governments across Canada. These services and programs help residents of Canada comply with tax obligations and receive benefits, which contributes to social-economic well-being. The CRA is undertaking significant efforts to increase awareness and access to CRA benefits and services, participation in the tax system, and understanding of roles and responsibilities in the tax and benefit system.

Benefits and credits provide additional income or tax relief to individuals to help make life more affordable since many depend on benefit payments and tax credits for a significant part of their household income. For modest-income households, benefits can increase income by as much as 50 percent.

However, certain segments of the vulnerable population such as modest-income households, Indigenous Peoples, those new to Canada, and youth aged 18-24 face a variety of barriers when doing their taxes and receiving their benefits and credits. These barriers include a lack of general awareness of benefits and credits, lack of support to complete their taxes, living in remote locations, language barriers, low financial literacy and/or mental or physical health issues. Given the range of backgrounds and circumstances, it is important for the Government of Canada (GC) to make proactive and targeted efforts to reach vulnerable populations with important tax-related information. Moreover, greater awareness is needed so these populations know that doing their taxes every year is necessary to be eligible for these benefits and credits. Increasing awareness of benefits and credits will help to contribute to the government priority of poverty reduction, health, and well-being, as found in the GC’s Gender Results Framework.

The Benefits and Credits advertising Campaign

This Benefits and Credits advertising campaign is part of a sustained effort by the CRA to raise awareness amongst vulnerable populations[1] of the benefits and credits that they are entitled to when they do their taxes and the availability of free tax help. The campaign featured the Canada child benefit (CCB), Canada workers benefit (CWB), GST/HST credit, the disability tax credit (DTC), and the Community Volunteer Income Tax Program’s (CVITP) free tax clinics. The campaign also promoted tax measures related to housing, including the Tax-Free First Home Savings Account, Multigenerational Home Renovation Tax Credit, Home Buyers’ Plan, GST/HST new housing rebate, home buyers’ amount and First-Time Home Buyer Incentive.

The advertising campaign aims to:

• Promote/raise awareness about benefits and credits
• Inform people that getting benefits and credits is dependent on filing a tax return
• Inform people that continuing to receive benefits and credits requires filing every year
• Help individuals who need assistance doing their taxes by promoting the CVITP’s free tax clinics to individuals with modest incomes and simple tax situations
• Promote/raise awareness of tax measures related to housing to support those buying or renovating a home

Media Tactics

• Television (mainstream and Indigenous channels) and Connected TV
• Social media:
  • Facebook
  • Instagram
  • YouTube
• Search Engine Marketing (SEM)
• Digital Audio
• Websites
• Mobile apps
• Web banners
• Indigenous Radio
• Indigenous Print for official language compliancy

An Advertising Campaign Evaluation Tool (ACET) is required for all Government of Canada (GC) advertising campaigns with a media buy over $1 million. The ACET is used to consistently evaluate these campaigns and comply with key requirements in the GC Communications Policy.

Ipsos was contracted to conduct the data collection and tabulation, and to provide a methodology report. Ipsos programmed, hosted, and provided sample management services, while the CRA provided the online questionnaire. Ipsos was responsible for data collection and data storage in Canada, data processing, and data weighting. The data collection from Canadians was handled in accordance with government-wide Public Opinion Research procedures.

The research consisted of an online post-campaign survey conducted in English and French, between May 19 and 29th, 2023.

The total contract value of this research was $30,284.00, including HST.

2. Objectives

Advertising campaign evaluations assist in determining the effectiveness of the selected advertising campaigns by measuring (and comparing) awareness of the subject matter with the audiences. The results will be used to evaluate the effectiveness of the Benefits and Credits advertising campaign with the target audiences. The results will also help assess whether the ads were compelling / caught the audiences' attention, and whether the campaign raised awareness of the benefits and credits being promoted. Finally, the evaluation will assist the CRA in improving or modifying future ad campaigns.

For this research, CRA was seeking to evaluate the campaign which ended in May 2023. This post-campaign survey used the standardized ACET Post-Campaign online survey to assess:

• The unaided and aided awareness of the CRA advertisements and general GC advertising
• The message recall
• Whether the advertisements were:
  • Attention catching
  • Relevant
  • Difficult to follow
  • Favoring one political party over another
  • About an important topic
  • Provide new information
• Aided and unaided awareness of the subject matter
• Intention to act

Target Audience

The target audiences of the research were as follows:

Primary Target Audience:

• Vulnerable populations including:
  • Indigenous peoples, First Nations populations on- and off- reserve including the territories
  • Modest-income individuals, with household income <$40K
  • Newcomers to Canada (resident of Canada for less than 5 years)
  • People with disabilities[2]
• Youth aged 18-24
• Adults aged 65+

Secondary Target Audience:

• Home buyers in Canada[3] (considering purchasing a home in the next five years)

3. Methodology

3.1 Online Sample

Respondents for the survey samples were drawn from a trusted partner panel vendor, Canadian Viewpoint Inc. The survey was conducted with a sample of n=2,010 Canadians ages 18+. Interviews were conducted in the language of their choice, either English or French.

A pre-test was launched on May 19th, 2023, which garnered 49 completes (29 English / 20 French). An open-ended question was asked at the end of the survey where any problems, questions, or unclear questions could be brought to our attention; no issues were flagged. The survey was fully launched and ran between May 20th and 29th, 2023.

Quotas were set to ensure representation by region, age, and gender, according to the latest Census information.

The sampling methodology utilized email invitations and router technology to invite participants. Each participant received a unique URL link. This link could only be used once, with respondents being allowed to pause during completion and return to complete. On average, the online interviews took 10 minutes to complete.

3.2 Participant Recruitment

Sample Source

Ipsos partnered with sub-contractor Canadian Viewpoint Inc. Canadian Viewpoint has one of the largest consumer panels with ~300,000 active panelists. Like Ipsos, Canadian Viewpoint uses mixed-medium strategies (phone/online/in-person) to build its panel to allow more representative samples. Canadian Viewpoint’s online consumer panel is also recruited using post screeners during telephone surveys, Facebook, online vetted lists and in-person studies. Incentives were not used for recruitment purposes to ensure quality, but respondents were incentivized for completing the survey, which is directly proportionate to the amount of time taken to complete the survey and with comparable incentives offered by other online panel sources.

The comprehensive background profiling data were gathered when respondents joined the panel through screener questions, which allowed for the targeting of respondents based on key criteria, such as region, age, gender, education, and income level, intention to buy a home within 12 months, profession, and other characteristics.

The online survey was conducted using Computer Assisted Web Interviewing (CAWI). Every panelist opted-in or chose to participate in research surveys. Panelists that meet the criteria we were looking for were randomly selected and sent an email invitation to complete the survey. Participants were offered a choice to complete the survey in either English or French.

The survey platform was Accessibility for Ontarians with Disabilities Act (AODA) compliant, according to Web Content Accessibility Guidelines (WCAG2.0AA).

Sample Weighting

The table below shows the unweighted and weighted distributions of the online sample. Weighting was applied to the sample to ensure that the final data reflects the general adult population by region, age, and gender according to the 2021 Census. A Random Iterative Method (RIM) technique was applied for weighing.

Weighted and Unweighted Online Sample

The figures presented in the table above show minimal differences between the unweighted and weighted samples. As a result of the strong representativeness of the unweighted data the largest weight factor that was applied for any respondent is 1.08, which is well within acceptable ranges for a survey of the general population.

Within natural fallout, we obtained our quota targets without having to oversample. The unweighted counts are shown below.

Incentives and Quality Control Measures

Canadian Viewpoint use a multi-faceted approach to quality, integrating controls and best practices into every phase of the process. Panel members come from diverse fully certified sources. A dedicated team monitors performance by source using feedback loops and continuous reporting to prevent fraud. Our dedicated team monitors the panels for potentially fraudulent behavior, and blocks these people from ever entering a survey via safeguards that includes:

• A double opt-in authentication process
• Cash incentives, in the form of cheques to ensure confirmation of our panelist identity
• Respondents provide us with legitimate information about their identity and address and allow them to receive their incentive cheque
• Immediate examination of suspicious email addresses, suspicious behavior, or unrealistic completion
• Fast and permanent removal of offenders
• CVI works with our clients to set up customized data quality checks for the surveys that we program. When we are providing Sample Only, the client is ultimately responsible for setting up their own data quality checks
• We limit invitations to 3 x a week. The average respondent receives 1.38 surveys a month from Ipsos and completes 5 - 6 surveys per year
• CVI keeps the participation history, join date and source of the respondent on file. Project analysis of such data is available to our clients upon request
• Proprietary Verification is a suite of quality control procedures, including digital fingerprinting, address verification against USPS databases (standard in the US and Canada,) and third-party verification upon request

Email Statistics

For this survey, a sample router was used. Therefore, a response rate cannot be calculated. However, the participation rate for the survey was 90%, which is calculated as follows: (qualified completes + over quota + terminates/click-through).

Non-Response Analysis

The results of this survey are not statistically projectable to the target population because the sampling method used does not ensure that the sample represents the target population with a known margin of sampling error. Reported percentages are not generalizable to any group other than the sample studied, and therefore no formal statistical inferences can be drawn between the sample results and the broader target population it may be intended to reflect. The data have been weighted to reflect the demographic composition of Canadian adults aged 18 years and over.

The table below compares the unweighted survey samples to the 2021 Census results by region, age, and gender. Overall, the sample is highly representative of the national adult population.

* Denotes variables included in the weighting scheme.

Differences among the variables used in the weighting scheme are minimal. The distribution in the sample is consistent, with only slight differences observed between the unweighted percentages and the 2021 Census data.

[1] The term “vulnerable” is being used in this instance to describe individuals who may be hard to reach or those who experience certain situational or systemic challenges that can negatively impact their outcomes or put them at a disadvantage.

[2] According to the 2017 Canadian Survey on Disability, more than 6 million Canadians aged 15 and over (22% of the population) identify as having a disability, and it is expected actual numbers are likely higher. (https://www.canada.ca/en/employment-social-development/programs/accessible-canada.html) The CSD definition of disability includes anyone who reported being "sometimes," "often" or "always" limited in their daily activities due to a long-term condition or health problem, as well as anyone who reported being "rarely" limited if they were also unable to do certain tasks or could only do them with a lot of difficulties.

[3] The CRA is also looking for potential new homebuyers earning $60K or more (as part of the natural fallout in the total sample of the 2000 general population; not part of set quotas). There will be questions about the intention to purchase a home among this income segment within the next three years to capture the natural fallout within the sample.