Prepared for the Canada Revenue Agency
Supplier name: Ipsos Public Affairs
Contract Number: EP363-140002/009/CY
Contract value: $48,921.00
Award Date: February 3rd, 2022
Delivery Date: June 10th, 2022
Registration number: POR 118-21
For more information on this report, please contact Canada Revenue Agency at: cra-arc.media@cra-arc.gc.ca
Ce rapport est aussi disponible en français.
I hereby certify as Senior Officer of Ipsos that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.
Mike Colledge
President
Ipsos Public Affairs
Prepared for Canada Revenue Agency (CRA)
Supplier name: Ipsos Public Affairs
June 2022
This public opinion research report presents the methodology of the 2021-2022 CRA Benefits and Credits Campaign - Advertising Campaign Evaluation Tool (ACET) online survey conducted by Ipsos Public Affairs on behalf of Canada Revenue Agency (CRA) of Canada. The research study was conducted with a total sample of 4033 Canadians aged 18+: n=2002 in the pre-campaign wave between February 4th and 11h, 2022, and n=2040 in the post-campaign wave between May 4th and May 12th, 2022.
Cette publication est aussi disponible en français sous le titre : La campagne publicitaire des prestations et crédits de l’ARC de 2021-2022 - Outil d’évaluation des campagnes publicitaires (OECP) - Sommaire exécutif.
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from the Canada Revenue Agency (CRA). For more information on this report, please contact the CRA at cra-arc.media@cra-arc.gc.ca
101 Colonel By Drive
Ottawa, Ontario K1A 0K2
Canada
Catalogue Number: Rv4-137/2-2022E-PDF
International Standard Book Number (ISBN): 978-0-660-44565-6
Related publications (registration number: POR 118-21):
Catalogue Number: Rv4-137/2-2022F-PDF (Executive Summary, French)
ISBN 978-0-660-44567-0
© His Majesty the King in Right of Canada, as represented by the Minister of the Canada Revenue Agency, 2022.
The Canada Revenue Agency (CRA) administers taxes, benefits, and related programs for governments across Canada. These services and programs help Canadians comply with tax obligations and receive benefits, which contributes to social economic well-being. The CRA is undertaking significant efforts to increase awareness and access to CRA benefits and services, participation in the tax system and understanding of roles and responsibilities in the tax and benefit system.
Tax benefits and credits provide additional income or tax relief to individuals to help make life more affordable since many depend on benefit payments and tax credits for a significant part of their household income. For modest income households, benefits can increase income by as much as 50 per cent.
However, vulnerable populations such as low-income households, Indigenous peoples, newcomers to Canada, and youth aged 18-24 face a variety of barriers when doing their taxes and receiving their benefits and credits. These barriers include a lack of general awareness of benefits and credits, lack of support to complete their taxes, living in remote locations, language barriers, low financial literacy and/or mental or physical health issues. Given the range of backgrounds and circumstances, it is important for the Government of Canada (GC) to make proactive and targeted efforts to reach vulnerable populations with important tax-related information. Moreover, greater awareness is needed so these populations know that doing their taxes every year is necessary to be eligible for these benefits and credits. Increasing awareness of benefits and credits will help to contribute to the government priority of poverty reduction, health and wellbeing, as found in the GC’s Gender Results Framework.
The Benefits and Credits advertising campaign is part of a sustained effort by the CRA to raise awareness amongst vulnerable populations footnote 1 of the benefits and credits that they are entitled to when they do their taxes and of the availability of free tax clinics. The ad campaign featured the Canada child benefit (CCB), Canada workers benefit (CWB), GST/HST credit, the disability tax credit (DTC) and the Community Volunteer Income Tax Program’s (CVITP) free tax clinics.
The Phase One campaign was focused on benefits and credits awareness. The launch date was February 14, 2022, and ran until May 2, 2022. The Phase Two campaign was focused on late filer messaging. The launch date was May 3, 2022, and ran until June 30, 2022.
The advertising campaign aims to:
Ipsos was contracted to conduct the data collection and tabulation, and to provide a methodology report. Ipsos programmed, hosted, and provided sample management services, while the CRA provided the online questionnaires. Ipsos was responsible for data collection and data storage in Canada, data processing, and data weighting. The data collection from Canadians was handled in accordance with government-wide Public Opinion Research procedures.
The research consisted of online surveys conducted in English and French, with the pre-campaign evaluation taking place between February 4th and 11th, 2022 and the post-campaign evaluation taking place between May 4th to 12th, 2022.
The total contract value of this research was $48,921.00, including HST.
The objective of the research was to assist in determining the effectiveness of the advertising campaign by measuring awareness of the subject matter with the audiences. The research involved a baseline survey to collect information on the status of awareness and use of these resources prior to the campaign. The post-campaign survey measured the impact of the campaign on awareness and usage.
This research has two components:
The Government of Canada’s Policy on Communications and Federal Identity requires the evaluation of all advertising campaigns exceeding $1 million in media through the Advertising Campaign Evaluation Tool (ACET).
The target audience of the research was as follows:
Primary Target Audience:
Secondary Target Audience
This project involved two (2) online surveys – a baseline survey and a post-campaign survey. Both surveys were executed online using a non-probability online panel. This is the standard approach for all Government of Canada advertising evaluation surveys. The initial baseline survey was conducted before the campaign launched and the post-campaign survey was conducted following the completion of the campaign.
Respondents for the survey were drawn from the Ipsos’ i-Say panel and a trusted partner panel vendor, Dynata. The baseline survey was conducted with a sample of n=2,002 Canadians ages 18+ and the post-campaign survey was conducted with a sample of n=2,040 Canadians ages 18+. Survey respondents took the survey in the official language of their choice, either English or French.
For the baseline survey a pre-test was conducted on February 4th, 2022, with 36 completes (24 English / 12 French), to confirm survey length before fully deploying the questionnaire. An open-ended question was asked at the end of the survey where any problems with the clarity of the survey questions could be brought to our attention; no issues were flagged. The survey was fully launched and ran between February 4th and 11th, 2022.
For the post-campaign survey, a pre-test was conducted on May 4th, 2022, with 42 completes (27 English / 15 French). No issues were flagged. The survey was fully launched and ran between May 5th and 12th, 2022.
For both surveys, quotas were set to ensure representation by region, age and gender, according to the latest Census information.In the end, within natural fallout we obtained our targets without having to oversample. The unweighted counts are shown below.
| Target audience | Baseline Survey Sample Size | Post-campaign Survey Sample Size |
|---|---|---|
| Indigenous | 121 | 126 |
| Moderate-income (<$40K) | 607 | 586 |
| Newcomers <5 years | 106 | 58 |
| People with disabilities | 449 | 400 |
| Youth 18-24 | 199 | 208 |
The sampling methodology utilized email invitations + router technology to invite participants. Each participant received a unique URL link. This link could only be used once, with respondents being allowed to pause during completion and return to complete. On average, baseline survey was completed within 7 minutes and post survey was completed within 10 minutes.
Ipsos partnered with Dynata on the fieldwork and in obtaining the required sample. Ipsos and Dynata have over 300,000 active panelists. Dynata’s panels are continually refreshed and recruited through various channels including 1) loyalty program sourcing across travel, entertainment and other sectors; 2) online banners, cable TV advertising, mailings, social media influencers, mobile app, etc.; 3) integrated channels including access to online communities, social media platforms, publishers and others (this last group has not opted into a panel) but each participant has a Dynata profile.
Dynata’s panel includes members who have opted in and receive a form of incentive. Incentives are based on a point-based system cashed in for rewards (electronic gift certificates, high street vouchers, charity donations, and long-term loyalty rewards), as well as sweepstakes. The amount of incentives is based on the specific requirements of each survey, depending on the length and complexity of the survey, the subject matter of the study, and the time required to complete a minimum number of interviews.
The comprehensive background profiling data gathered when respondents join a panel allows for the targeting of respondents based on key criteria, such as region, age, gender, education and income level, intention to buy a home within 12 months, profession, and other characteristics.
The data excludes any duplicate respondents based on IP capture and excludes panelists who have completed another Government of Canada survey in the previous 30 days as a member of that panel.
The table below shows the unweighted and weighted distributions of the online sample. Weighting was applied to the sample to ensure that the final data reflects the general adult population by region, age, and gender according to the 2016 Census. A Random Iterative Method (RIM) technique was applied for weighting. Note: totals may not add to 100% due to rounding.
| Demographics | Baseline Survey | Post-campaign Survey | ||
|---|---|---|---|---|
| Unweighted Sample Size | Weighted Sample Size | Unweighted Sample Size | Weighted Sample Size | |
| Canada | 2002 | 2002 | 2040 | 2040 |
| Region* | ||||
| British Columbia/Yukon | 278 | 268 | 276 | 275 |
| Alberta/ Northwest Territories | 222 | 225 | 230 | 230 |
| Prairies (MB/SK)/ Nunavut | 143 | 133 | 147 | 134 |
| Ontario | 759 | 769 | 778 | 784 |
| Quebec | 460 | 470 | 467 | 479 |
| Atlantic Canada | 140 | 137 | 142 | 139 |
| Gender* | ||||
| Male | 975 | 980 | 967 | 993 |
| Female | 1024 | 1019 | 1060 | 1034 |
| Diverse | 3 | 3 | 13 | 13 |
| Age* | ||||
| 18-34 | 540 | 561 | 532 | 571 |
| 35-54 | 670 | 681 | 691 | 694 |
| 55+ | 792 | 761 | 817 | 775 |
The figures presented in the table above show minimal differences between the unweighted and weighted samples. As a result of the strong representativeness of the unweighted data the largest weight factor that was applied for any respondent is 1.07, which is well within acceptable ranges for a survey of the general population.
For this survey, a sample router was used. Therefore, a response rate cannot be calculated. However, the participation rate for the survey was 97% for the Baseline-campaign wave and 93% for the post-campaign wave. Participation rate is calculated as follows: (qualified completes + over quota + terminates)/click-through).
| Completions | Baseline-campaign | Post-campaign |
|---|---|---|
| Click-Through | 2543 | 2822 |
| - Partial Completes | 82 | 173 |
| - Terminates | 438 | 244 |
| - Over quota | 21 | 365 |
| Qualified Completes | 2002 | 2040 |
| Participation Rate | 97% | 94% |
The results of this survey are not statistically projectable to the target population because the sampling method used does not ensure that the sample represents the target population with a known margin of sampling error. Reported percentages are not generalizable to any group other than the sample studied, and therefore no formal statistical inferences can be drawn between the sample results and the broader target population it may be intended to reflect. The data have been weighted to reflect the demographic composition of Canadian adults aged 18 years and over.
The table below compares the unweighted survey samples to the 2016 Census results by region, age, and gender. Overall, the sample is highly representative of the national adult population.
| Demographics | Baseline Survey | Post-campaign Survey | ||
|---|---|---|---|---|
| Unweighted Sample Size | Weighted Sample Size | Unweighted Sample Size | Weighted Sample Size | |
| Region* | ||||
| British Columbia/Yukon | 14% | 13% | 14% | 13% |
| Alberta/ Northwest Territories | 11% | 11% | 11% | 11% |
| Prairies (MB/SK)/ Nunavut | 7% | 7% | 7% | 7% |
| Ontario | 38% | 38% | 38% | 38% |
| Quebec | 23% | 23% | 23% | 23% |
| Atlantic Canada | 7% | 7% | 7% | 7% |
| Gender* | ||||
| Male | 49% | 49% | 47% | 49% |
| Female | 51% | 51% | 52% | 51% |
| Diverse | <1% | <1% | 1% | <1% |
| Age* | ||||
| 18-34 | 27% | 28% | 26% | 28% |
| 35-54 | 33% | 34% | 34% | 34% |
| 55+ | 40% | 38% | 40% | 38% |
* Denotes variables included in the weighting scheme.
Differences among the variables used in the weighting scheme are minimal. The distribution in the sample is consistent, with only slight differences observed between the unweighted percentages and the 2016 Census data.
Footnote 1. The term “vulnerable” is being used in this instance to describe individuals who may be hard to reach or those who experience certain situational or systemic challenges that can negatively impact their outcomes or put them at a disadvantage. Return to footnote 1 source paragraph
Footnote 2. According to the 2017 Canadian Survey on Disability, more than 6 million Canadians aged 15 and over (22% of the population) identify as having a disability, and it is expected actual numbers are likely higher. (https://www.canada.ca/en/employment-social-development/programs/accessible-canada.html) The CSD definition of disability includes anyone who reported being "sometimes," "often" or "always" limited in their daily activities due to a long-term condition or health problem, as well as anyone who reported being "rarely" limited if they were also unable to do certain tasks or could only do them with a lot of difficulty. Return to footnote 2 source paragraph