Supplier name: Ipsos
Contract Number: CW2230192
Contract value: $42,488.57
Award Date: April 27th, 2024
Delivery Date: May 24th, 2024
Registration number: POR 143-23
For more information on this report, please contact Canadian Revenue Agency at:
cra-arc.media@cra-arc.gc.ca
Ce rapport est aussi disponible en français
Prepared for Canadian Revenue Agency (CRA)
Supplier name: Ipsos
May 2024
This public opinion research report presents the methodology of the 2023-2024 CRA Benefits and Credits Campaign – ACET online survey conducted by Ipsos on behalf of the Canadian Revenue Agency (CRA).
The online survey was conducted with 2,148 respondents between May 2nd and 15th, 2024.
Cette publication est aussi disponible en français sous le titre Campagne publicitaire sur les prestations et les crédits de l'ARC de 2023-2024
Outil d'évaluation des campagnes publicitaires (OECP) – Sommaire Exécutif
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Canada Revenue Agency. For more information on this report, please contact Canada Revenue Agency at: cra-arc.media@cra-arc.gc.ca or at:
101 Colonel By Drive
Ottawa, Ontario K1A 0K2
Canada
Catalogue Number: Rv4-137/1-2024E-PDF
International Standard Book Number (ISBN): 978-0-660-72621-2
Related publications (registration number: POR 143-23):
Catalogue Number: Rv4-137/1-2024F-PDF (Methodological Report, French)
ISBN: 978-0-660-72622-9
© His Majesty the King in Right of Canada, as represented by the Minister of the Canada Revenue Agency, 2024
POLITICAL NEUTRALITY STATEMENT
I hereby certify as Senior Officer of Ipsos that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.
Brad Griffin
President
Ipsos Public Affairs
The Canada Revenue Agency (CRA) administers taxes, benefits, and related programs for governments across Canada. These services and programs help residents of Canada comply with tax obligations and receive benefits, contributing to socio-economic well-being. The CRA is undertaking significant efforts to increase awareness of and access to CRA benefits and services, participation in the tax system, and understanding of roles and responsibilities within the tax and benefit system.
Benefits and credits provide additional income or tax relief to individuals to help make life more affordable. Many depend on benefit payments and tax credits for a significant part of their household income. For modest-income households, benefits can increase income by as much as 50 percent.
However, certain segments of the vulnerable population, such as modest-income households, Indigenous Peoples, newcomers to Canada, and youth aged 18–24, face various barriers when filing their taxes and receiving their benefits and credits. These barriers include a lack of general awareness of benefits and credits, a lack of support to complete their taxes, living in remote locations, language barriers, low financial literacy, and/or mental or physical health issues. Given the range of backgrounds and circumstances, it is important for the Government of Canada (GC) to make proactive and targeted efforts to reach vulnerable populations with important tax-related information. Moreover, greater awareness is needed so these populations know that filing their taxes every year is necessary to be eligible for these benefits and credits. Increasing awareness of benefits and credits will help contribute to the government's priorities of poverty reduction, health, and well-being, as found in the GC's Gender Results Framework.
This Benefits and Credits advertising campaign is part of a sustained effort by the CRA to raise awareness among target audiences about the benefits and credits they are entitled to when they do their taxes. The campaign featured the Canada Child Benefit (CCB), Canada workers benefit (CWB), GST/HST credit, the disability tax credit (DTC), and the Community Volunteer Income Tax Program's (CVITP) free tax clinics. The campaign also promoted housing programs, including the First Home Savings Account, the Multigenerational Home Renovation Tax Credit, the Home Buyers' Plan, the GST/HST new housing rebate, and the home buyers' amount
Advertising campaign evaluation is mandatory for all advertising campaigns in which media buys exceed $1 million. The Advertising Campaign Evaluation Tool (ACET) is used for all major Government of Canada (GC) advertising campaigns in order to consistently evaluate these campaigns and comply with key requirements in the GC Communications Policy.
Ipsos was contracted to conduct the data collection and tabulation, and to provide a methodology report. Ipsos programmed, hosted, and provided sample management services, while the CRA provided the online questionnaire. Ipsos was responsible for data collection and data storage in Canada, data processing, and data weighting. The data collection from Canadians was handled in accordance with government-wide Public Opinion Research procedures.
The research consisted of an online post-campaign survey conducted in English and French, between May 2nd and 15th, 2024.
The total contract value of this research was $42,488.57, including HST.
Advertising campaign evaluations assist in determining the effectiveness of the selected advertising campaigns by measuring (and comparing) awareness of the subject matter with the audiences. The results will be used to evaluate the effectiveness of the Benefits and Credits advertising campaign with the target audience. The results will also help assess whether the ads were compelling / caught the audiences' attention, and whether the campaign raised awareness of the benefits and credits being promoted. Finally, the evaluation will assist the CRA in improving or modifying future ad campaigns.
For this research, CRA was seeking to evaluate the campaign which ended in April 2024. This post-campaign survey used the standardized ACET Post-Campaign online survey to assess:
The target audience of the research was as follows:
Primary Target Audience:
Secondary Target Audience:
Respondents for the survey samples were drawn from a trusted partner panel vendor, Canadian Viewpoint Inc. The survey was conducted with a sample of 2,148 Canadians ages 18+. Interviews were conducted in the language of their choice, either English or French.
A pre-test was launched on May 2nd, 2024, which garnered 37 completes (25 English / 12 French). An open-ended question was asked at the end of the survey where any problems, questions, or unclear questions could be brought to our attention; no issues were flagged. The survey was fully launched and ran between May 3rd and 15th, 2024.
Quotas were set to ensure representation by region, age, and gender, according to the latest Census information.
The sampling methodology utilized email invitations and router technology to invite participants. Each participant received a unique URL link. This link could only be used once, with respondents being allowed to pause during completion and return to complete. On average, the online interviews took 10 minutes to complete.
Ipsos partnered with sub-contractor Canadian Viewpoint Inc. Canadian Viewpoint has one of the largest consumer panels with ~300,000 active panelists. Like Ipsos, Canadian Viewpoint uses mixed-medium strategies (phone/online/in-person) to build its panel to allow more representative samples. Canadian Viewpoint's online consumer panel is also recruited using post screeners during telephone surveys, Facebook, online vetted lists and in-person studies. Incentives were not used for recruitment purposes to ensure quality, but respondents were incentivized for completing the survey, which is directly proportionate to the amount of time taken to complete the survey and with comparable incentives offered by other online panel sources.
The comprehensive background profiling data were gathered when respondents joined the panel through screener questions, which allowed for the targeting of respondents based on key criteria, such as region, age, gender, education, and income level, intention to buy a home within 12 months, profession, and other characteristics.
The online survey was conducted using Computer Assisted Web Interviewing (CAWI). Every panelist opted-in or chose to participate in research surveys. Panelists that meet the criteria we were looking for were randomly selected and sent an email invitation to complete the survey. Participants were offered a choice to complete the survey in either English or French.
The survey platform was Accessibility for Ontarians with Disabilities Act (AODA) compliant, according to Web Content Accessibility Guidelines (WCAG2.0AA).
Sample Weighting
The table below shows the unweighted and weighted distributions of the online sample. Weighting was applied to the sample to ensure that the final data reflects the general adult population by region, age, and gender according to the 2021 Census. A Random Iterative Method (RIM) technique was applied for weighing.
| Demographics | Post-wave Survey | |
| Unweighted Sample Size | Weighted Sample Size | |
| Canada | 2148 | 2148 |
| Region | ||
| British Columbia/Yukon | 283 | 301 |
| Alberta/Northwest Territories | 239 | 236 |
| Prairies (MB/SK) | 161 | 129 |
| Ontario/ Nunavut | 850 | 838 |
| Quebec | 473 | 494 |
| Atlantic Canada | 142 | 150 |
| Gender | ||
| Male | 1008 | 1048 |
| Female | 1133 | 1092 |
| Diverse / Prefer not to answer | 7 | 8 |
| Age | ||
| 18-24 | 236 | 253 |
| 25-34 | 302 | 327 |
| 35-44 | 252 | 347 |
| 45-54 | 247 | 340 |
| 55-64 | 276 | 223 |
| 65+ | 835 | 658 |
The figures presented in the table above show minimal differences between the unweighted and weighted samples, except for the age group 35-64. As a result of the strong representativeness of the unweighted data the largest weight factor that was applied for any respondent is 1.38, which is well within acceptable ranges for a survey of the general population.
Within natural fallout, we obtained our quota targets, apart from an oversample of Indigenous respondents (n=50).
The unweighted counts are shown below.
| Primary target audience | Target | Achieved |
| Indigenous | 100 | 109 |
| Moderate-income (<$50K) | 400 – 500 | 664 |
| Newcomers <7years | ~50 | 108 |
| People with disabilities | 400 – 440 | 431 |
| Youth aged 18-24 | 200 – 250 | 226 |
| Secondary target audience | ||
| Potential new homebuyers | -- | 453 |
Incentives and Quality Control Measures
Canadian Viewpoint use a multi-faceted approach to quality, integrating controls and best practices into every phase of the process. Panel members come from diverse fully certified sources. A dedicated team monitors performance by source using feedback loops and continuous reporting to prevent fraud. Our dedicated team monitors the panels for potentially fraudulent behavior, and blocks these people from ever entering a survey via safeguards that includes:
Email Statistics
For this survey, a sample router was used. Therefore, a response rate cannot be calculated. However, the participation rate for the survey was 91%, which is calculated as follows: (qualified completes + over quota + terminates)/click-through).
| Completions | Post-wave Survey |
| Click-Through | 3987 |
| - Partial Completes | 56 |
| - Terminates | 696 |
| - Over quota | 793 |
| Qualified Completes | 2148 |
| Participation Rate | 91% |
Non-Response Analysis
The results of this survey are not statistically projectable to the target population because the sampling method used does not ensure that the sample represents the target population with a known margin of sampling error. Reported percentages are not generalizable to any group other than the sample studied, and therefore no formal statistical inferences can be drawn between the sample results and the broader target population it may be intended to reflect. The data have been weighted to reflect the demographic composition of Canadian adults aged 18 years and over.
The table below compares the unweighted survey samples to the 2021 Census results by region, age, and gender. Overall, the sample is highly representative of the national adult population.
| Demographics | Post-wave Survey | |
| Unweighted Percentage | Census 2021 Proportions | |
| Canada | ||
| Region | ||
| British Columbia/Yukon | 13% | 14% |
| Alberta/Northwest Territories | 11% | 11% |
| Prairies (MB/SK) | 7% | 7% |
| Ontario/ Nunavut | 40% | 38% |
| Quebec | 22% | 23% |
| Atlantic Canada | 7% | 7% |
| Gender | ||
| Male | 47% | 49% |
| Female | 53% | 51% |
| Diverse / Prefer not to answer | <1% | <1% |
| Age | ||
| 18-24 | 11% | 11% |
| 25-34 | 14% | 17% |
| 35-44 | 12% | 17% |
| 45-54 | 11% | 16% |
| 55-64 | 13% | 18% |
| 65+ | 39% | 24% |
Differences among the variables used in the weighting scheme are minimal. The distribution in the sample is consistent, with slight differences observed between the unweighted percentages and the 2021 Census data.
[1] According to the 2017 Canadian Survey on Disability, more than 6 million Canadians aged 15 and over (22% of the population) identify as having a disability, and it is expected actual numbers are likely higher. (https://www.canada.ca/en/employment-social-development/programs/accessible-canada.html) The CSD definition of disability includes anyone who reported being "sometimes," "often" or "always" limited in their daily activities due to a long-term condition or health problem, as well as anyone who reported being "rarely" limited if they were also unable to do certain tasks or could only do them with a lot of difficulties.
[2] The CRA is also looking for potential new homebuyers earning $60K or more (as part of the natural fallout in the total sample of the 2000 general population; not part of set quotas). There will be questions about the intention to purchase a home among this income segment within the next three years to capture the natural fallout within the sample.