Prepared for the Canada Revenue Agency
Supplier name: Ipsos Public Affairs
Contract Number: EP363-140002/009/CY
Contract value: $48,921.00
Award Date: February 3rd, 2022
Delivery Date: June 10th, 2022
Registration number: POR 118-21
For more information on this report, please contact Canada Revenue Agency at: cra-arc.media@cra-arc.gc.ca
Ce rapport est aussi disponible en français.
I hereby certify as Senior Officer of Ipsos that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.
Mike Colledge
President
Ipsos Public Affairs
Prepared for Canada Revenue Agency (CRA)
Supplier name: Ipsos Public Affairs
June 2022
This public opinion research report presents the methodology of the 2021-2022 CRA Benefits and Credits Campaign - Advertising Campaign Evaluation Tool (ACET) online survey conducted by Ipsos Public Affairs on behalf of Canada Revenue Agency (CRA) of Canada. The research study was conducted with a total sample of 4033 Canadians aged 18+: n=2002 in the pre-campaign wave between February 4th and 11h, 2022, and n=2040 in the post-campaign wave between May 4th and May 12th, 2022.
Cette publication est aussi disponible en français sous le titre : La campagne publicitaire des prestations et crédits de l’ARC de 2021-2022 - Outil d’évaluation des campagnes publicitaires (OECP) - Rapport méthodologique.
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from the Canada Revenue Agency (CRA). For more information on this report, please contact the CRA at cra-arc.media@cra-arc.gc.ca
101 Colonel By Drive
Ottawa, Ontario K1A 0K2
Canada
Catalogue Number: Rv4-137/1-2022E-PDF
International Standard Book Number (ISBN): 978-0-660-44564-9
Related publications (registration number: POR 118-21):
Catalogue Number: Rv4-137/1-2022F-PDF (Methodological Report, French)
ISBN 978-0-660-44566-3
© His Majesty the King in Right of Canada, as represented by the Minister of the Canada Revenue Agency, 2022.
The Canada Revenue Agency (CRA) administers taxes, benefits, and related programs for governments across Canada. These services and programs help Canadians comply with tax obligations and receive benefits, which contributes to social economic well-being. The CRA is undertaking significant efforts to increase awareness and access to CRA benefits and services, participation in the tax system and understanding of roles and responsibilities in the tax and benefit system.
Tax benefits and credits provide additional income or tax relief to individuals to help make life more affordable since many depend on benefit payments and tax credits for a significant part of their household income. For modest income households, benefits can increase income by as much as 50 per cent.
However, vulnerable populations such as low-income households, Indigenous peoples, newcomers to Canada, and youth aged 18-24 face a variety of barriers when doing their taxes and receiving their benefits and credits. These barriers include a lack of general awareness of benefits and credits, lack of support to complete their taxes, living in remote locations, language barriers, low financial literacy and/or mental or physical health issues. Given the range of backgrounds and circumstances, it is important for the Government of Canada (GC) to make proactive and targeted efforts to reach vulnerable populations with important tax-related information. Moreover, greater awareness is needed so these populations know that doing their taxes every year is necessary to be eligible for these benefits and credits. Increasing awareness of benefits and credits will help to contribute to the government priority of poverty reduction, health and wellbeing, as found in the GC’s Gender Results Framework.
The Benefits and Credits advertising campaign is part of a sustained effort by the CRA to raise awareness amongst vulnerable populations footnote 1 of the benefits and credits that they are entitled to when they do their taxes and of the availability of free tax clinics. The ad campaign featured the Canada child benefit (CCB), Canada workers benefit (CWB), GST/HST credit, the disability tax credit (DTC) and the Community Volunteer Income Tax Program’s (CVITP) free tax clinics.
The Phase One campaign was focused on benefits and credits awareness. The launch date was February 14, 2022, and ran until May 2, 2022. The Phase Two campaign was focused on late filer messaging. The launch date was May 3, 2022, and ran until June 30, 2022.
The advertising campaign aims to:
Ipsos was contracted to conduct the data collection and tabulation, and to provide a methodology report. Ipsos programmed, hosted, and provided sample management services, while the CRA provided the online questionnaires. Ipsos was responsible for data collection and data storage in Canada, data processing, and data weighting. The data collection from Canadians was handled in accordance with government-wide Public Opinion Research procedures.
The research consisted of online surveys conducted in English and French, with the pre-campaign evaluation taking place between February 4th and 11th, 2022 and the post-campaign evaluation taking place between May 4th to 12th, 2022.
The total contract value of this research was $48,921.00, including HST.
The objective of the research was to assist in determining the effectiveness of the advertising campaign by measuring awareness of the subject matter with the audiences. The research involved a baseline survey to collect information on the status of awareness and use of these resources prior to the campaign. The post-campaign survey measured the impact of the campaign on awareness and usage.
This research has two components:
The Government of Canada’s Policy on Communications and Federal Identity requires the evaluation of all advertising campaigns exceeding $1 million in media through the Advertising Campaign Evaluation Tool (ACET).
The target audience of the research was as follows:
Primary Target Audience:
Secondary Target Audience
This project involved two (2) online surveys – a baseline survey and a post-campaign survey. Both surveys were executed online using a non-probability online panel. This is the standard approach for all Government of Canada advertising evaluation surveys. The initial baseline survey was conducted before the campaign launched and the post-campaign survey was conducted following the completion of the campaign.
Respondents for the survey were drawn from the Ipsos’ i-Say panel and a trusted partner panel vendor, Dynata. The baseline survey was conducted with a sample of n=2,002 Canadians ages 18+ and the post-campaign survey was conducted with a sample of n=2,040 Canadians ages 18+. Survey respondents took the survey in the official language of their choice, either English or French.
For the baseline survey a pre-test was conducted on February 4th, 2022, with 36 completes (24 English / 12 French), to confirm survey length before fully deploying the questionnaire. An open-ended question was asked at the end of the survey where any problems with the clarity of the survey questions could be brought to our attention; no issues were flagged. The survey was fully launched and ran between February 4th and 11th, 2022.
For the post-campaign survey, a pre-test was conducted on May 4th, 2022, with 42 completes (27 English / 15 French). No issues were flagged. The survey was fully launched and ran between May 5th and 12th, 2022.
For both surveys, quotas were set to ensure representation by region, age and gender, according to the latest Census information.In the end, within natural fallout we obtained our targets without having to oversample. The unweighted counts are shown below.
| Target audience | Baseline Survey Sample Size | Post-campaign Survey Sample Size |
|---|---|---|
| Indigenous | 121 | 126 |
| Moderate-income (<$40K) | 607 | 586 |
| Newcomers <5 years | 106 | 58 |
| People with disabilities | 449 | 400 |
| Youth 18-24 | 199 | 208 |
The sampling methodology utilized email invitations + router technology to invite participants. Each participant received a unique URL link. This link could only be used once, with respondents being allowed to pause during completion and return to complete. On average, baseline survey was completed within 7 minutes and post survey was completed within 10 minutes.
Ipsos partnered with Dynata on the fieldwork and in obtaining the required sample. Ipsos and Dynata have over 300,000 active panelists. Dynata’s panels are continually refreshed and recruited through various channels including 1) loyalty program sourcing across travel, entertainment and other sectors; 2) online banners, cable TV advertising, mailings, social media influencers, mobile app, etc.; 3) integrated channels including access to online communities, social media platforms, publishers and others (this last group has not opted into a panel) but each participant has a Dynata profile.
Dynata’s panel includes members who have opted in and receive a form of incentive. Incentives are based on a point-based system cashed in for rewards (electronic gift certificates, high street vouchers, charity donations, and long-term loyalty rewards), as well as sweepstakes. The amount of incentives is based on the specific requirements of each survey, depending on the length and complexity of the survey, the subject matter of the study, and the time required to complete a minimum number of interviews.
The comprehensive background profiling data gathered when respondents join a panel allows for the targeting of respondents based on key criteria, such as region, age, gender, education and income level, intention to buy a home within 12 months, profession, and other characteristics.
The data excludes any duplicate respondents based on IP capture and excludes panelists who have completed another Government of Canada survey in the previous 30 days as a member of that panel.
The table below shows the unweighted and weighted distributions of the online sample. Weighting was applied to the sample to ensure that the final data reflects the general adult population by region, age, and gender according to the 2016 Census. A Random Iterative Method (RIM) technique was applied for weighting. Note: totals may not add to 100% due to rounding.
| Demographics | Baseline Survey | Post-campaign Survey | ||
|---|---|---|---|---|
| Unweighted Sample Size | Weighted Sample Size | Unweighted Sample Size | Weighted Sample Size | |
| Canada | 2002 | 2002 | 2040 | 2040 |
| Region* | ||||
| British Columbia/Yukon | 278 | 268 | 276 | 275 |
| Alberta/ Northwest Territories | 222 | 225 | 230 | 230 |
| Prairies (MB/SK)/ Nunavut | 143 | 133 | 147 | 134 |
| Ontario | 759 | 769 | 778 | 784 |
| Quebec | 460 | 470 | 467 | 479 |
| Atlantic Canada | 140 | 137 | 142 | 139 |
| Gender* | ||||
| Male | 975 | 980 | 967 | 993 |
| Female | 1024 | 1019 | 1060 | 1034 |
| Diverse | 3 | 3 | 13 | 13 |
| Age* | ||||
| 18-34 | 540 | 561 | 532 | 571 |
| 35-54 | 670 | 681 | 691 | 694 |
| 55+ | 792 | 761 | 817 | 775 |
The figures presented in the table above show minimal differences between the unweighted and weighted samples. As a result of the strong representativeness of the unweighted data the largest weight factor that was applied for any respondent is 1.07, which is well within acceptable ranges for a survey of the general population.
For this survey, a sample router was used. Therefore, a response rate cannot be calculated. However, the participation rate for the survey was 97% for the Baseline-campaign wave and 93% for the post-campaign wave. Participation rate is calculated as follows: (qualified completes + over quota + terminates)/click-through).
| Completions | Baseline-campaign | Post-campaign |
|---|---|---|
| Click-Through | 2543 | 2822 |
| - Partial Completes | 82 | 173 |
| - Terminates | 438 | 244 |
| - Over quota | 21 | 365 |
| Qualified Completes | 2002 | 2040 |
| Participation Rate | 97% | 94% |
The results of this survey are not statistically projectable to the target population because the sampling method used does not ensure that the sample represents the target population with a known margin of sampling error. Reported percentages are not generalizable to any group other than the sample studied, and therefore no formal statistical inferences can be drawn between the sample results and the broader target population it may be intended to reflect. The data have been weighted to reflect the demographic composition of Canadian adults aged 18 years and over.
The table below compares the unweighted survey samples to the 2016 Census results by region, age, and gender. Overall, the sample is highly representative of the national adult population.
| Demographics | Baseline Survey | Post-campaign Survey | ||
|---|---|---|---|---|
| Unweighted Sample Size | Weighted Sample Size | Unweighted Sample Size | Weighted Sample Size | |
| Region* | ||||
| British Columbia/Yukon | 14% | 13% | 14% | 13% |
| Alberta/ Northwest Territories | 11% | 11% | 11% | 11% |
| Prairies (MB/SK)/ Nunavut | 7% | 7% | 7% | 7% |
| Ontario | 38% | 38% | 38% | 38% |
| Quebec | 23% | 23% | 23% | 23% |
| Atlantic Canada | 7% | 7% | 7% | 7% |
| Gender* | ||||
| Male | 49% | 49% | 47% | 49% |
| Female | 51% | 51% | 52% | 51% |
| Diverse | <1% | <1% | 1% | <1% |
| Age* | ||||
| 18-34 | 27% | 28% | 26% | 28% |
| 35-54 | 33% | 34% | 34% | 34% |
| 55+ | 40% | 38% | 40% | 38% |
* Denotes variables included in the weighting scheme.
Differences among the variables used in the weighting scheme are minimal. The distribution in the sample is consistent, with only slight differences observed between the unweighted percentages and the 2016 Census data.
Thank you for taking the time to complete this survey dealing with current issues of interest to Canadians. Si vous préférez répondre au sondage en français, veuillez cliquer sur français [SWITCH TO FRENCH VERSION].
Your participation is voluntary, and your responses will be kept entirely confidential. The survey takes about 7 minutes to complete.
Click here if you wish to verify the authenticity of this survey (please type in the following project code: 20210203-IP835). To view our privacy policy, click here.
If you require any technical assistance, please contact xxx
a) Does anyone in your household work for any of the following organizations?
SELECT ALL THAT APPLY
IF “NONE OF THESE ORGANIZATIONS” CONTINUE, OTHERWISE THANK AND TERMINATE.
b) Are you…
c) In what year were you born?
________
YYYY
ADMISSIBLE RANGE 1901-2004
IF > 2003, THANK AND TERMINATE
ASK D IF QUESTION C IS LEFT BLANK
d) In which of the following age categories do you belong?
SELECT ONE ONLY
IF “LESS THAN 18 YEARS OLD” OR “BLANK”, THANK AND TERMINATE
e) Where were you born?
ASK IF E=BORN OUTSIDE CANADA
f) In what year did you first move to Canada?
________
YYYY
ADMISSIBLE RANGE: 1900- 2021
g) In which province or territory do you live?
SELECT ONE ONLY
IF NO PROVINCE OR TERRITORY IS SELECTED, THANK AND TERMINATE
h) Are you an Indigenous person? (An Indigenous person is a North American Indian or a member of a First Nation, a Métis or an Inuk (Inuit). North American Indians or members of a First Nation include status, treaty or registered Indians, as well as non-status and non-registered Indians.)
i) Which of the following categories best describes your total annual household income, including income from all household members, before taxes are deducted?
SELECT ONE ONLY
j) Are you a person with a disability?
A person with a disability has a long-term or recurring physical, mental, sensory, psychiatric or learning impairment and:
Persons with disabilities are also those whose functional limitations owing to their impairment have been accommodated in their current job or workplace. Disabilities include:
ASK ALL RESPONDENTS
Q1: Over the past three weeks, have you seen, read or heard any advertising from the Government of Canada?
=> GO TO T1A
Q2: Think about the most recent ad from the Government of Canada that comes to mind. Where have you seen, read or heard this ad?
SELECT ALL THAT APPLY
Q3: What do you remember about this ad?
__________________________________________________________________
Q4: How did you know that it was an ad from the Government of Canada?
__________________________________________________________________
ASK ALL RESPONDENTS
T1A: Over the past three weeks, have you seen, read or heard any Government of Canada advertising about benefits, tax credits and/or free tax help?
=> GO TO T1D
T1A: Where have you seen, read or heard this Government of Canada ad about benefits, tax credits and/or free tax help?
SELECT ALL THAT APPLY
T1C: What do you remember about this ad?
__________________________________________________________________
ASK ALL RESPONDENTS
CAMPAIGN SPECIFIC ATTITUDINAL AND BEHAVIOURAL QUESTIONS MAY BE ADDED HERE
T1D: Are you aware that you may qualify for Government of Canada benefits, tax credits?
T2D: Are you aware that there is free tax help for eligible people to get their taxes done by volunteers at free tax clinics?
T2D: Do you feel confident to do your own taxes?
T4D: Who does your taxes?
T5D: [IF DO NOT DO MY OWN] Why do you not do your own taxes?
__________________________________________________________________
D1: Which of the following categories best describes your current employment status? Are you…?
SELECT ONE ONLY
D2: What is the highest level of formal education that you have completed?
SELECT ONE ONLY
D3: Are there any children under the age of 18 currently living in your household?
D4: What is the language you first learned at home as a child and still understand?
SELECT UP TO TWO
That concludes the survey. This survey was conducted on behalf of the Canada Revenue Agency In the coming months the report will be available from Library and Archives Canada. We thank you very much for taking the time to answer this survey, it is greatly appreciated.
Thank you for taking the time to complete this survey dealing with current issues of interest to Canadians. Si vous préférez répondre au sondage en français, veuillez cliquer sur français [SWITCH TO FRENCH VERSION].
Your participation is voluntary, and your responses will be kept entirely confidential. The survey takes about 10 minutes to complete.
Click here if you wish to verify the authenticity of this survey. To view our privacy policy, click here.
If you require any technical assistance, please contact xxxx
a) Does anyone in your household work for any of the following organizations?
SELECT ALL THAT APPLY
IF “NONE OF THESE ORGANIZATIONS” CONTINUE, OTHERWISE THANK AND TERMINATE.
b) Are you…?
c) In what year were you born?
________
YYYY
ADMISSIBLE RANGE 1900-2022
IF > 2004, THANK AND TERMINATE
ASK D IF QUESTION C IS LEFT BLANK
d) In which of the following age categories do you belong?
SELECT ONE ONLY
IF “LESS THAN 18 YEARS OLD” OR “BLANK”, THANK AND TERMINATE
e) Where were you born?
ASK IF E=BORN OUTSIDE CANADA
f) In what year did you first move to Canada?
________
YYYY
ADMISSIBLE RANGE: 1900-2022
g) In which province or territory do you live?
SELECT ONE ONLY
IF NO PROVINCE OR TERRITORY IS SELECTED, THANK AND TERMINATE
h) Are you an Indigenous person? (An Indigenous person is a North American Indian or a member of a First Nation, a Métis or an Inuk (Inuit). North American Indians or members of a First Nation include status, treaty or registered Indians, as well as non-status and non-registered Indians.)
i) Which of the following categories best describes your total annual household income, including income from all household members, before taxes are deducted?
SELECT ONE ONLY
j) Are you a person with a disability?
A person with a disability has a long-term or recurring physical, mental, sensory, psychiatric or learning impairment and:
Persons with disabilities are also those whose functional limitations owing to their impairment have been accommodated in their current job or workplace. Disabilities include:
ASK ALL RESPONDENTS
Q1: Over the past three weeks, have you seen, read or heard any advertising from the Government of Canada?
=> GO TO T1A
Q2: Think about the most recent ad from the Government of Canada that comes to mind. Where have you seen, read or heard this ad? [RANDOMIZE]
SELECT ALL THAT APPLY
Q3: What do you remember about this ad?
__________________________________________________________________
Q4: How did you know that it was an ad from the Government of Canada?
__________________________________________________________________
ASK ALL RESPONDENTS
T1A: Over the past three weeks, have you seen, read or heard any Government of Canada advertising about benefits, tax credits and/or free tax help?
=> GO TO T1D
T1B: Where have you seen, read or heard this ad about benefits, tax credits and/or free tax help? [RANDOMIZE]
SELECT ALL THAT APPLY
T1C: What do you remember about this ad?
__________________________________________________________________
ASK ALL RESPONDENTS
CAMPAIGN SPECIFIC ATTITUDINAL AND BEHAVIOURAL QUESTIONS MAY BE ADDED HERE
T1D: Are you aware that you may qualify for Government of Canada benefits and tax credits?
T2D: Are you aware that there is free tax help for eligible people to get their taxes done by volunteers at free tax clinics?
T3D: Do you feel confident to do your own taxes?
T4D: Who does your taxes?
T5D: [IF DO NOT DO MY OWN] Why do you not do your own taxes?
__________________________________________________________________
T1H: Here are some ads that have recently been broadcast on various media.
Over the past three weeks, have you seen, read or heard these ads?
=> GO TO T1J
T1I: Where have you seen, read or heard these ads? [RANDOMIZE]
SELECT ALL THAT APPLY
T1J:
What do you think is the main point these ads are trying to get across?
__________________________________________________________________
T1K:
Please indicate your level of agreement with the following statements about these ads?
RANDOMIZE STATEMENTS
| Statements | 1 Strongly Disagree |
2 | 3 | 4 | 5 Strongly Agree |
|---|---|---|---|---|---|
| These ads catch my attention | ○ | ○ | ○ | ○ | ○ |
| These ads are relevant to me | ○ | ○ | ○ | ○ | ○ |
| These ads are difficult to follow | ○ | ○ | ○ | ○ | ○ |
| These ads do not favour one political party over another | ○ | ○ | ○ | ○ | ○ |
| These ads talk about an important topic | ○ | ○ | ○ | ○ | ○ |
| These ads provide new information | ○ | ○ | ○ | ○ | ○ |
| These ads clearly convey that I may qualify for Government of Canada benefits, tax credits and/or free tax help. | ○ | ○ | ○ | ○ | ○ |
T1L: [ASK IF YES IN T1H] Which of the following actions did you take as a result of seeing/hearing this advertising? [ACCEPT MULTIPLE RESPONSES]
D1: Which of the following categories best describes your current employment status? Are you…
SELECT ONE ONLY
D2: What is the highest level of formal education that you have completed?
SELECT ONE ONLY
D3: Are there any children under the age of 18 currently living in your household?
D4: What is the language you first learned at home as a child and still understand?
SELECT UP TO TWO
That concludes the survey. This survey was conducted on behalf of The Canada Revenue Agency. In the coming months the report will be available from Library and Archives Canada. We thank you very much for taking the time to answer this survey, it is greatly appreciated.
Footnote 1. The term “vulnerable” is being used in this instance to describe individuals who may be hard to reach or those who experience certain situational or systemic challenges that can negatively impact their outcomes or put them at a disadvantage. Return to footnote 1 source paragraph
Footnote 2. According to the 2017 Canadian Survey on Disability, more than 6 million Canadians aged 15 and over (22% of the population) identify as having a disability, and it is expected actual numbers are likely higher. (https://www.canada.ca/en/employment-social-development/programs/accessible-canada.html) The CSD definition of disability includes anyone who reported being "sometimes," "often" or "always" limited in their daily activities due to a long-term condition or health problem, as well as anyone who reported being "rarely" limited if they were also unable to do certain tasks or could only do them with a lot of difficulty. Return to footnote 2 source paragraph