Prepared for the Canada Revenue Agency
Supplier name: The Strategic Counsel
Contract Number: # 46558-195330/001/CY
Contract value: $39,194.05 (including HST)
Award date: December 11, 2018
Delivery date: February 21, 2019
Registration number: POR 096-18
For more information on this report, please email media.relations@cra-arc.gc.ca.
Ce rapport est aussi disponible en français.
Climate Action Incentive Advertising Campaign: Concept Testing
Executive Summary
Prepared for the Canada Revenue Agency by The Strategic Counsel
February 2019
This public opinion research report presents the results of an online survey conducted by The Strategic Counsel on behalf of Canada Revenue Agency. The research study was conducted with 1100 Canadian residents in New Brunswick, Ontario, Manitoba and Saskatchewan in January 2019.
Cette publication est aussi disponible en français sous le titre: Test de concepts pour la campagne publicitaire sur l'Incitatif à agir pour le climat – Sommaire.
Permission to Reproduce
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from the Canada Revenue Agency. For more information on this report, please contact the Canada Revenue Agency at: media.relations@cra-arc.gc.ca
Catalogue Number:
Rv4-132/2019E-PDF
International Standard Book Number (ISBN):
978-0-660-31206-4
Related Publication (Registration Number: POR 096-18):
Catalogue number Rv4-132/2019F-PDF (Executive Summary, French)
ISBN: 978-0-660-08694-1
© Her Majesty the Queen in Right of Canada, as represented by the Minister of National Revenue, 2019
As part of the Government of Canada’s carbon pollution pricing system, the Canada Revenue Agency (CRA) is responsible for the delivery of the Climate Action Incentive (CAI) payment to residents of Saskatchewan, Manitoba, Ontario, and New Brunswick, through their respective income tax and benefit returns.
Canada Revenue Agency is launching an advertising campaign in March 2019, utilizing radio and print advertising to encourage Canadians in these four provinces to file their income tax and benefit returns to claim the CAI payment, and to drive target audiences to the web page where more information can be found. A direct mail postcard will also be sent to residents of the four provinces at the end of March.
The key objective of this research study was to test a series of advertising concepts, specifically to evaluate:
The results of this research study offer CRA and the creative agency valuable guidance with respect to the creative approach and messaging which will be most effective in encouraging residents in the four provinces to claim the climate action incentive payment at the time they complete and submit their income tax and benefit return. In particular, results for the key ad diagnostics, combined with respondents’ answers to open-ended questions regarding what they liked and disliked about the concepts, by format, and what they perceived to be the call-to-action, reveal the relative strengths and weaknesses of each concept and provide some direction in terms of improving their overall effectiveness.
The Strategic Counsel undertook a 20-minute survey online with a total of 1100 Canadians, drawn from an online panel. The fieldwork took place from January 22-30, 2019. Because the sample for this survey is based on those who opted-in to participate in the panel, no estimates of sampling error can be calculated. Therefore, the results cannot be considered to be statistically projectable to the target population in the four provinces in which the survey was undertaken.
Quotas were set to ensure the final sample reflected a 50/50 gender split overall, and in each of the four provinces, and reasonable representation across age groups (aged 18 and older). Regionally, a disproportionate sample design of n=1100 was employed to provide a minimum number of completions to analyze each of the four provinces sampled: Manitoba, Saskatchewan, Ontario and New Brunswick.
Further details on the methodology, including the response rate calculation as well as the English and French survey instruments can be found in the Appendix.
The survey was designed so that each respondent evaluated:
Following the announcement in October, 2018, there are modest levels of awareness of the CAI payment (25% have heard of it), ranging from 21% in New Brunswick, to 23% in Ontario and Manitoba, and 35% in Saskatchewan.
Eligibility is the preferred concept, of the three concepts tested, for both the print and direct mail formats by over half of all respondents (59%). Respondents reacted positively to the clear message in the design (i.e., the dollar sign) and the text that mentioned an incentive or tax rebate. Unlike the other two concepts, Future (24%) and Incentive (17%), those who preferred the Eligibility concept found it to be more eye catching and the message to be clearer and more direct.
Examples of Print Ads Tested – 3 Concepts
Eligibility
Future
Incentive
The findings were not quite as clear-cut for the two radio concepts tested, although Eligibility (57%) was the preferred concept with a 14-point margin over Future (43%). Here again, respondents’ preference for Eligibility was based on their perception that the message was clear and understandable. Notably, the use of children/children’s voices as a creative element (in the Future concept) was seen as a somewhat divisive or polarizing feature. It worked as a positive for those who preferred the Future radio concept, but was also viewed negatively, and was explicitly stated as a reason why some chose Eligibility as their overall preference.
Likeability scores for each concept, by format, were in line with respondents’ overall preference. Eligibility received the highest likeability scores for both the print (61% somewhat/strongly liked this concept) and direct mail (37% somewhat/strongly liked this concept). Likeability scores were, however, somewhat more competitive between the two radio concepts. Here again, Eligibility came out ahead, but only slightly by an 8-point margin (69% likeability for Eligibility; 61% for Future).
For the print and direct mail creative, the message to collect the CAI payment by filing your income tax and benefit return comes across reasonably clearly for many respondents regardless of the concept or the format, although it was somewhat more effective in the Eligibility and Incentive print ads compared to Future. For the latter concept, respondents were twice as likely to take away the message that pollution has a cost, rather than the key message about claiming the incentive. And, although most respondents felt that this phrase – pollution has a cost – was more meaningful and understandable than Climate Action Incentive, the combination of this phrase with the depiction of pollution in the graphic may be inadvertently overriding or diluting the call-to-action, which is to claim the incentive by filing an income tax return.
Regarding the radio advertisements, both concepts appear to relay a clear message around the Incentive. In addition, the phrase pollution has a cost is a key message takeaway for the Eligibility concept and is a phrase that respondents find quite memorable.
Each concept was tested with respect to a number of key diagnostics, including: memorability, ability to grab audience attention, comprehension/clarity, believability, relevance and clarity of the call to action (to file taxes in order to claim the Incentive). Eligibility scored reasonably well, and typically higher than the other concepts, on most of these measures, regardless of the format. All concepts (in both print and radio formats) received strong scores for clarity of the call-to-action. This was the highest score of the six diagnostic metrics for which ratings were captured. Lower ratings were usually given for memorability and relevance, suggesting that these are areas that could be amplified through the creative and the messaging in all of the concepts, including Eligibility.
Two-thirds to three-quarters of respondents said they would be somewhat or very likely to claim the CAI payment when filing their return, after being exposed to any of these concepts, in print, direct mail or on the radio. Thus, the call-to-action is strong.
Notably, however, the direct mail version of Eligibility seemed to have the strongest impact on respondents, relative to other concepts and other formats, in terms of encouraging them to visit the Government of Canada website for more information or tell a family member or friend. Almost two-thirds (62%) said they would be somewhat or very likely to take these actions after seeing the direct mail version of this concept compared to half, or fewer, who said the same for the other two concepts tested.
Across all four provinces, Eligibility was the clear ‘winning’ concept for print/direct mail. While, as noted, results were closer between the two concepts tested in radio format, respondents in Ontario, Manitoba, and Saskatchewan were more likely to favour Eligibility over Future. Only in New Brunswick were the results mixed (50% preferred Eligibility; 50% preferred Future).
It is important to note, however that respondents in Saskatchewan were less enthusiastic about any of the concepts – likeability scores for all concepts across all formats were typically lower in Saskatchewan compared to scores given by respondents in the other three provinces. At the same time, diagnostic scores for each of the concepts were not significantly different across each of the formats.
With respect to the call-to-action, respondents in New Brunswick and Ontario are somewhat more likely to visit the Government of Canada website for more information about the CAI payment, compared to those in Manitoba or Saskatchewan. Otherwise, there were no significant differences in how the public could be expected to respond to the CAI payment campaign, across the four provinces in which it applies.
Finally, with respect to the phrase pollution has a cost – while almost two thirds in Saskatchewan agreed this phrase to be both meaningful (63%) and easy to understand (69%), this was much lower level of agreement as compared to Ontario (82% and 84%, respectively).
Women and younger respondents generally tended to respond more positively to all the concepts. Typically, women gave higher likeability ratings, compared to men and also stronger rates on all six diagnostics. The same pattern held for younger respondents, aged 18 to 24, compared to those in older age brackets.
Those aged 25 to 44 were more likely to say they would visit the Government of Canada website to get more information on the Incentive and to tell a friend, specifically in response to the Eligibility ad, but it was those aged 65 and older who were most likely to say they would claim the CAI payment when filing their income tax and benefit return (and this pattern held regardless of the concept or the format). By contrast, the direct mail version of Future seemed to have most impact on those aged 18 to 24 in terms of encouraging them to file their taxes and claim the CAI payment.
The findings from this research offer some clear direction in moving forward with the Climate Action Incentive payment campaign.
Unless otherwise noted, results shown in this report are expressed as percentages and may not add up to 100% due to rounding and/or multiple responses to a given question.
MORE INFORMATION
Supplier Name: The Strategic Counsel
PWGSC Contract Number: 46558-195330/001/CY
Contract Award Date: 2018-12-11
Contract Budget: $39,194.05
To obtain more information on this study, please e-mail media.relations@cra-arc.gc.ca.
Statement of Political Neutrality
I hereby certify as a Senior Officer of The Strategic Counsel that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.
Signed:
Donna Nixon, Partner