Prepared for Canada Revenue Agency
Supplier name: Earnscliffe Strategy Group
Contract number: CW2330603
Contract value: $94,411.50
Award date: September 15, 2023
Delivery date: November 1, 2023
Registration number: POR 047-23
For more information on this report, please contact Canada Revenue Agency at:
cra-arc.media@cra-arc.gc.ca
Ce rapport est aussi disponible en français.
Prepared for Canada Revenue Agency
Supplier name: Earnscliffe Strategy Group
November 2023
This public opinion research report presents the results of focus groups conducted by Earnscliffe Strategy Group on behalf of Canada Revenue Agency. The qualitative research was conducted in October 2023.
Cette publication est aussi disponible en français sous le titre : Mise à l'essai des concepts de la campagne sur les prestations et les crédits de 2023-2024
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Canada Revenue Agency. For more information on this report, please contact Canada Revenue Agency at: cra-arc.media@cra-arc.gc.ca or at:
Canada Revenue Agency
4th Floor, 555 Mackenzie Ave.
Ottawa, Ontario K1A 0L5
Catalogue Number: Rv4-123/2024E-PDF
International Standard Book Number (ISBN): 978-0-660-70691-7
Related publications (registration number: POR 047-23)
Catalogue Number: Rv4-123/2024F-PDF (final report, French)
International Standard Book Number (ISBN): 978-0-660-70692-4
© His Majesty the King in Right of Canada, as represented by the Minister of National Revenue, 2023
Given the current mood of people living in Canada around the economy and personal financial outlook resulting from increased pressure around the rising cost of living, interest, inflationary rates, and others, the CRA would like to ensure people living in Canada are aware of benefits and credits to which they are entitled when they do their personal income taxes. These benefits and credits provide additional income or tax relief to help make life more affordable since many depend on benefit payments and tax credits for a significant part of their household income.
That being said, certain segments of the population, such as modest-income households, Indigenous Peoples, those new to Canada, people living with disabilities, and youth aged 18 to 24 years face a variety of barriers when doing their taxes and receiving their benefits and credits. These are individuals who may be hard to reach or those who experience certain situational or systemic challenges that can negatively impact their outcomes or put them at a disadvantage. To meet these challenges, it is important for the Government of Canada to make proactive and targeted efforts to reach these populations with important tax-related information. Greater awareness is required to ensure that these populations know that doing their taxes every year is necessary to be eligible for benefits and credits and that they are aware of tax literacy tools and supports like free tax clinics that are available to them.
The objective of this research was to evaluate the effectiveness of proposed concepts aimed at raising awareness of benefits and credits available to those living in Canada when they do their personal taxes. Additionally, the research was designed to produce recommendations to modify the concepts that would increase effectiveness. The CRA wanted to ensure a successful ad campaign that would capture the attention of the target audiences.
The total contract value of the multi-phased project was $94,411.50 including HST.
To meet the research objectives, Earnscliffe conducted qualitative research that involved ten online focus groups, with five target audiences, between October 14 and 17, 2023. The target audiences for the research included: Indigenous Peoples, peoples with disabilities, modest income members of the general population, modest income youth, and modest income newcomers to Canada. Modest income referred to those with a maximum annual household income of $50,000 or less. With each audience, one group was conducted in English and one in French. Between ten and 12 participants were recruited and between six and 12 participated in each group, and all groups were 90 minutes in length.
It is important to note that for the focus groups and interviews, qualitative research is a form of scientific, social, policy, and public opinion research. Focus group and interview research is not designed to help a group reach a consensus or to make decisions, but rather to elicit the full range of ideas, attitudes, experiences, and opinions of a selected sample of participants on a defined topic. Unlike the survey results, because of the small numbers involved in the qualitative component of the research, the participants cannot be expected to be thoroughly representative in a statistical sense of the larger population from which they are drawn. Findings are directional in nature, and cannot reliably be generalized beyond their number.
The key findings of this research are presented below:
Participants reviewed three creative concepts, one at a time in randomized order. For each concept, they were shown a video ad followed by an audio ad.
Overall reactions to the concepts
Silver Linings
Because You Did Your Taxes
Left Behind
Research firm: Earnscliffe Strategy Group (Earnscliffe)
Contract number: CW2330603
Contract value: $94,411.50
Contract award date: September 15, 2023
I hereby certify as a representative of Earnscliffe Strategy Group that the final deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.
Signed: Date: November 1, 2023
Stephanie Constable
Principal, Earnscliffe
Earnscliffe Strategy Group (Earnscliffe) is pleased to present this report to the Canada Revenue Agency (CRA) summarizing the results of qualitative research undertaken to test creative concepts for its benefits and credits advertising campaign.
Given the current mood of Canadians around the economy and personal financial outlook resulting from increased pressure around the rising cost of living, interest, inflationary rates, and others, the CRA would like to ensure Canadians are aware of benefits and credits to which they are entitled when doing their personal income taxes. These benefits and credits provide additional income or tax relief to help make life more affordable since many depend on benefit payments and tax credits for a significant part of their household income.
That being said, certain segments of the population, such as modest-income households, Indigenous Peoples, those new to Canada, people living with disabilities, and youth aged 18 to 24 years face a variety of barriers when doing their taxes and receiving their benefits and credits. These are individuals who may be hard to reach or those who experience certain situational or systemic challenges that can negatively impact their outcomes or put them at a disadvantage. To meet these challenges, it is important for the Government of Canada to make proactive and targeted efforts to reach these populations with important tax-related information. Greater awareness is required to ensure that these populations know that doing their taxes every year is necessary to be eligible for benefits and credits and that they are aware of tax literacy tools and supports like free tax clinics that are available to them.
The objective of this research was to evaluate the effectiveness of proposed concepts aimed at raising awareness of benefits and credits available to Canadians when they do their personal taxes. Additionally, the research was designed to produce recommendations to modify the concepts that would increase effectiveness. The CRA wanted to ensure a successful ad campaign that would capture the attention of the target audiences.
To meet the research objectives, Earnscliffe conducted qualitative research that involved ten online focus groups, with five target audiences, between October 14 and 17, 2023. The target audiences for the research included: Indigenous Peoples, peoples with disabilities, modest income members of the general population, modest income youth, and modest income newcomers to Canada. Modest income referred to those with a maximum annual household income of $50,000 or less. With each audience, one group was conducted in English and one in French. Between ten and 12 participants were recruited and between six and 12 participated in each group, and all groups were 90 minutes in length. The structure of the groups was as follows:
|
Audience |
English |
French |
Total |
|
Indigenous Peoples |
1 |
1 |
2 |
|
People with disabilities |
1 |
1 |
2 |
|
Modest income general population |
1 |
1 |
2 |
|
Modest income youth |
1 |
1 |
2 |
|
Modest income newcomers |
1 |
1 |
2 |
|
Total |
5 |
5 |
10 |
Participants were provided with a monetary incentive in recognition of their time. In the full research report, Appendix A provides greater detail on how the groups were recruited, while Appendix C provide the discussion guide used to facilitate the focus groups and Appendix B provides the screener used for recruiting the focus groups.
It is important to note that qualitative research is a form of scientific, social, policy and public opinion research. Focus group and interview research is not designed to help a group reach a consensus or to make decisions, but rather to elicit the full range of ideas, attitudes, experiences, and opinions of a selected sample of participants on a defined topic. Because of the small numbers involved the participants cannot be expected to be thoroughly representative in a statistical sense of the larger population from which they are drawn. Findings are directional in nature and cannot reliably be generalized beyond their number.
This qualitative report is divided into three sections. The first two sections explore the reactions to the creative concepts and calls to action. The third, and final section, explores the effectiveness of various communications channels and how CRA can best frame their messaging.
It is worth noting that although the groups were segregated based on cultural and ethnic identity, income and language, discussions tended to be quite similar across audiences. Unless otherwise noted, findings in this report are common across all audiences; only noteworthy differences are mentioned. Quotations used throughout the report were selected to bring the analysis to life and provide unique verbatim commentary from participants across the various audiences.
Participants reviewed three creative concepts one at a time in randomized order. For each concept, they were shown a video ad followed by an audio ad. The video ad was in the form of an animatic (an animated storyboard). Each video ad was played twice. The scripts for the audio ads were read to participants. Participants were asked to review the concepts in silence.
After each component of the concept was presented (first the animatic then the audio ad), participants were led through a group discussion to gauge their overall reactions and preferences. The three concepts are depicted below. All the content tested for each concept can be found in Appendix D.
Overall reactions to the concepts
Overall reaction to the three potential concepts for the 2023-2024 Benefits and Credits campaign can be described as lukewarm to positive.
In terms of overall strengths of the campaign, though it was not explored specifically, some participants volunteered they appreciated the message “every dollar counts” conveyed in the website address. It was felt to be succinct, memorable, relevant, and accurate particularly given the current economic situation. The website address was not only seen as relevant, but necessary. A few participants questioned if it would be clear in the audio ads where the dashes belonged in between “every dollar counts” in the web address and suggested that be made clear to listeners.
More often than not, participants seemed to appreciate the wording messages conveyed in the audio ads over those conveyed in the video ads, including many who volunteered this impression. While we will explore these differences in more detail below, in some instances, the audio ads seemed to be clearer and to the point.
While for the most part the concepts were deemed appropriate for the federal government, there was some confusion as to the main message of the concepts. Some wondered whether it was to encourage (or remind) Canadians to do their taxes annually, by offering federal benefits and tax credits. This was particularly true for those with disabilities and Indigenous Peoples. Others hoped the focus of the campaign was to point them in the right direction of federal benefits and tax credits for which they may qualify. That said, nearly all felt that the main message, when asked directly, was one of those two messages.
Most participants felt that informing Canadians about the benefits and credits available to them was important and appreciated that any of these ads would remind them it was time to do their taxes (if they aired during tax season, which participants assumed they would). That said, the messages about benefits and credits for some felt insulting as they felt they were only getting back some of their own money. A few suggested that not all people who file their taxes qualify for benefits and credits, or worse might owe money. Others noted that there are other steps required in order to apply for benefits and credits other than filing your taxes (especially for the disability tax credit). For these reasons, the messaging was misleading for some participants.
A common response for all concepts was that they would “skip” the ad, in online or audio streaming environments. To this end, participants often felt that placing the audio ads on the radio where the content cannot be skipped would be a more effective strategy.
After exposure to and discussion about all the concepts was completed, participants were asked to select the concept they preferred most. The votes were evenly split between two concepts, “Silver Linings” and “Because You Did Your Taxes,” with fewer participants indicating they preferred the concept “Left Behind.” The results of this poll have been provided in Appendix E.
Reactions to each concept will be explored in the sections that follow, including any important differences by audience type and language. Please refer to Appendix D for visual references to each concept.
Silver Linings
With respect to Silver Linings, participants described it as realistic and very relatable (that is, not wanting to wake up early or do your taxes). As a result, they felt it was credible and attention-grabbing. Youth, in particular, identified with this concept.
In terms of areas of improvement, there was a sense that it was a little too urban and metropolitan in its choice of settings and images, including visiting a coffee shop in the morning and working in an office tower. A handful of participants felt that the imagery of Priya was too “European” or “white-washed” despite her name (a non-issue in the audio version). One suggestion provided was to shine light on more than one individual to improve relatability.
Others were turned off by the negative tone and emphasis on things people do not like to do. Some did not appreciate that exercising, and even doing their taxes, were framed in this light, or particularly resonated with the 'silver linings' provided.
By this token, participants felt that the audio ad did a better job of framing what people do not like in the positive. Because the positive outcome was tied directly to the perceived negative effort, the message did not come across as negatively. For example, “Priya doesn't love exercising, but loves the way it makes her feel.”
Lastly, in terms of positive attributes, a handful of participants noted, and appreciated, that the audio script was poetic.
However, where the audio concept could be improved was in the sense that it took too long to get to the point. Many questioned whether they would have continued listening through to the end. They explained it could be an ad for just about anything and the link to doing your taxes was not obvious until much later in the ad and the transition was a little jarring (that is, sometimes even the things we don't love can have benefits).
Because You Did Your Taxes
What participants liked about Because You Did Your Taxes was the positive tone and that it conveyed the focus on federal benefits and tax credits very early in the ad. Participants felt it was informative, advising them of specific benefits and credits, and was effective in terms of conveying that extra money can help with expenses or splurges.
Having said that, many participants felt that the examples provided in the video ad, about how to use the extra money, were a little over the top. In particular, the references to creating lasting memories or new friendships. Moreover, the ways the video ad suggested the money could be spent felt like an exaggeration. Aside from Canada child benefit, most who received benefits or credits did not feel that the amount they received had a large impact on their life.
In this regard, participants felt the audio ad did a much better job of conveying how the extra money could be helpful (that is, for everyday essentials, getting repairs done, or even treating someone). These examples were deemed to be much more relevant to their current situations, and commensurate with their expectations of the anticipated benefit and credit amounts. Indeed, reaction to this audio ad was generally very positive – it was clear, to the point, informative, and relevant to them.
All that being said, some did wonder whether this ad would capture their attention if they saw or heard it. Some questioned the tone conveyed by the emotional music throughout with a preference for something more upbeat, which was viewed as a more attention-grabbing and positive approach.
Worth noting, and this only seemed to have been noted by one or two participants, was the timing of the illustration of a woman (could be an Indigenous person) at a computer and a voiceover about the disability tax credit. These participants felt it could be perceived to imply/perpetuate former stereotypes of Indigenous people (people with disabilities).
Left Behind
Reactions to the Left Behind concept were mixed. Some really liked the concept and gravitated toward the humorous approach. What worked was the upbeat music, and thought it was catchy. Some enjoyed trying to figure out the story of the video.
The suggestion that one could be leaving money behind if they do not do their taxes (and claim the benefits and credits for which they qualify) was also effective for many; though for youth in French, the saying 'laisser de l'argent sur la table' was not familiar or understood.
Many did not understand the concept (or felt that it took too long to get to the point). Others did not think the visuals and wording messages correlated in the video ad. Trying to figure out the story of the video was challenging, and some participants ended up being more distracted by it than anything. Many feared they would have skipped the ad (especially if viewing it online) right away. Several offered that they would have mistaken it for a public transit ad.
Lastly, compared to the other two concepts, this concept was perceived to provide the least amount of information. Some suggested that the running scene could be condensed, in order to provide room for more examples of the kinds of benefits or credits one might be eligible for.
The audio ad, on the other hand, was generally well received and, again, felt to be to the point and more relevant. Participants clearly understood the story and message. The only concern, tended to relate to the sound effect of low battery noise; most questioned what that would sound like or why it was placed after the mention of the word instead of before like the other sounds.
After reviewing and discussing the creative concepts, participants were asked to consider potential calls to action to accompany the campaign. To start this discussion, participants were shown a list of five potential calls to action and were asked to select their favourite. The poll results of those calls to actions and the number of participants, by audience and language, that selected each as their favourite have been provided in Appendix E.
Compared to the concepts, preferences for the potential calls to action were a little more obvious. The vast majority of participants preferred when examples of federal benefits and tax credits were cited, compared to when they were not. This is perhaps not all that surprising given the findings of the initial warm-up discussion which revealed that unaided awareness of the availability of federal benefits and tax credits was low. Participants argued it would be helpful to provide examples of federal benefits and tax credits to encourage recollection and understanding.
Asked which they preferred of five calls to action, and preferences were fairly evenly split between the following two:
“Do your taxes every year…” had wide appeal as it provided a list of potential benefits and credits, and included “and more” which made many interested in others they might be eligible for that they do not know about.
“By doing your taxes every year…” was particularly resonant because it was not a command and was felt to be more polite, respectful, and subtle in the sense that it started from a place of appreciation that most Canadians are compliant (and do their taxes) every year with a gentle reminder for those who are not.
In fact, many participants volunteered they would prefer a combination of the two by adding the citation of specific federal benefits and tax credit examples to the message, to make, “By doing your taxes every year, you may be eligible for benefits and credits, like the Canada child benefit, GST/HST credit, and more.”
Worth noting, the reference to “access” benefits and credits was also well received.
In terms of specific wording messages, participants were asked which they preferred: “do” your taxes or “file” your taxes. Most participants indicated that the way they commonly refer to the exercise is to “do” taxes over “file” taxes.
Participants were also shown a list of six benefits and credits and were asked to select the three they would prefer to see highlighted in a CRA Benefits and Credits advertising campaign. The benefits and credits selected, by audience and language, have been detailed in Appendix E.
With respect to the specific examples cited in the ads, other than the one mention of the disability tax credit in Because You Did Your Taxes, there was a sense that awareness of the Canada child benefit and the GST/HST credit was broad and expressed a desire to cite benefits and credits that were less familiar to most. Participants felt that highlighting lesser-known benefits and credits would encourage them to learn more about what they may qualify for.
The federal benefits or tax credits that participants felt should feature in an ad, therefore, were the Canada workers benefit, housing measures and the Climate action incentive payment. The GST/HST credit, the disability tax credit (especially among those with disabilities), and the Canada child benefit were selected less frequently and by a similar number of participants.
Lastly, a handful of participants were surprised that the list of benefits and credits was only six long, and expressed disappointment if these were the only ones they may qualify for.
To wrap up the discussion, participants were asked if understanding how taxes are used would encourage them to file their taxes. Almost all participants indicated they would like to learn more about how the taxes they pay are used. It was deemed important mainly for reasons of transparency, though it may encourage some to file, or at least feel better about filing, their taxes. This sentiment, and desire for transparency, was particularly acute for Indigenous Peoples. There was a sense that this may, even if only partly, explain why some Indigenous Peoples do not file their taxes now.
As a follow up, they were provided with the specific example to consider. That example read, “In addition to funding benefit and credit payments, like the Canada child benefit (CCB), the government invests taxes in important programs and services. This includes education, health care, and infrastructure.”
Once provided with this example, participants were very disappointed (if not hostile and insulted). Participants felt it was a platitude (being both obvious and vague) and certainly did not meet their hopes or desires for such information.
Participants volunteered that they had hoped to learn more about specific projects in their communities (for example, which hospital received improvements or new equipment, which schools had been renovated, which highways were improved, or bridges were built). They also explained they had hoped there may have been numbers/data shared to demonstrate the tax money collected, the investment, and impact.
To the extent that the primary objective of this research was to gauge reactions to the three advertising concepts, the results of this research suggest that none of the concepts or messages tested were considered inappropriate, culturally insensitive (aside from a couple minor refinements), or unwise for the Government of Canada to use.
The three concepts were met with neutral to generally favourable reaction. None of the concepts seemed to convey a strong call to action or were particularly compelling (with some questioning the main message or purpose of the campaign). On balance, the messaging used in the audio ads was preferred to the visual ads.
While most felt that informing Canadians of the benefits and credits that may be eligible to them is an important undertaking, some caution should be taken in the messaging surrounding how Canadians become eligible. Some participants suggested that the messaging was misleading since there are other steps required to apply (beyond filing one's return), and some who file their return may not qualify for any benefits or credits (or may actually owe money).
The Silver Linings and Because You Did Your Taxes concepts were preferred by the most, and similar number of participants. Silver Linings was considered more relatable and attention-grabbing, while Because You Did Your Taxes provided more information, quickly. The last concept, Left Behind, was preferred by fewer participants, though it could be described as the most engaging, but also confusing.
Among the calls to action proposed, participants were divided between “Do your taxes every year to get the benefits and credits you may qualify for, like the Canada child benefit, GST/HST, and more” and “By doing your taxes every year, you may be eligible for the benefits and credits.” The former due to its listing of specific benefits and credits, and the latter with its polite and gentle suggestion. Indeed, some participants volunteered that combining the two would make for the most compelling call to action. Furthermore, participants clearly preferred references to doing one's taxes over filing one's taxes.
Again, while most participants felt that a benefits and credits campaign was a worthy and important one to undertake, there was interest in being exposed to benefits and credits that may be lesser known. To that end, among the proposed benefits and credits, the Canada workers benefit, housing measures and the Climate action incentive payment were selected as the top three to highlight in such a campaign.
And finally, while there is unquestionable interest in understanding where and how tax dollars are used in Canada, when the detail provided is considered vague and obvious (as with the example provided), that effort is likely to cause more harm than good.
Earnscliffe conducted ten online focus groups, with five target audiences, between October 14 and 17, 2023. The target audiences for the research included: Indigenous Peoples, peoples with disabilities, modest income members of the general population, modest income youth, and modest income newcomers to Canada. Modest income referred to those with a maximum annual household income of $50,000 or less. With each audience, one group was conducted in English and one in French. Between ten and 12 participants were recruited and between six and 12 participated in each group, and all groups were 90 minutes in length.
|
Group |
Audience |
Language |
No of recruits |
No of participants |
Time |
|
Saturday, October 14, 2023 |
|||||
|
1 |
Modest income general population |
English |
10 |
6 |
9:00 am PT / 10:00 am MT / 11:00 am CT / 12:00 pm ET / 1:00 pm AT |
|
2 |
Modest income youth |
English |
10 |
7 |
11:00 am PT / 12:00 pm MT / 1:00 pm CT / 2:00 pm ET / 3:00 pm AT |
|
Monday, October 16, 2023 |
|||||
|
3 |
Modest income youth |
French |
10 |
9 |
5:00 pm ET / 6:00 pm AT |
|
4 |
Modest income general population |
French |
10 |
10 |
7 :00 pm ET / 8:00 pm AT |
|
5 |
Indigenous Peoples |
English |
10 |
8 |
4:00 pm PT / 5:00 pm MT / 6:00 pm CT / 7:00 pm ET / 8:00 pm AT |
|
Tuesday, October 17, 2023 |
|||||
|
6 |
Peoples with disabilities |
French |
10 |
6 |
1:00 pm ET / 2:00 pm AT |
|
7 |
Peoples with disabilities |
English |
10 |
10 |
10:00 am PT / 11:00 am MT / 12:00 pm CT / 1:00 pm ET / 2:00 pm AT |
|
8 |
Modest income newcomers |
French |
10 |
7 |
5:00 pm ET / 6:00 pm AT |
|
9 |
Indigenous Peoples |
French |
10 |
9 |
7:00 pm ET / 8:00 pm AT |
|
10 |
Modest income newcomers |
English |
12 |
12 |
4:00 pm PT / 5:00 pm MT / 6:00 pm CT / 7:00 pm ET / 8:00 pm AT |
Participants were recruited using a five-minute screening questionnaire (included in Appendix B).
Our fieldwork subcontractor, Quality Response, relied on panels and databases of Canadians. This is the approach employed most often. Quality Response reaches out to members of their database first via email and follows-up with telephone calls to pre-qualify respondents.
Quality Response's database includes approximately 35,000 Canadians with profiling on a range of attributes including standard personal demographics, household composition, medical background, technology usage, financial services, health and wellness, business profiles, and other relevant criteria. Their database is constantly being updated and replenished and operates out of their own, onsite telephone room in Toronto, Ontario. Potential group participants are recruited to their database via mixed-mode: following a proprietary telephone survey, online, referral, social media and print advertising. Initial contact is often made via email or online pre-screening for speed and economies, followed up by personal telephone recruitment and pre-group attendance confirmation.
Quality Response understands the nuances of qualitative recruiting and the importance of locating qualified, interested respondents. Their recruiting is undertaken in strict accordance with the Standards for the Conduct of Government of Canada Public Opinion Research – Qualitative Research.
Reminder calls were made prior to the groups to confirm participants' intention to attend and to encourage higher rates of participation. As well, all participants received a cash honorarium at the end of the group discussion. The honorarium equaled $100 for modest income members of the general population, modest income youth, and modest income newcomers to Canada; $125 for people living with disabilities; and $150 for Indigenous persons. The variance in honorarium is equal to the variance in difficulty in recruiting each audience (often but not always driven by the incidence or each group within the population).
A total of ten participants were recruited for each group. All participants agreed to the presence of observers and recording of the session during the screening process and at the beginning of the session (for those who attended).
We relied on two qualified moderators. Given the timeline for the project, using two moderators allowed us to conduct all of the focus groups over the course of one week (three nights).
Both moderators attended the kick-off night of focus groups. This ensured that both were aware of the flow of the focus groups and were involved in any conversation about potential changes to the discussion guide or flow of conversation for each subsequent night.
In our experience, there is value in using multiple moderators (within reason) as it ensures that no single moderator develops early conclusions. Each moderator takes notes and summarizes their groups after each night. The moderators each provide a debrief on their groups including the functionality of the discussion guide; any issues relating to recruiting, turnout, or technology; and key findings including noting instances where they were unique and where they were similar to previous sessions. Together, they discuss the findings both on an ongoing basis in order to allow for probing of areas that require further investigation in subsequent groups, and before the final results are reported.
It is important to note, when reading the qualitative findings, that qualitative research is a form of scientific, social, policy, and public opinion research. Focus group or interview research is designed to elicit the full range of ideas, attitudes, experiences, and opinions of a selected sample of participants on a defined topic. Because of the small numbers involved, the participants cannot be expected to be thoroughly representative in a statistical sense of the larger population from which they are drawn, and findings cannot reliably be generalized beyond their number.
The following is a glossary of terms used throughout the report to impart the qualitative findings. These phrases are used when groups of participants share a specific point of view. Unless otherwise stated, it should not be taken to mean that the rest of participants disagreed with the point; rather others either did not comment or did not have a strong opinion on the question.
|
Generalization |
Interpretation |
|---|---|
|
Few |
Few is used when less than 10% of participants have responded with similar answers. |
|
Several |
Several is used when fewer than 20% of the participants responded with similar answers. |
|
Some |
Some is used when more than 20% but significantly fewer than 50% of participants with similar answers. |
|
Many |
Many is used when nearly 50% of participants responded with similar answers. |
|
Majority/Plurality |
Majority or plurality are used when more than 50% but fewer than 75% of the participants responded with similar answers. |
|
Most |
Most is used when more than 75% of the participants responded with similar answers. |
|
Vast majority |
Vast majority is used when nearly all participants responded with similar answers, but several had differing views. |
|
Unanimous/Almost all |
Unanimous or almost all are used when all participants gave similar answers or when the vast majority of participants gave similar answers and the remaining few declined to comment on the issue in question. |
|
Group # (moderator) |
Date |
Language |
Time |
|---|---|---|---|
|
Saturday, October 14th, 2023 |
|||
|
1 (SC) |
Modest income general population |
English |
9:00 am PT / 10:00 am MT / 11:00 am CT / 12:00 pm ET / 1:00 pm AT |
|
2 (DA) |
Modest income youth |
English |
11:00 am PT / 12:00 pm MT / 1:00 pm CT / 2:00 pm ET / 3:00 pm AT |
|
Monday, October 16th, 2023 |
|||
|
3 (SC) |
Modest income youth |
French |
5:00 pm ET |
|
4 (SC) |
Modest income general population |
French |
7:00 pm ET |
|
5 (DA) |
Indigenous Peoples |
English |
4:00 pm PT / 5:00 pm MT / 6:00 pm CT / 7:00 pm ET / 8:00 pm AT |
|
Tuesday, October 17th, 2023 |
|||
|
6 (SC) |
Peoples with disabilities |
French |
1:00 pm ET |
|
7 (DA) |
Peoples with disabilities |
English |
10:00 am PT / 11:00 am MT / 12:00 pm CT / 1:00 pm ET / 2:00 pm AT |
|
8 (SC) |
Modest income newcomers |
French |
5:00 pm ET |
|
9 (SC) |
Indigenous Peoples |
French |
7:00 pm ET |
|
10 (DA) |
Modest income newcomers |
English |
4:00 pm PT / 5:00 pm MT / 6:00 pm CT / 7:00 pm ET / 8:00 pm AT |
Respondent's name: Interviewer:
Respondent's phone number: (cell) Date:
Respondent's email: Validated:
On quotas:
Hello, this is _______________ calling on behalf of Earnscliffe, a national public opinion research firm. We are organizing a series of online discussions on important issues on behalf of the Government of Canada. Discussion participants will receive compensation for their time. May I continue?
Yes Continue
No Thank and terminate
Recruiter note: For English groups, if participant would prefer to continue in French, please respond with: “Nous recherchons des gens qui parlent anglais pour participer à ce groupe de discussion. Êtes-vous intéressé à participer à un autre groupe de discussion qui se tiendra en français? Si oui, quelqu'un fera le suivi avec vous dans les prochains jours.
Puis-je avoir votre nom afin que quelqu'un communique avec vous ? [Record name]
J'aimerais confirmer votre numéro de téléphone : [Confirm phone number]
Participation is voluntary. We are interested in hearing your opinions; no attempt will be made to sell you anything or change your point of view. The format will be either a 'round table' discussion or a telephone interview led by a research professional. All opinions expressed will remain anonymous and views will be grouped together to ensure no particular individual can be identified. The information is being collected under the authority of the Privacy Act and other applicable privacy laws. I would like to ask you a few questions to see if you or someone in your household qualify to participate. This will take about five minutes. May I continue?
Yes Continue
No Thank and terminate
Monitoring text:
Read to all: “This call may be monitored or audio taped for quality control and evaluation purposes.
Additional clarification if needed:
To ensure that I (the interviewer) am reading the questions correctly and collecting your answers accurately;
To assess my (the interviewer) work for performance evaluation;
To ensure that the questionnaire is accurate/correct (i.e. evaluation of CATI programming and methodology – we're asking the right questions to meet our clients' research requirements – kind of like pre-testing)
If the call is audio taped, it is only for the purposes of playback to the interviewer for a performance evaluation immediately after the interview is conducted or it can be used by the Project Manager/client to evaluate the questionnaire if they are unavailable at the time of the interview – all audio tapes are destroyed after the evaluation.
1. Do you or does anyone in your immediate family or household work in or has retired from any of the following areas?
|
Yes |
No |
|
|---|---|---|
|
A marketing or market research firm |
1 |
2 |
|
A magazine or newspaper, online or print |
1 |
2 |
|
A radio or television station |
1 |
2 |
|
A public relations company |
1 |
2 |
|
An advertising agency or graphic design firm |
1 |
2 |
|
An online media company or as a blog writer |
1 |
2 |
|
The government, whether federal, provincial, or municipal |
1 |
2 |
|
A political party |
1 |
2 |
|
Accounting or tax preparation |
1 |
2 |
|
An organization that provides services to immigrants or refugees |
1 |
2 |
If “yes” to any of the above, thank and terminate.
2. In which province or territory do you live?
3. How would you describe the area in which you live? [Ensure good mix]
4. What language do you speak most often at home?
5. Which of the following age categories do you fall in to? Are you…? [Ensure good mix]
For Groups 2 and 3 with youth, must be 18-24 years of age.
6. Are you…? [Ensure good mix]
7. Which of the following categories best describes your total household income in 2022; that is, the total income of all persons in your household combined, before taxes? [Read list] [Ensure good mix]
We only ask this question to ensure a mix of participants based on their income level. Your response and identity remain confidential. If still “Don't know/Prefer not to say”, thank and terminate.
For groups with modest income earners (Groups 1, 2, 3, 4, 8, 10, total household income must be under $50,000.00.
8. Are you an Indigenous person, that is, First Nations (Status or non-Status), Métis, or Inuk (Inuit)?
9. Are you First Nations, Métis, or Inuk (Inuit)? [Ensure good mix]
10. [Only First Nations] Do you live… [Ensure good mix]
11. [If not Indigenous at Q8] To make sure that we speak to a diversity of people, could you please tell me what is your ethnic background? [Do not read, ensure good mix]
12. Were you born in Canada?
13. Have you lived in Canada for…?
If not born in Canada (Q12) and have lived in Canada for 5 years or less (Q13), qualifies as newcomer.
14. Do you identify as a person with a disability? A person with a disability is a person who has a severe and prolonged condition that inhibits a person from performing normal and routine daily activities. These include both mental and physical disabilities.
If “yes”, qualifies for groups with people with disabilities (Groups 6 and 7).
15. Which of the following categories best describes the situation of the people living in your household? Please note that “a couple” refers to either a married or a “common-law” couple. Are the people living in your household…? [Ensure good mix]
16. What is your current employment status? [Ensure good mix]
17. What is the highest level of schooling that you have completed? [Ensure good mix]
We only ask this question to ensure a mix of participants based on their education level. Your response and identity remain confidential. If still “Don't know/Prefer not to say”, thank and terminate.
18. Have you participated in a discussion or focus group before? A discussion group brings together a few people in order to find out their opinion about a given subject.
19. When was the last time you attended a discussion or focus group?
20. How many of these sessions have you attended in the last five years?
This research will require participating in a video call online over Zoom. You may join from a multitude of devices but we prefer you join from a desktop computer, laptop, or tablet (rather than your cell phone). You will need to be visible on camera.
21. Do you have access to a desktop computer, laptop or tablet with high-speed internet which will allow you to participate in an online discussion group?
22. [If yes] Does your desktop computer/laptop/tablet have a camera that will allow you to be visible to the moderator and other participants as part of an online discussion group?
23. [If yes] Do you have a personal email address that is currently active and available to you?
Invitation
24. Participants in discussion groups and interviews are asked to voice their opinions and thoughts. How comfortable are you in voicing your opinions in front of others? Are you…? (Read list)
25. Sometimes participants are asked to read text, review images, or type out answers during the discussion. Is there any reason why you could not participate?
26. Is there anything we could do to ensure that you can participate?
27. What specifically? [Open end]
Interviewer to note for potential one-on-one interview
28. Based on your responses, it looks like you fit the criteria we are looking for.
I would like to invite you to participate in a small group discussion, called an online focus group, we are conducting at [time], on [date].
As you may know, focus groups are used to gather information on a particular subject matter. The discussion will consist of 6 to 8 people and will be very informal.
It will last up to 90 minutes and you will receive $[insert amount] as a thank you for your time. Would you be willing to attend?
Privacy questions
Now I have a few questions that relate to privacy, your personal information and the research process. We will need your consent on a few issues that enable us to conduct our research. As I run through these questions, please feel free to ask me any questions you would like clarified.
P1) First, we will be providing a list of respondents' first names and profiles (screener responses) to the moderator so that they can sign you into the group. Do we have your permission to do this? I assure you it will be kept strictly confidential.
We need to provide the first names and background of the people attending the focus group because only the individuals invited are allowed in the session and this information is necessary for verification purposes. Please be assured that this information will be kept strictly confidential. Go to P1a
P1a) Now that I've explained this, do I have your permission to provide your first name and profile?
P2) A recording of the group session will be produced for research purposes. The recordings will be used by the research professional to assist in preparing a report on the research findings and may be used by the Government of Canada for internal reporting purposes.
Do you agree to be recorded for research and reporting purposes only?
It is necessary for the research process for us to record the session as the researchers need this material to complete the report.
P2a) Now that I've explained this, do I have your permission for recording?
P3) Employees from the Government of Canada may also be online to observe the groups.
Do you agree to be observed by Government of Canada employees?
P3a) It is standard qualitative procedure to invite clients, in this case, Government of Canada employees, to observe the groups online. They will be there simply to hear your opinions firsthand although they may take their own notes and confer with the moderator on occasion to discuss whether there are any additional questions to ask the group.
Do you agree to be observed by Government of Canada employees?
Invitation:
Wonderful, you qualify to participate in one of our discussion sessions.
|
Group # (moderator) |
Date |
Language |
Time |
|---|---|---|---|
|
Saturday, October 14th, 2023 |
|||
|
1 (SC) |
Modest income general population |
English |
9:00 am PT / 10:00 am MT / 11:00 am CT / 12:00 pm ET / 1:00 pm AT |
|
2 (DA) |
Modest income youth |
English |
11:00 am PT / 12:00 pm MT / 1:00 pm CT / 2:00 pm ET / 3:00 pm AT |
|
Monday, October 16th, 2023 |
|||
|
3 (SC) |
Modest income youth |
French |
5:00 pm ET |
|
4 (SC) |
Modest income general population |
French |
7:00 pm ET |
|
5 (DA) |
Indigenous Peoples |
English |
4:00 pm PT / 5:00 pm MT / 6:00 pm CT / 7:00 pm ET / 8:00 pm AT |
|
Tuesday, October 17th, 2023 |
|||
|
6 (SC) |
Peoples with disabilities |
French |
1:00 pm ET |
|
7 (DA) |
Peoples with disabilities |
English |
10:00 am PT / 11:00 am MT / 12:00 pm CT / 1:00 pm ET / 2:00 pm AT |
|
8 (SC) |
Modest income newcomers |
French |
5:00 pm ET |
|
9 (SC) |
Indigenous Peoples |
French |
7:00 pm ET |
|
10 (DA) |
Modest income newcomers |
English |
4:00 pm PT / 5:00 pm MT / 6:00 pm CT / 7:00 pm ET / 8:00 pm AT |
Can I confirm your email address so that we can send you the link to the online discussion group? We ask that you login a few minutes early to be sure you are able to connect and to test your sound (speaker and microphone). If you require glasses for reading, please make sure you have them handy as well.
As we are only inviting a small number of people, your participation is very important to us. If for some reason you are unable to attend, please call us so that we may get someone to replace you. You can reach us at [insert phone number] at our office. Please ask for [name]. Someone will call you in the days leading up to the discussion to remind you.
So that we can call you to remind you about the discussion group/interview or contact you should there be any changes, can you please confirm your name and contact information for me?
First name
Last Name
email
Daytime phone number
Evening phone number
If the respondent refuses to give his/her first or last name, email or phone number please assure them that this information will be kept strictly confidential in accordance with the privacy law and that it is used strictly to contact them to confirm their attendance and to inform them of any changes to the discussion group. If they still refuse, thank and terminate.
Thank you for your interest in our study. We look forward to hearing your thoughts and opinions!
|
Group agenda |
Duration |
|---|---|
|
Introduction |
10 minutes |
|
Warm-up: Experience with taxes |
10 minutes |
|
Concept evaluation |
55 minutes |
|
Communications |
10 minutes |
|
Conclusion |
5 minutes |
|
Total |
90 minutes |
[Moderator will go around the “table” and ask participants to introduce themselves.]
To begin, I would like to start off by understanding your initial thoughts on the topic of our discussion tonight, federal benefits and tax credits.
In the coming months, the Canada Revenue Agency is launching a campaign aimed at raising awareness of benefits and credits that people may be eligible for when they do their taxes.
The purpose of the remainder of our discussion today/tonight is to show you a series of draft creative concepts, which refers to visuals and wording messages, to gauge your reactions.
We have three different concepts to share with you each with a variety of different components including content for audio and video ads. We will go through each concept one at a time and I will ask you to review them in silence and then we will discuss reactions as a group.
Before we begin, I wanted to mention that as these concepts are still in the draft stage. I want you to use your imagination to consider them as final products.
Concept testing
To begin, I would like to show you three concepts that are currently being developed. For each concept, there are a number of communications components including storyboards that will be used to develop a video ad, as well as mock ups for web banner ads, out-of-home, and print ads, etc. Again, these concepts are rough versions of ads and you will need to use your imagination to imagine them as fully developed and executed ads.
[Moderator to randomize order of presentation of the three concepts. Moderator to lead a conversation about each with the following prompts.]
Audio scripts
Now I would like to share the audio scripts. These concepts are rough versions of ads that you might hear on the local radio or digital audio, like Spotify. I would like to get your thoughts on the ideas underlying each concept.
[Moderator to repeat order of presentation of the three concepts (in the same order as previous). Moderator to lead a conversation about each with the following prompts.]
Concept wrap-up:
Now that we have reviewed each of the three concepts, I'd like to understand your overall preferences.
Call to action
Now I would like to review the specific calls to action. We have already seen a few of these in the ads. Just a reminder each one of these call to actions will be accompanied by the website to get more information.
Currently in the scripts:
Generic:
[Moderator to lead a conversation about each with the following prompts.]
[Moderator to request additional questions are sent via the chat box directly to the moderator and probe on any additional areas of interest]
Music:
Emotional music throughout.
Announcer:
When you do your taxes every year, you may qualify for benefits and credits.
Announcer:
And whether that means paying for everyday essentials…
Announcer:
…getting some repairs done to make things easier…
Announcer:
…or buying something special to brighten someone's day…
Announcer:
…a little extra can mean a lot.
Announcer:
Do your taxes every year to get the benefits and credits you may qualify for, like the Canada child benefit, GST/HST credit, and more.
Announcer:
Learn more at canada.ca/every-dollar-counts.
Announcer:
A message from the Government of Canada.
Sound effect:
Government of Canada musical signature.
Announcer:
Priya doesn't love exercising…
Announcer:
…but loves the way it makes her feel.
Announcer:
And she dislikes having to cook…
Announcer:
…but loves the taste of a home-cooked meal.
Announcer:
Sometimes, even the things we don't love can have benefits.
Announcer:
Like doing your taxes every year to get the benefits and credits you may qualify for, including the Canada child benefit, GST/HST credit, and more.
Announcer:
Learn more at canada.ca/every-dollar-counts.
Announcer:
A message from the Government of Canada.
Sound effect:
Government of Canada musical signature.
Announcer:
It's no fun leaving things behind…
Sound effect:
Pouring rain.
Announcer:
Like that time you forgot your umbrella on a rainy day.
Sound effect:
Quiet room.
Announcer:
Or when you left your charger at home.
Sound effect:
Low battery noise.
Announcer:
Or worse yet, leaving money behind.
Announcer:
That's why it's important to do your taxes every year to get the benefits and credits you may qualify for, like the Canada child benefit, GST/HST credit, and more.
Announcer:
Learn more at canada.ca/every-dollar-counts.
Announcer:
A message from the Government of Canada.
Sound effect:
Government of Canada musical signature.
|
Concept |
Total |
Indigenous Peoples |
Peoples with disabilities |
Modest income general population |
Modest income youth |
Modest income newcomers |
|
Silver Linings |
33 |
10 |
4 |
1 |
12 |
6 |
|
Because You Did Your Taxes |
31 |
6 |
6 |
10 |
0 |
9 |
|
Left Behind |
20 |
1 |
6 |
5 |
4 |
4 |
|
Concept |
Total |
English |
French |
|
Silver Linings |
33 |
18 |
15 |
|
Because You Did Your Taxes |
31 |
11 |
20 |
|
Left Behind |
20 |
14 |
6 |
|
Calls to action |
Total |
Indigenous Peoples |
Peoples with disabilities |
Modest income general population |
Modest income youth |
Modest income newcomers |
|
Do your taxes every year to get the benefits and credits you may qualify for, like the Canada Child Benefit, GST/HST, and more. |
20 |
1 |
7 |
1 |
11 |
3 |
|
By doing your taxes every year, you may be eligible for the benefits and credits. |
19 |
2 |
3 |
13 |
1 |
5 |
|
File your taxes every year to access the benefits and credits that you may qualify for. |
7 |
3 |
4 |
0 |
0 |
4 |
|
Do your taxes every year to get the benefits and credits you qualify for. |
6 |
0 |
0 |
3 |
3 |
0 |
|
File your taxes every year to get the benefits and credits you may qualify for. |
1 |
0 |
0 |
0 |
1 |
0 |
|
Calls to action |
Total |
English |
French |
|
Do your taxes every year to get the benefits and credits you may qualify for, like the Canada Child Benefit, GST/HST, and more. |
20 |
15 |
8 |
|
By doing your taxes every year, you may be eligible for the benefits and credits. |
19 |
11 |
13 |
|
File your taxes every year to access the benefits and credits that you may qualify for. |
7 |
10 |
1 |
|
Do your taxes every year to get the benefits and credits you qualify for. |
6 |
4 |
2 |
|
File your taxes every year to get the benefits and credits you may qualify for. |
1 |
0 |
1 |
Note that the participants in the modest income youth English group were only able to select their top (one) choice of preferred benefits and credits.
|
Concept |
Total |
Indigenous Peoples |
Peoples with disabilities |
Modest income general population |
Modest income youth |
Modest income newcomers |
|
The Canada workers benefit |
57 |
10 |
9 |
15 |
9 |
14 |
|
Housing measures |
45 |
14 |
3 |
10 |
9 |
9 |
|
The Climate action incentive payment |
35 |
11 |
7 |
5 |
5 |
7 |
|
The GST/HST credit |
32 |
4 |
7 |
8 |
5 |
8 |
|
The disability tax credit |
31 |
5 |
13 |
5 |
3 |
5 |
|
The Canada child benefit |
30 |
5 |
6 |
4 |
3 |
12 |
|
Concept |
Total |
English |
French |
|
The Canada workers benefit |
57 |
26 |
31 |
|
Housing measures |
45 |
20 |
25 |
|
The Climate action incentive payment |
35 |
21 |
14 |
|
The GST/HST credit |
32 |
17 |
15 |
|
The disability tax credit |
31 |
14 |
17 |
|
The Canada child benefit |
30 |
14 |
16 |