Prepared for the Canada Revenue Agency
Supplier name: Quorus Consulting Group Inc.
Contract Number: #46558-201138/001/CY
Contract value: $65,779.16 (including HST)
Award date: 2020-01-24
Delivery date: 2020-03
Registration number: POR 078-19
For more information on this report, please contact the CRA at: cra-arc.media@cra-arc.gc.ca.
Ce rapport est aussi disponible en français.
Digital Services Enhancements
Final Report
Prepared for Canada Revenue Agency
Supplier name: Quorus Consulting Group Inc.
March 2020
This public opinion research report presents the results of a series of focus groups conducted by Quorus Consulting Group Inc. on behalf of the Canada Revenue Agency. A total of 8 focus groups were conducted between February 24 and the 27, 2020 in four different cities across Canada.
Cette publication est aussi disponible en français sous le titre : Amélioration des services numériques.
Permission to Reproduce
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from the Canada Revenue Agency. For more information on this report, please contact the Canada Revenue Agency at: cra-arc.media@cra-arc.gc.ca.
Catalogue Number: Rv4-142/2020E-PDF
International Standard Book Number (ISBN): 978-0-660-35765-2
Related Publication (Registration Number: POR 078-19)
Catalogue number: Rv4-142/2020F-PDF (rapport final – français
ISBN: 978-0-660-35766-9
© Her Majesty the Queen in Right of Canada, as represented by the Minister of National Revenue, 2020
Political Neutrality Certification
I hereby certify as Senior Officer of Quorus Consulting Group Inc. that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Policy on Communications and Federal Identity and the Directive on the Management of Communications - Appendix C.
Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.
Signed: Rick Nadeau, President, Quorus Consulting Group Inc.
Table of contents
In September 2018, the Canada Revenue Agency (CRA) conducted focus groups for the Digital Services Strategy, and this current research is intended to build on those insights, in addition to gathering detailed input from taxpayers (clients) and clearly identify the areas of improvement for digital services, in order to improve the digital service experience.
Gathering this insight supports the Government of Canada priority of improving services in general, and specifically improving digital services, for Canadians.
It is crucial to consult with Canadians to ensure that CRA delivers services that meet Canadians’ evolving needs and expectations. Specifically, this research focused on the following service areas:
This report is based on eight focus groups that Quorus completed between February 24 and February 27, 2020. Two focus groups were held in each of the following locations: St. John’s (NL), Toronto, Montreal, and Vancouver. All sessions were in English except for Montreal, which were in French. All sessions were conducted with benefit recipients (HST/GST credit, and/ or CCB) and a mix of My Account user and non-users was achieved. Efforts were made to ensure that some participants in each group had contacted the CRA within the past two years for a benefit-related matter, or, if not for a benefit-related matter, had contacted them for another issue. In total, 70 individuals participated in these focus groups. Each session lasted two hours, participants received a $100 honorarium and the recruitment process sought a good representation of My Account Users and Non-User, gender, age and income. All focus groups were moderated by Rick Nadeau, one of Quorus’ bilingual senior researchers on the Government of Canada Standing Offer.
Qualitative Research Disclaimer
Qualitative research seeks to develop insight and direction rather than quantitatively projectable measures. The purpose is not to generate “statistics” but to hear the full range of opinions on a topic, understand the language participants use, gauge degrees of passion and engagement and to leverage the power of the group to inspire ideas. Participants are encouraged to voice their opinions, irrespective of whether or not that view is shared by others.
Due to the sample size, the special recruitment methods used, and the study objectives themselves, it is clearly understood that the work under discussion is exploratory in nature. The findings are not, nor were they intended to be, projectable to a larger population.
Specifically, it is inappropriate to suggest or to infer that few (or many) real world users would behave in one way simply because few (or many) participants behaved in this way during the sessions. This kind of projection is strictly the prerogative of quantitative research.
Participants were most likely to talk about the online and mobile banking tools they use to help them stay on top of their finances. They enjoy the convenience these tools bring, especially in terms of being able to easily pay bills and check balances and how they can do these tasks any time they want. Not having to go to brick and mortar locations was also a meaningful advantage of using these tools.
By and large participants were contacting CRA for file-specific questions and issues. In terms of how they contacted CRA, the overarching themes included:
Very few participants claimed to have found the information they were looking for online and then followed up with a phone call to CRA. If they did, it was to:
When asked how the CRA could improve its online resources and services, especially when it comes to helping citizens self-serve, participants recommended:
Those who had contacted the CRA by telephone for a benefit-related matter were most likely to do so for the following reasons:
Participants, especially those using My Account, explained they did not need to contact CRA regarding payment amounts and dates since that information was easily found in their portal.
For the most part, My Account users were quite pleased with their portal. They considered it easy to navigate and the information easy to understand. By and large, these participants did not have any suggestions to improve the portal.
The main issue or challenge related to My Account came from participants who had had issues remembering their password, or others who had given up with the many steps they felt they needed to do to get a new password. On a related matter, those who were using a Sign-in Partner (mostly their financial institution) were quite pleased with that feature and admitted that it saved them from having to remember one more password, especially one that is so rarely used.
Very few participants were aware of CRA’s mobile digital applications but at first mention, interest in an app was high, mainly for the following reasons:
When it came time to discuss notifications and reminders, many participants had already explained that they would like to receive notifications (unaware that the service is available) or explained that they were already receiving notifications by email from CRA.
CRA’s email notification and reminder service was explained to all participants, at which point many of those already receiving these emails expressed strong satisfaction with the service, whereas those not receiving them showed interest. When prompted to suggest situations in which they would like to receive a notification, participants seemed interested in the following:
A few participants suggested that notification registrants should be given the option to pre-select the types of notifications they would receive from a long list of options.
If they were to receive notifications from CRA, most would prefer to receive them via email, although there was some interest in text messages as well. Irrespective of the approach, participants voiced strong concerns with fraudulent emails and texts which constantly leave them questioning the legitimacy of any correspondence they receive claiming to be from the CRA.
Given the choice, participants would opt for minimal personal information in the notifications they would receive from CRA. At most, participants would be open to the notifications giving some indication of the general nature of the notification.
Participants are very aware of live chat options on many of the websites they visit. For the most part, participants seemed to prefer this option over calling the various companies. There was also some awareness of chatbots, although actual familiarity with what they were and how they worked seemed low. Opinions regarding chatbots were negative and fairly strong.
Ultimately, nearly all participants agreed that the CRA should offer an online communication channel for enquiries or assistance of one kind or another. Interest was strongest for a live chat option. The next best thing would be some sort of “callback option.”
Many participants actively suggested that some sort of online communication channel should be housed within My Account believing that this would ensure that they are corresponding directly with CRA and also allow them to securely ask file-specific questions.
Participants did seem open to the idea of perhaps using chatbots for general website navigation purposes or for general queries. However, the moment that a file-specific issue needed clarification, participants were not at all confident that a chatbot would suffice. When specifically prompted about emailing with CRA, participants did not feel comfortable doing this outside of My Account.
In terms of service delivery, participants would expect a “live chat” to be instant communication. Expectations were noticeably different for emails, where participants felt that a 24 to 48-hour turnaround time would be more acceptable. At a minimum they would expect a confirmation of receipt of the email shortly after sending in their initial note to CRA.
Participants were presented with a list of six possible types of communications they could receive in a secure CRA message centre. The most popular, or most valued, would be “Mail from the CRA such as an annual Notice of Assessment or an annual Benefits Statement.” Participants believe this is the main reason they would want to visit My Account.
Participants were then presented with a list of nine possible functionalities they could have in a secure CRA message centre. The two most valued would be to “Reply directly to an online message received from the CRA” or “Send a message to a CRA employee working on your file.” Ultimately, participants felt any feature that would allow them to communicate with someone at CRA would be of utmost value, especially if it allows them to ask file-specific questions.
Examples of other suggestions for the message centre included:
Having now obtained some idea of the types of communications and functionalities that a CRA message centre could offer, nearly all participants agreed that a message centre should be developed and that it be housed in My Account.
Supplier Name: Quorus Consulting Group Inc.
Contract number: 46558-201138/001/CY
Contract Award Date: January 24, 2020
Contract Amount (including HST): $65,779.16
For more information, please contact the Canada Revenue Agency at: cra-arc.media@cra-arc.gc.ca
In September 2018, the CRA conducted focus groups for the Digital Services Strategy, and this current research is intended to build on those insights, in addition to gathering detailed input from taxpayers (clients) and clearly identify the areas of improvement for digital services, in order to improve the digital service experience. This includes gathering information on service expectations to ensure the CRA continues to design and deliver digital services that meets its clients’ needs and expectations.
The Agency frequently receives telephone enquiries from clients, including benefit recipients, about information that is available online through CRA’s online portals. According to previous research, about 20% of Canadians have contacted the CRA in the past 12 months. Furthermore, of the 20 million tax filers, 40% are benefit recipients and do not have My Account, and would therefore need to contact CRA via telephone. These enquiries include, for instance, confirmation of information about payment dates and payment amounts, which are currently available through their online portals, in addition to other types of information.
The information gathered will be used to establish processes and projects that improve service delivery and product design/scope to help improve the CRA’s interactions with benefit recipients. Gathering this insight supports the Government of Canada priority of improving services in general, and specifically improving digital services, for Canadians.
This research is crucial to ensure that CRA delivers services that meet Canadians’ evolving needs and expectations. The findings from this research are also needed to validate the information gathered through various client feedback mechanisms available to clients (telephone, online portals).
Specifically, this research focused on the following areas to clarify taxpayer expectations:
By asking participants to explain how they use various websites and mobile apps to manage their personal and financial affairs, we were able to understand the backdrop against which tools like My Account are used and maybe even compared.
Participants were most likely to talk about the online and mobile banking tools they use to help them stay on top of their finances. They enjoy the convenience these tools bring, especially in terms of being able to easily pay bills and check balances and how they can do these tasks any time they want. A few also liked that they can take a picture of a cheque to deposit it. Some of the younger participants indicated using their mobile apps exclusively or nearly all the time to do these tasks rather than logging onto a computer. Conversely, some participants explained that they don’t trust a cell phone to manage their financial affairs in case they lose their phone, it is stolen or it is hacked.
Not having to go to brick and mortar locations was also a meaningful advantage of using these tools. One mother explained that not needing to bring her young children to a branch to do her banking was a significant benefit.
Apps and secure websites from other types of companies and institutions were mentioned as well, including those from cell phone companies (to monitor their data consumption for instance), cable companies and insurance companies. There were some references to interacting with government through secure websites although these tended to be much less frequent or common compared to those used to interact with banks and utilities. In this context, My Account was one of the more common online resources mentioned.
Having targeted individuals who had contacted CRA at least once over the past few years, it was to be expected that participants had some feedback on this topic. When asked which channels of communication were used over this period, most participants mentioned they had used the CRA website. Many participants have used both the website and called to interact with the CRA, while fewer use calling exclusively. Both channels were used to mixed degrees of success but were the preferred means of contact for a variety of reasons. Those having obtained information about the CRA via social media were extremely rare.
When asked how often they used these communication channels, most participants felt that contact was relatively infrequent and is more often than not on an as-needed basis. Participants explained they will contact CRA when an issue arises, or when they receive a piece of mail or an email notification from CRA, and they are most likely to contact during the tax season.
Those relatively more prone to contact the CRA included:
Most participants have sought information using CRA’s online resources, either its general website or My Account. Having specifically targeted My Account users as a sub-segment of interest for this research, many participants did refer to using My Account to resolve some of their questions and enquiries. Specific findings related to My Account are discussed later in this chapter.
Participants expressed mixed feelings about their use of the “general” CRA website, meaning the information on the CRA website outside of My Account. While some did find the information they were looking for, others felt the site was too dense, too technical and not particularly user-friendly (in terms of page layout, information organization and language used). These challenges have hampered some participants’ ability to fully or confidently understand the material they are reading and to sometimes doubt they are accessing the correct information or documents. Examples of this include accidentally exiting a web page or downloading unnecessary or incorrect documents.
The ability to use a search function on the website when having trouble accessing information was appreciated. A few seemed to prefer using Google to do their searches.
Some avoided using the general CRA website altogether based on an expectation that it would be difficult to navigate.
Several participants, including many who also use the website, mentioned having called the CRA. Among those who prefer to use the website, calling is considered a last resort because they do not want to manage long wait times and they generally see calling as inconvenient. For a few, calling the CRA was the only approach they would ever consider using – these tended to be individuals with self-proclaimed low technology skills, individuals who would not trust a website to provide them with the answers or guidance they need, those who were concerned about the security of their personal information online and individuals who simply prefer speaking with someone.
Generally, those who preferred calling felt that the benefit of being able to completely resolve an issue on one call, to obtain the peace of mind that they were getting complete or accurate information or guidance, or to be able to ask multiple questions on a single call outweighed the inconvenience of long wait-times. Some simply felt they had no alternative but to call CRA.
When asked to explain the reason(s) they called the CRA, a variety of responses were provided:
Overall, very few participants claimed to have the need to follow up with a phone call to the CRA after finding the information they were looking for online. The most common reasons for a follow-up call included:
There was consensus among participants that the wait times over the phone can be a burden. Some also confirmed that their attempts to call had led to being disconnected. Many of those who did get through to an agent did feel that the service and knowledge levels were high and that agents were quite helpful.
Ultimately, many participants called CRA because they placed significant importance on “getting it right the first time” since, in their opinion, the consequences of misinterpreting information or getting it wrong are quite severe. The consequences could include unnecessary delays in getting an application approved, missing out on a tax credit or benefit, an incorrect tax return, getting audited, or having to deal with the CRA more than they already do. If any or all of these could be avoided by calling CRA, the wait times were seen as a small price to pay.
When asked if they had contacted the CRA regarding benefit-related matters, the most common issues included the following:
Participants, especially those using My Account, explained they did not need to contact CRA regarding payment amounts and dates since that information was easily found in their portal. Others explained that this information was also provided to them in the paper documents they received from CRA related to their benefits.
Many participants were making use of at least a few of the services in My Account, including:
For the most part, My Account users were quite pleased with their portal. They considered it easy to navigate, comprehensive, and providing information that is easy to understand. They also appreciate the partner sign-in feature.
The main inconvenience related to My Account came from participants who had had issues remembering their password, or others who had given up with the many steps they felt they needed to do to get a new password. Those who were using a Sign-in Partner were quite pleased with that feature and admitted that it saved them from having to remember one more password, especially one that is so rarely used. All those who used this approach to access their portal explained they were doing so through their financial institution. Having heard about the feature from other participants, those not or less familiar with this feature often showed interest in accessing their portal this way.
Other than improving the sign-in approach, by and large participants did not have many other suggestions to improve the portal.
Very few participants were aware of CRA’s mobile digital applications. A few had used MyCRA before and seemed pleased with the fact that they could directly access their portal from their mobile phone. There was interest among a few of the users in having access to as many features and information on their mobile phone as what they have through their desktop portal.
Otherwise, participants were certainly intrigued by the possibility of having a CRA app on their mobile devices, although it would appear that their interest would lie mostly in a “traditional” app that would be downloaded from their app store (rather than a mobile web app).
The appeal of having a CRA mobile app is driven by:
Those less interested in a mobile application expressed concerns with trust and the security of their information being accessed from a mobile device relative to accessing the information more securely through the desktop version of My Account. A few also felt that they would not want to “take up space” on their mobile device by downloading an app that would only be accessed once or twice per year.
When asked how the CRA could improve its online resources and services to make getting information online easier and to help Canadians self-serve, participants offered several suggestions. These included:
When it came time to discuss notifications and reminders specifically, many participants, had already explained that they were receiving notifications by email from CRA.
It should also be noted that by this point in the focus groups, a few participants had also suggested that it would be helpful if CRA could send them email notifications, fully unaware that the service was already available.
Once the service had been fully explained to all participants, there was clear interest in subscribing to these notifications among many of those not already receiving them, and there was clear interest and satisfaction among those already receiving the notifications.
Many participants revealed that this service is the only thing that prompts them to log onto their My Account. Many wouldn’t check their My Account unless an email notification comes in to advise them.
There was also some appreciation for the fact that the notification they receive only gives minimal information – that it simply prompts them to go to My Account and that “just in case someone has access to your email, they can't just go in and see exactly what the CRA's telling you.”
When prompted to suggest situations in which they would like to receive a notification, participants seemed interested in the following:
Given the variety of situations when a notification could be issued, some participants suggested that notification registrants should be given the option to pre-select the types of notifications they would receive from a long list of options. This way they would know that the notifications they receive are relevant and important.
If they were to receive notifications from CRA, most would prefer to receive them via email, although there was some interest in text messages as well. While some liked the idea of text messages, they felt that texts wouldn’t necessarily prompt them to go onto their account, whereas if they received an email, they would also see it on their computer and could go right to their account.
Texts were also perceived as a greater security risk than emails given the frequency at which fraudulent texts are received in general. Irrespective of the approach, participants voiced strong concerns with fraudulent emails and texts which constantly leave them questioning the legitimacy of any correspondence they receive claiming to be from the CRA.
Given the choice, participants would opt for minimal personal information in the notifications they would receive from the CRA. At most, participants would be open to the notifications giving some indication of the general nature of the notification, for instance: “You have received a new message from the CRA related to your benefits. For more information please log into My Account.” Many participants firmly believe that the less information in the notification, the better. Given the mix of preferences, a few participants suggested that the CRA should give registrants the option to customize the level of detail of their notifications.
Once in My Account, a few participants indicated that it would be helpful if the CRA provided supporting guidance, documentation, materials, etc. to help them manage certain types of notifications. For instance:
A few suggested embedding links in the email notifications to this type of information however when they were informed that emails from CRA never contain links for security reasons, these participants quickly understood the reasoning. The idea of having these links embedded in messages received inside My Account was more acceptable.
Participants were asked if they have access to any online or digital communication tools with any companies or organizations they deal with. Participants are very aware of live chat options on many of the websites they visit in general and on the websites of the companies with whom they regularly interact. For the most part, participants seemed to prefer this option over calling the various companies. In fact, some indicated that a good number of companies they deal with do not even offer a telephone option and that all correspondence needs to happen via live chat, something with which they seemed quite comfortable.
Examples of the types of companies that offer a live chat option included:
There was also some awareness of chatbots (e.g. Amazon), although actual familiarity with what they were and how they worked seemed low. In each session, the moderator needed to explain how these worked and that there were text-based chatbots and voice-command chatbots. Irrespective, when it comes to chatbots as a customer service tool, opinions were almost unanimously negative and fairly strong. Few seemed convinced that AI could replace a human, especially if the questions are detailed or file-specific.
A callback feature was raised in each group as well. Participants indicated that many telecommunications companies offer this feature which they find convenient because they do not have to wait on hold and they can specify a time or a timeframe for the callback so that it aligns with their availability. For instance, one participant explained that Bell seems to offer a feature that allows customers the option to send a message online and receive a callback once they have assessed the question or the request.
None of the participants indicated having the ability to email with the customer support from the various companies they deal with, or at least having to resort to that approach to get support.
Ultimately, nearly all participants agreed that the CRA should offer an online communication channel for enquiries or assistance of one kind or another. Interest was strongest for a live chat option. The next best thing would be some sort of “callback option.”
Participants did have specific expectations regarding a live chat feature:
Many participants actively suggested that the online communication channel should be housed within My Account believing that this would ensure that they are corresponding directly with CRA and also allow them to securely ask file-specific questions. There was an assumption that the agent at the other end would automatically have access to the file seeing as the client would already be logged into their portal.
Participants did seem open to the idea of perhaps using chatbots or a live-chat feature for general website navigation purposes or for general queries. Some questioned how different or better a chatbot could be compared to a general search feature on the website. Irrespective, the moment that a file-specific issue needed clarification, participants were not at all confident that a chatbot would suffice.
When specifically prompted about emailing with CRA, participants did not feel comfortable doing this outside of My Account.
The most common scenario to which participants would agree was:
Ultimately, participants generally seemed open to the idea of having different tools available to them to get at the answers they need insofar as at least one of the tools involved a “real agent” at the other end. It should also be noted that many participants stressed the importance of always having an option to speak to a “live” agent over the phone. Although this may not be an option they might use that often, when they do foresee needing to speak to someone it will be for something quite important and that option should be available.
In terms of service delivery, participants would expect a “live chat” to be instant communication. Expectations were noticeably different for emails, for which participants felt that a 24 to 48-hour turnaround time would be more acceptable. At a minimum they would expect a confirmation of receipt of their email shortly after sending it.
Generally, participants are familiar with the concept of a “message centre.” They have similar centres with their financial institutions (e.g. banks, insurance companies, etc.) and with some of their utilities and telecommunications service providers. Most do acknowledge however that they rarely visit those message centres since they tend to be statement notices, changes to policies, or marketing and promotions.
Participants were presented with a list of six possible types of communications they could receive in a secure CRA message centre. The most popular, or most valued, would be “Mail from the CRA such as an annual Notice of Assessment or an annual Benefits Statement.” When participants were asked what their top two communication items on this list were, the majority of participants selected this item.
The least popular was “a list of contact or attempts to contact you by a CRA employee”. This was the type of communication that had the most participants indicating they would never use or need it or would consider this type of communication “nice to have.” The remaining four types of communications were equally popular, which again prompted a few participants to suggest that users should be able to choose from a list the types of messages they receive.
Description of image above: Level of interest in six communication items, indicating whether participants felt each is “must have”, “nice to have”, or “would never use or need”.
When participants were asked why “Mail from the CRA such as an annual Notice of Assessment or an annual Benefits Statement” is a must have, they explained that this is the main reason they would want to visit My Account – these types of statements are at the heart of their interactions with the CRA. Most people also agreed it is great to have everything all together. Whether they owe money or are owed money, they are aware and can ensure it all happens in a timely manner.
“Copies of the email notifications that have been sent to you” were considered important because participants want to ensure all changes have been done correctly and can go back to confirm at any time. A few participants also mentioned that this could be a way for them to detect fraud (by double-checking that the notification they received via email is legitimate), and to make sure no one else is changing their information.
“Generic tax information such as announcements when a new tax credit or new online service is available” was also a very popular item. Participants wanted to know about anything new that was unavailable to them before and to make sure they aren’t missing anything. This particularly resonated with those who do their own taxes and don’t have someone to inform them of new tax credits.
One of the least popular items, “a list of contact or attempts to contact you by a CRA employee” would not be used by some participants because they felt that simply informing them when they tried to call wouldn’t solve the problem, especially if there is no way to easily return the call. Participants did not seem to appreciate that a confirmation of attempted contact via My Account could reassure them that the call was in fact legitimate.
Participants were then presented with a list of nine possible functionalities they could have in a secure CRA message centre. The two most valued would be to “Reply directly to an online message received from the CRA” or “Send a message to a CRA employee working on your file.” Ultimately, participants felt features that would allow them to communicate with someone at CRA would be of utmost value, especially if it allows them to ask file-specific questions. It leaves participants feeling more confident communicating with someone who is responsible for their file and knows their situation.
The least important features would be the ones that allow them to flag, delete or organize their messages. Participants did not think they would receive so much correspondence that they would need to organize it. As well, they felt any and all correspondence they receive from CRA would be considered important, so they did not see a need to flag or delete any of them. When it came to deleting specifically, a few participants thought they either “wouldn’t want to miss a small gap in history” or don’t think they would even have enough to delete.
For environmental reasons, some explained they would never use the print feature.
Examples of other suggestions for the message centre, or for My Account more broadly, included:
Having now obtained some idea of the types of communications and functionalities that a CRA message centre could offer, nearly all participants agreed that a message centre should be developed and that it should be housed in My Account.
Description of above image: Level of interest in nine online features and functionalities, indicating whether participants felt each is “must have”, “nice to have”, or “would never use or need”.
Overview: The research methodology consisted of a series of 8 focus groups in four different locations across the country: St. John’s, NL, Montreal, QC, Toronto, ON and Vancouver, BC.
The target population for this research consisted of benefit recipients who have contacted CRA for a benefit-related issue in the past two years, and who are active on the Internet. Benefit recipients were specifically targeted for this research for the following reasons:
A benefit recipient was defined for the purposes of this research as someone who receives at least one of the following benefits administered by the CRA:
Within these, there was a mix of My Account users and Non-Users of My Account. These two segments were defined as follows:
The CRA provided a list of benefit recipients. Further screening was done to ensure a mix of My Account Users and Non-Users of My Account and to ensure that some of the participants had contacted the CRA within the past 2 years for a benefit-related matter or for any other tax-related matter.
Quorus designed and translated the recruitment screener and the moderation guide for this study. In the design of the recruitment screener, specific questions were inserted to ensure a good representation of My Account Users and Non-User, gender, age and income.
Quorus recruited 10 participants to achieve 8 participants per focus group and recruited participants were offered an honorarium of $100.00 at the end of the focus groups.
Furthermore:
At the recruitment stage and at the beginning of each focus group, participants were informed that the research is for the Government of Canada/Canada Revenue Agency. Participants were also informed of audio/video taping of the focus group sessions, in addition to the presence of Canada Revenue Agency observers. Quorus ensured that prior consent was obtained at the recruitment stage and before participants entered the focus group room.
All focus groups were held in a facility that allowed the client team to observe the sessions. A total of 70 individuals participated in these focus groups. The locations and dates of the sessions are presented in the grid below:
| Location | Time (Local Time) | Language | Number of Participants | Date and Time |
|---|---|---|---|---|
| St. John’s, NL | 5:30 pm | English | 9 | February 24, 2020 |
| St. John’s, NL | 7:30 pm | English | 7 | February 24, 2020 |
| Toronto, ON | 5:30 pm | English | 9 | February 25, 2020 |
| Toronto, ON | 7:30 pm | English | 10 | February 25, 2020 |
| Montreal, QC | 5:30 pm | French | 8 | February 26, 2020 |
| Montreal, QC | 7:30 pm | French | 7 | February 26, 2020 |
| Vancouver, BC | 5:30 pm | English | 10 | February 27, 2020 |
| Vancouver, BC | 7:30 pm | English | 10 | February 27, 2020 |
Each focus group lasted 2 hours and all focus groups were moderated by Rick Nadeau, one of Quorus’ bilingual senior researchers on the Government of Canada Standing Offer.
Qualitative Research Disclaimer
Qualitative research seeks to develop insight and direction rather than quantitatively projectable measures. The purpose is not to generate “statistics” but to hear the full range of opinions on a topic, understand the language participants use, gauge degrees of passion and engagement and to leverage the power of the group to inspire ideas. Participants are encouraged to voice their opinions, irrespective of whether or not that view is shared by others.
Due to the sample size, the special recruitment methods used, and the study objectives themselves, it is clearly understood that the work under discussion is exploratory in nature. The findings are not, nor were they intended to be, projectable to a larger population.
Specifically, it is inappropriate to suggest or to infer that few (or many) real world users would behave in one way simply because few (or many) participants behaved in this way during the sessions. This kind of projection is strictly the prerogative of quantitative research.
| Date: | February 24 | February 25 | February 26 | February 27 |
| City: | St.John’s | Toronto | Montreal | Vancouver |
| Language: | English | English | French | English |
| 5:30 pm | Group 1 | Group 3 | Group 5 | Group 7 |
| 7:30 pm | Group 2 | Group 4 | Group 6 | Group 8 |
A. Screener
INTRO: Hello/Bonjour, my name is [RECRUITER]. May I please speak with [INSERT NAME FROM DATABASE]. I'm calling from Quorus Consulting Group, a Canadian research company. We're organizing a series of discussion groups on behalf of the Government of Canada to explore issues of relevance to Canadians.
Would you prefer that I continue in English or French? / Préférez-vous continuer en français ou anglais?
RECRUITER NOTE: FOR ENGLISH GROUPS, IF PARTICIPANT WOULD PREFER TO CONTINUE IN FRENCH, PLEASE RESPOND WITH, « Malheureusement, nous recherchons des gens qui parlent anglais pour participer à ces groupes de discussion. Nous vous remercions de votre intérêt ».
As I mentioned, we’re conducting a research study for the Government of Canada—specifically, the Canada Revenue Agency*, to explore issues related to the agency’s services. The groups will last up to two hours and people who take part will receive an honorarium to thank them for their time. Participation is completely voluntary and your decision to participate or not will not affect any dealings you may have with the Canada Revenue Agency. We are interested in your opinions only. The format is a “round table” discussion led by a research professional with up to 10 participants. All opinions will remain anonymous and will be used for research purposes only in accordance with laws designed to protect your privacy.**
*IF AN INDIVIDUAL QUESTIONS THE VALIDITY OF THE RESEARCH, INVITE HIM/HER TO CALL OR GO ONLINE:
**IF ASKED, SAY: The personal information you provide to the Canada Revenue Agency is governed in accordance with the Privacy Act. In addition to protecting your personal information, the Privacy Act gives you the right to request access to and correction of your personal information. You also have the right to file a complaint with the Privacy Commissioner of Canada if you think your personal information has been handled improperly.
ADDITIONAL INFO FOR THE INTERVIEWER IF RESPONDENT ASKS FOR INFORMATION ON HOW THE RECRUITER SELECTED THEM OR WHERE THEY GOT THEIR NAME:
You were randomly selected from a large list of phone numbers provided by the Canada Revenue Agency solely to assist in recruiting for these focus groups. We do not tell the Canada Revenue Agency who we have contacted from their large list. The list will be deleted immediately following the research. The list provided to us contains only the information required to assist with recruiting participants for the discussion sessions.
Would you be interested in taking part in this study?
Before we invite you to attend, I need to ask you a few questions to make sure we are getting a good mix of people for each discussion groups. This will take 5 minutes. May I continue?
Record gender by observation. [50/50 SPLIT]
1. We have been asked to speak to participants from all different ages. May I have your age please? [GET MIX]
NOTE: WHEN TERMINATING A CALL WITH SOMEONE, SAY: Thank you for your cooperation. We already have enough participants who have a similar profile to yours, so we are unable to invite you to participate.
2. Do you, or any member of your immediate family, work for? [READ LIST]
3. Do you currently receive any of the following benefits or credits from the Government of Canada? [READ LIST, RECRUIT MIX]
RECRUIT MIX
4. Have you contacted CRA by telephone for a benefit related issue in the past 2 years?
5. Have you contacted CRA for any other issue in the past 2 years?
6. Are you a registered My Account user? My Account is a secure portal provided by Canada Revenue Agency, or C-R-A, that lets you view your personal income tax and benefit information online. To access My Account, you need to enter your CRA user ID and password. IF NEEDED, SAY: My Account is not to be confused with Netfile. Netfile allows you to file your personal income tax and benefit return directly to the CRA using the Internet, but not to manage your tax affairs online.
7. In the past 12 months, have you logged into My Account to do any of the following?
8. In the past year, have you done any of the following online?
9. And, in the last few years, have you done any of the following online with any level of government in Canada?
IMPORTANT: WE WILL TRY AND GET A MIX OF MY ACCOUNT USERS AND NON-USERS FOR EACH GROUP
10. Could you please tell me what is the last level of education that you have completed? [GET MIX; READ LIST]
11. Which of the following best describes your employment situation? Are you?[GET MIX; READ LIST]
12. During the last 12 months, what was your total household income before taxes and deductions, from all sources? Was it? [GET MIX UNDER $80,000; READ LIST, IF NEEDED] PROVIDE A REMINDER THAT THIS INFORMATION IS CONFIDENTIAL. MAX 2-3 PER GROUP FOR ABOVE $80,000 CATEGORIES.
13. Have you ever attended a discussion group or taken part in an interview on any topic that was arranged in advance and for which you received money for participating?
14. When did you last attend one of these discussion groups or interviews? [DO NOT READ UNLESS HELPFUL]
15. Thinking about the groups or interviews that you have taken part in, what were the main topics discussed?
16. How many discussion groups or interviews have you attended in the past 5 years?
17. Participants will be asked to look at some items and voice their opinions in front of others. Do you feel comfortable doing this?
18. The discussion group will be video-recorded. These recordings are used to help with analyzing the findings and writing the report. The results from the discussions will be grouped together in the research report, which means that individuals will not be identified in anyway. Neither your name nor your specific comments will appear in the research report. Is this acceptable?
19. There may be some people from the Canada Revenue Agency who have been involved in this project observing the session in-person. They will not take part in the discussion and they will not know your name. Is this acceptable?
B. Invitation to Participate
Thank you. We would like to invite you to attend one of the discussion groups, which will be led by a researcher from the national public opinion research firm, Quorus Consulting Group. The group will take place on [DAY OF WEEK], [DATE], at [TIME] and it will last two hours. Following your participation, you will receive $100 to thank you for your time.
20. Are you willing to attend?
21. May I have your email address so that we can also send you an email message with the address where the group will be held, as well as a consent form to read and bring with you to the focus group?
At the facility, you will be asked to produce photo identification, so please remember to bring something with you (for example, a driver's license). In order to participate, you will also be asked to sign a consent form, which we will be available at the facility upon your arrival.
If you use glasses to read, please bring them with you.
As we are only inviting a small number of people to attend, your participation is very important to us. If for some reason you are unable to attend, please call us so that we can get someone to replace you. You can reach us at [INSERT NUMBER] at our office. Please ask for [INSERT NAME].
Someone from our company will call you the day before to remind you about the session. To do that, we need to have your contact information.
First name:
Last Name:
Daytime phone number:
Evening phone number:
Email:
Thank you!
Focus Groups with Benefit Recipients Moderation Guide
So let's go around the table and have everyone introduce themselves. I'll be curious to know the following:
Tonight's research is intended to help Canada Revenue Agency (CRA) understand how to design and deliver services to best meet your needs, preferences, and expectations, with a focus on improving its online or digital service offerings.
We would like to start the discussion this evening with getting your thoughts and opinions on how you interact with the Government of Canada or the CRA when you need to contact them for information or assistance.
1. In general, when you think of the times you have needed to contact the Government of Canada or the CRA over the past few years…
MODERATOR TO EXPLORE FOR EACH INSTANCE
2. IF BENEFITS ARE MENTIONED:
Some of you indicated that you were trying to find out information about when you would receive your benefits payment or how much the benefits payment would be. My understanding is that payment dates and amounts are pretty much set and don't change so I am curious as to why you would contact the Government or CRA for that kind of information?
3. Do any of you use the CRA's online services such as the "My Account" portal or the mobile Apps to help you look up information or to seek assistance with your tax and benefits file?
MODERATOR TO GET A CLEAR READING OF WHO USES THE PORTAL VERSUS EACH OF THE APPS AND WHEN THE LAST TIME THEY WERE USED
IF NEEDED:
My Account is a secure service for Canadians to conduct their federal tax affairs online using the CRA's website.
MyCRA is a mobile web app for individuals that lets you securely access key portions of your tax information and pay your tax balance owing.
MyBenefits CRA is a mobile web app for individual benefit recipients that lets you quickly view your benefit and credit payment details and eligibility information.
Please note that using tax software to electronically file your income tax return or making online payments to the CRA does NOT mean you have registered with the My Account service.
Next we'd like to talk about CRA's ability to send you notifications and reminders.
1. Did you know that the CRA offers an electronic notification service by email that can send alerts for specific account changes such as a change of address or direct deposit?
Moderators Info: Alerts are available for
2. In what other situations would you find it helpful to receive a similar type of alert from CRA?
3. How would you want to receive these notifications? Ex. Text, email, in one of the CRA apps, in My Account, other?
4. Does the type of notification you'd like to receive depend on the subject? Would it be any different, for instance, for payments notifications, payments due, change of address?
5. What level of personal information are your comfortable receiving in these notifications?
6. Do you have any (other) suggestions to help CRA make the electronic notifications service as useful as possible for you?
So far, we’ve discussed one-way alerts coming from the CRA to Canadians.
However, many Canadians have expressed an interest in corresponding or interacting (which suggests an exchange of some sort) with the CRA electronically rather than having to call to get answers to their questions.
1. I’d like you to think about online or digital conversations or interactions you might be having with other institutions or companies you deal with, either regularly or from time to time – which ones do you feel are doing this really well? In other words, they have it figured out.
2. Thinking specifically about your interactions or potential interactions with CRA, do you feel CRA should offer an online communication channel for enquiries or assistance of one kind or another? [GET HAND COUNT]
3. What types of services would you like to see offered and how would you see yourself using them? QUICK FLIPCHART LIST
PROBE:
4. Do you feel the CRA should offer various service channels depending on the nature of the enquiry? For example, secure vs non-secure enquiries.
5. IF NOT FULLY EXPLORED IN Q3 ABOVE: What types of enquiries or assistance would you consider contacting CRA about through an online channel and why?
6. And just as before, where do you draw the line in terms of the types or amounts of personal information you would expect to share or receive via non-secure correspondence (i.e. regular email)?
7. What would be your expectations for service delivery when contacting CRA through an online communication channel?
PROBE:
8. Do you have any (other) suggestions about online communication channels that would be useful to CRA?
Our last subject this evening will be about message items and functionality within an online message centre or communications centre within the secure portal.
1. Thinking about the online services you currently use or have experienced in your personal life, such as your online bank portal, utilities company, phone company or maybe another government online service, what types of communication items do you generally receive in these online communication centres?
2. Thinking specifically about your communications items or potential communication items with CRA, what items would you expect or like to receive in a message centre?
VALIDATE EACH ITEM BELOW USING HANDOUT A: I have some ideas I’d like to run by you.
3. Now, when you think of the actions you generally take related to your communications items, or actions you may want to take, what functionalities or features do you need beyond just being able to read the message sent to you?
VALIDATE EACH ITEM BELOW USING HANDOUT B:
4. TIME PERMITTING: Focusing on you trying to communicate with the CRA. What types of communications would you like to be able to create and send from this message center?
1. When you consider everything we’ve discussed tonight, what feature or features are you the most excited about, if any?
2. Some of you here are not currently signed up to My Account and some of you may be signed up but you are not using it all that much.
[BACKROOM CHECK]
3. Do you have any last comments or suggestions about anything we have discussed tonight?
Please remember to sign out as you leave the focus group room – this is just to confirm you’ve received the incentive we promised you. [FOR GROUP 1: Take care as well not to discuss what has been discussed here as you leave the facility since I have participants from my next session in the lobby/ waiting area.]
Thank participants
Thinking specifically about your communications items or potential communication items with CRA, what items would you expect or like to receive in an online message centre?
Put a ✓ in the box that you think applies to each statement
| Must have | Nice to have | Would never use or need | |
|---|---|---|---|
| A. Mail from the CRA such as an annual Notice of Assessment or an annual Benefits Statement. | |||
| B. Copies of the email notifications that have been sent to you. Ex. Your address was changed | |||
| C. A list of contact or attempts to contact you by a CRA employee. | |||
| D. Copies of enquiries you have sent the CRA online. | |||
| E. Generic tax information such as announcements when a new tax credit or new online service is available. | |||
| F. Account specific tax announcements, such as a filing extension for your region due to flooding. |
When you think of the actions you may want to take related to communications items in an online message centre, how important would you consider each of the following functionalities and features:
Put a ✓ in the box that you think applies to each statement
| Must have | Nice to have | Would never use or need | |
|---|---|---|---|
| A. Print a piece of correspondence | |||
| B. File away in a folder – being able to archive certain messages that you may wish to keep on file, but not see in your main inbox | |||
| C. Download a correspondence item to your computer (“Save As” feature) | |||
| D. Delete (would only be available for non-mandatory items such as copy of a notification. Items such as a Notice of Assessment could not be deleted. But maybe you want to file it or organize it a certain way.) | |||
| E. Organize your mail/messages into different, customizable folders, e.g. by tax year. | |||
| F. Reply directly to an online message received from the CRA | |||
| G. Flag certain messages for follow-up or mark them as important | |||
| H. Send a message to a CRA employee working on your file | |||
| I. Start an online account-specific enquiry |