Supplier name: Quorus Consulting Group Inc.
Contract #: CW2236398
Contract Amount (incl. HST): $88,818.00
Contract Award date: July 20, 2022
Delivery date: November 2022
Registration number: POR 016-22
For more information on this report, please contact the CRA at: cra-arc.media@cra-arc.gc.ca
Ce rapport est aussi disponible en français.
Accessibility Testing of CRA’s Forms and Guides
Final Report
Prepared for the Canada Revenue Agency
Supplier name: Quorus Consulting Group Inc.
November 2022
This public opinion research report summarizes the findings from 44 depth interviews with individuals from across the country. Recruited participants consisted of individuals who identify as having one of four different categories of disabilities. The interviews were held between August 23 and October 4, 2022. Each session lasted approximately one hour.
Cette publication est aussi disponible en français sous le titre : Essais d’accessibilité des formulaires et des guides de l’Agence This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from the Canada Revenue Agency. For more information on this report, please contact the Canada Revenue Agency at: cra-arc.media@cra-arc.gc.ca
101 Colonel By Drive
Ottawa, Ontario K1A 0K2
Canada
Catalogue Number: Rv4-126/1-2022E-PDF
International Standard Book Number (ISBN): 978-0-660-45873-1
Related publications (registration number: POR 016-22)
Catalogue Number: Rv4-161/1-2023F-PDF (rapport final, français)
ISBN: 978-0-660-45875-5
© His Majesty the King in Right of Canada, as represented by the Minister of the Canada Revenue Agency, 2022
Quorus Consulting Group Inc.
Contract number: CW2236398
POR registration number: 016-22
Contract award date: July 20, 2022
Contracted cost: $88,818.00
The Accessible Canada Act (the Act) was enacted in July 2019, cementing the Government of Canada’s commitment to proactively identify, remove, and prevent barriers to accessibility where Canadians interact with areas under federal jurisdiction, whether they be internal or external to an organization.
In an effort to achieve this, the Canada Revenue Agency’s (CRA) Information Programs Division (IPD), a responsible authorizing division of the CRA’s CRA tax and benefit administration publications, wanted to hear from people with disabilities about the ease or difficulty in finding the information they need, completing forms, as well as suggestions on how navigation could be improved.
The purpose of this research was to identify accessibility gaps in CRA tax and benefit administration publications, focusing on forms and guides. Specifically, the objectives were to identify the following:
The results of the research will be used primarily to improve tax and benefit administration publications for people with disabilities that allow all Canadians to meet their tax obligations both simply and accurately. The CRA will use the insight gained to update its processes, tools and publications in the implementation of the Accessible Canada Act as well as to create its Publication Accessibility Plan by December 2022, which will be included in CRAs agency wide Accessibility Plan.
The research methodology consisted of 44 one-on-one depth interviews conducted online and by telephone in English and French. The interviews were held between August 23 and October 4, 2022. Interviews were completed with individuals from across the country who identified as living with one or more of the following types of disabilities: a visual disability, a hearing disability, a physical disability and/or a neurodiverse disability. Each session lasted approximately one hour. Participants were informed upfront that the research was being conducted on behalf of the Canada Revenue Agency and they each received an honorarium of $250 for their participation.
Additional details on the research methodology can be found in the full report.
Note to reader: Qualitative research seeks to develop insight and direction rather than quantitatively projectable measures. The purpose is not to generate “statistics” but to hear the full range of opinions on a topic, understand the language participants use, gauge degrees of passion and engagement and to leverage the power of the group to inspire ideas. Participants are encouraged to voice their opinions, irrespective of whether or not that view is shared by others.
Overall, participants had limited experience using CRA guides and forms in the recent past. Overwhelmingly, the bulk of this experience revolved around filing their annual tax return, and to a slightly lesser degree with Disability Tax Credit forms and/or guides.
Many participants used tax software (either purchased or a free version) to file their annual tax return with a small proportion resorting to a paid tax service, and as such, they had limited direct experience using specific forms and guides.
Generally, participants indicated that they have been able to access and use the various guides and forms that they remembered needing. That said, some have had to make various adaptations to successfully use a guide or complete their forms. When thinking about their overall experiences, the most common challenges raised by participants included the following:
Focusing on guides, the following high-level findings were uncovered:
Focusing on forms, the following high-level findings were uncovered:
Process overview: Twelve different CRA documents (one guide and eleven forms) were selected for review. Each participant was provided three documents ahead of their interview in their preferred format. Format options available to participants included: print/paper-based, PDF, fillable PDF, large print, braille, Digital Audio (MP3), and Electronic text (Etext). Participants were instructed to browse each document and, for forms, attempt to complete various sections, while completing the various sections with fictional information to determine the usability of the form, the accuracy of the information entered was not relevant for the purpose of the study.
In some cases, particular features of the documents were seen as a strength to some, while the same feature was viewed as a weakness to others. This was sometimes due to the type of disability that a participant had (for example, some individuals with vision-related disabilities were fond of forms which included text in multiple columns, while those with a neurodiverse disability found this to be a weakness). In other cases, there may be some overlap between the strengths and weaknesses due to the particular documents that a participant was assigned, or due to the format they used for the accessibility review (for example, large font versus standard font), or due to the program in which they opened the document.
Some of the highlights of the feedback provided on the guide (Support Payments) include:
Some of the highlights of the feedback provided on forms include:
Some participants could not complete a fillable and saveable PDF because of the software or application they were using. This research revealed that many participants will open these types of documents in their browser or in PDF "readers" other than Adobe with varying degrees of success. Although the CRA website might indicate that the best user experience is with Adobe, not all form users obtain these forms directly from the site, and even if they do, they might not notice that recommendation or they might believe that if they are able to open a PDF in a given application, they should also be able to complete it. Not being able to do so, irrespective of the application, would be seen as an accessibility barrier by the end-user.
By the end of each interview, participants were impressed by the CRA's ability to provide forms and guides in alternate formats and that this is something that should be better promoted/marketed. When asked how they would go about accessing these types of files, the vast majority would do a search on the CRA website or perhaps Google. There is a high level of expectation of being able to download their preferred format directly from the CRA website - needing to email or call the CRA to "order" a document seemed like an antiquated approach.
I hereby certify as Senior Officer of Quorus Consulting Group Inc. that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Policy on Communications and Federal Identity and the Directive on the Management of Communications - Appendix C.
Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.
Signed:
November 1, 2022
Rick Nadeau, President
Quorus Consulting Group Inc.
The Accessible Canada Act (the Act) was enacted in July 2019, cementing the Government of Canada's commitment to proactively identify, remove, and prevent barriers to accessibility where Canadians interact with areas under federal jurisdiction, whether they be internal or external to an organization.
Accessibility is a fundamental part of client experience, and strengthening accessibility across their programs, services, and policies is at the heart of the Canada Revenue Agency's (CRA) People First philosophy.
In an effort to achieve this, the CRA's Information Programs Division (IPD) wanted to hear from people with disabilities about the ease or difficulty in finding the information they need, completing forms, as well as suggestions on how navigation could be improved.
The purpose of this research was to identify accessibility gaps in CRA tax and administration publications, focusing on forms and guides. Specifically, the objectives of this qualitative public opinion research were to identify the following:
The results of the research will be used primarily to improve tax and benefit administration publications for people with disabilities that allow all Canadians to meet their tax obligations both simply and accurately. The CRA will use the insight gained to update its processes, tools and publications in the implementation of the Accessible Canada Act.
The CRA will also use the insight gained to update its processes and tools to create its Publication Accessibility Plan by December 2022, which will be included in CRAs agency wide Accessibility Plan.
Before exploring the accessibility of specific guides and forms, all participants were asked to speak about their past experiences with these types of CRA documents. General explanations of a CRA form and a CRA guide were provided to participants and the moderator also asked that they focus on their more recent experiences.
Overall, participants had limited experiences with CRA guides and forms. For those who could recall using CRA forms and guides, the bulk of this experience revolved around filing their annual tax return, and to a slightly lesser degree, forms and guides relating to the Disability Tax Credit.
Many participants used tax software (either purchased or a free version) to file their annual tax return with a small proportion resorting to a paid tax service, and as such, they had limited use of the forms and guides that the CRA provides.
Generally, participants indicated that they have been able to access and use the various guides and forms that they remember needing. That said, some have had to make various adaptations or resort to various types of support to successfully use a guide or complete their forms. When thinking about their overall experiences, the most common challenges raised by participants included access to hardcopies, use of technical language, and small font.
Firstly, many participants mentioned having trouble finding hardcopies of the annual tax guide and related forms in their community. Not being able to easily find the format they prefer using was seen by many as an accessibility issue.
Next, a number of participants described struggling when it comes to the use of "tax jargon" or technical language in forms and guides. This typically leads to one or a combination of the following: needing help from someone they know, taking longer to complete their form, completing their from with some degree of uncertainty, and/or contacting the CRA for help or clarifications.
"It gets a little bit too technical… I really couldn't do the forms to be quite honest, so I had my daughter helped [sic] me a lot. I just found it too complex… and I just gave up." – female, Ontario, 55, hearing and visual disability
When discussing overall experiences with CRA forms and guides, many mentioned having issues when it came to font size. For some, this meant relying on help from someone else or needing a magnifying glass to read paper versions of guides and forms because the font size was too small to be easily read.
"I was using a magnifying glass and just going over it. I know it's not the best, but it took a lot longer than if I would've been able to see it just normally. I think that's something obviously that's always been challenging for me is, I find the font to be too small." – male, 33, Ontario, very low or no vision
Similarly, some discussed having difficulties with documents which lack spacing between lines or paragraphs of text. Not having an appropriate amount of spacing between text can make documents difficult to read for those who struggle with vision, as well as affect some participants' ability to concentrate on what they are reading.
"When you have ADHD, you're looking at a page and it's like your eyes are kind of going everywhere, especially when there's a bunch of stuff kind of clumped together, like words. And if you're double spacing it, I'd find that it's easier for me to read, and in a lot of cases, retain it." – male, 52, New Brunswick, person with a neurodiverse disability
That said, a few felt that there needs to be a balance - as one participant explained, pages with calculations are easier to navigate and fill out when the lines are closer together.
After discussing participants' overall experience with CRA forms and guides, the conversation next focused more in depth on CRA guides.
The use of CRA guides is relatively rare, primarily because participants have not needed them (rather than because of accessibility issues). The most common guide used is the one needed for annual tax returns, and as noted above, the main challenge for participants is related to finding the paper version. This challenge is almost exclusively encountered by those who mail in their tax return.
When it comes to alternate formats for guides, including a large print version, awareness was very low. Given the option, most participants would seem to prefer a digital PDF version of CRA guides. The main benefits to this alternate format include the ability to zoom in and enlarge the text as much as they desired.
"If I've got it on the computer... I can blow up the font." - female, 55, Ontario, hearing and visual disability
Additionally, this format allows individuals who save the document on their computer the ability to refer to it whenever they like without having to print it out. Some do not want or like to print documents, and a few cannot print documents since they do not have access to a printer.
"I would prefer a digital guide because, the thing is, I can also save it on to my desktop for future references. I don't have to worry about filing it away or anything like that. Like in a paper filing system." – male, 42, New Brunswick, person with a neurodiverse disability
Another benefit of a digital PDF format is the ability to search for key words.
"You can hit the control key and the letter F and I can find something on that page, so I use that a lot. That's a Windows feature, but if I didn't know that feature, it would definitely take me longer to find the things I'm looking for just because the topic doesn't jump right out sometimes." – female, 54, New Brunswick, very low or no vision
Those with a vision impairment have a strong preference for a digital version given their ability to control font size or their need of screen reader technology.
"I would prefer to get all guides as PDFs, just because PDFs are easier to work with, to zoom in to find things." – male, 37, Alberta, very low or no vision
Additionally, some participants who use software to accommodate their needs and preferences prefer to have a digital version so that they can control the settings on their device or software in a way that makes it easier and more comfortable for them to navigate the document.
"The digital is really good because like my eyesight, I get eye fatigue. It can get pretty bad looking at like bright colours for a long time and so with that, I can turn the brightness…I can't really turn down the brightness of paper or like the sun…so I can like turn down the brightness of my devices. I can change the contrast or the colour balances to bring out like more orangey colours so it's a lot easier on my eyes especially if it's a larger form." – male, 19, Alberta, person with a neurodiverse disability
Those who prefer a hardcopy also tend to be proponents of filing paper tax returns. These individuals are also less likely to be tech-savvy and prefer to keep a paper trail. They also enjoy marking up the document (for example., notes in the margin, highlighting, etc.).
"I would prefer paper because then at least I can go back and forth… Another thing I like to do is I like to highlight. So, if I'm using a guide and then let's say I get stuck somewhere I'll highlight." – female, 40, Ontario, very low or no vision
Some individuals have trouble looking at a computer screen for a long period of time, especially those who need to zoom into the document. Additionally, other participants (mostly among those with a neurodiverse disability) explained they can become too easily distracted using a computer, hence the preference for hardcopies.
"Staring at the computer for me is not ideal because then I will end up with a migraine… I'm straining my eyes, regardless of what font it is because I don't see the full word. So, if I'm looking at a page, for example, I don't see the full eight by 10 document… So, if I'm reading the paragraph, I would only see half of the paragraph. So, I have to keep moving my head to keep reading." - female, 40, Ontario, very low or no vision
When asked about accessibility barriers to using CRA guides, few could recall having encountered any significant barriers in recent memory. For those who had used paper versions in the past, the main challenges were due to the font size and "dense text"/ lack of spacing between the text. However, these challenges with font size or text density tended to slow down a participant's use of a document rather than completely impede them from using it.
"I'd be able to use [a large print format], I think no problem. Again, it'll just take me a little bit longer than anybody else, like some things that will take you five minutes may take me 15 to 20 minutes." – female, 40, Ontario, very low to no vision
Another common challenge among participants was the use of technical language, which was seen as an issue irrespective of the format.
"The guides should be more simplified. It gets a little bit too technical." – female, 55, Ontario, hearing and visual disability
Participants were then asked what they recalled about their recent experiences with CRA forms.
The annual tax return and the form for the Disability Tax Credit were most often mentioned. Some participants confused other types of "forms" with CRA forms, such as the online forms for employment insurance. Forms such as these that are integrated into a website are generally seen as user-friendly and accessible.
When it comes to CRA forms, format preferences largely align with how a participant files their tax return. Participants' reasons for preferring a specific format of a form were consistent with their reasons for preferring a specific format of a guide.
As with CRA guides, participants had very low awareness of the option for alternate formats for forms, including large print versions.
Participants voiced that they would like to be able to submit digital forms more easily. Most participants find it inconvenient and time consuming to print forms out and mail them. Additionally, as noted earlier, not all participants have access to a printer at home.
"A preferred approach would have been for me to be able to fill it out electronically and send it back…. I had to leave my house to go to a physical store. I had to pay a service to take me there because I'd use my wheelchair to go there… like if it was a form I could fill it online, it all would have been done in five minutes." – female, 54, New Brunswick, very low or no vision
From memory, most participants could not remember encountering many accessibility barriers while completing CRA forms. Similar to guides, challenges were typically related to font size, lack of spacing/text fields being too small (especially for paper versions), and the use of technical language. That being said, some participants believed that certain forms are exceedingly complex or ask for too much information and that they should be streamlined and more straightforward.
A challenge specific to those who rely on screen readers is that they seem to recall that not all forms have historically cooperated with their device or software, which has required them to ask for help to complete their form.
For the remainder of each interview, the moderator and participant reviewed selected CRA forms and guides in order to gauge the extent to which they were accessible.
In total, twelve different CRA documents (one guide and eleven forms) were selected for review (the complete list is available in the Methodology section). Each participant was provided three documents ahead of their interview in their preferred format. Participants chose between having the documents sent to them by mail or email. Additionally, those who requested for the documents to be mailed for them could choose between regular print, braille, large print, or audio format.
Participants were instructed to browse each document and, for forms, also attempt to complete various sections.
For those who typically use a screen reader to navigate forms and guides, they were encouraged to test how their technology was able to navigate the documents, and flag any issues as needed. Across the interviews, various screen readers were mentioned including ChromeVox, Windows 11 Narrator, JAWS, as well as screen readers built in to various devices (computer, tablet or phone) with some specifically mentioning the built in apple screen reader and speech to text function.
One CRA guide, Support Payments, was presented to participants for review. For the most part, this document was considered accessible by nearly all participants, especially those who reviewed the digital PDF version.
In some cases, particular features of the documents were seen as a strength to some, while the same feature was viewed as a weakness to others. This was sometimes due to the type of disability that a participant had (for example, some individuals with vision-related disabilities were fond of forms which included text in multiple columns, while those with a neurodiverse disability found this to be a weakness). In other cases, there may be some overlap between the strengths and weaknesses due to the particular documents that a participant was assigned (for example, some felt that one form had an appropriate amount of spacing between text, while they felt another form did not), due to the format they used for the accessibility review (for example, large font versus standard font), or due to the program in which they opened the document.
Some key strengths of this document included:
"I think there isn't overly complex language where someone filling out or reading this guide for the first time would have too much trouble. I think the language is fine." – male, 33, Ontario, very low or no vision, referring to guide 3B
The presentation of text in two columns received mixed reviews. Those who zoom into the document on their computer appreciated having all the text they need to read spread over just half the page, reducing their need to repeatedly scroll left and right. This preference was most often noted by those with a vision-related disability.
Some however prefer a single column of text, which will increase their likelihood of staying focused and could help with better overall page layout, which in turn enhances readability. This was typically mentioned by participants with a neurodiverse disability.
Despite the guide receiving some positive feedback, some weaknesses were also highlighted. The self-assessment questionnaire flow chart on page 7 attracted the most criticism. Some participants with a neurodiverse disability expressed some difficulty following the flow and logic of the diagram and considered it overwhelming and confusing. Even more problematic was the poor interaction with screen readers when it came to this flow chart. Participants suggested that a different approach, such as showing one step at a time, rather than the flow chart layout would be advisable for all those who struggled with this page.
That being said, the same flowchart presented in the guide can also be accessed in a more accessible version within the online HTML format. The HTML version allows the user to view and answer one question at a time. This option may alleviate some or all of the difficulties experienced by the participants who struggled with the flowchart wither due to poor interaction with screen reading technology, or due to general confusion when following the logic of the diagram. However, this alternative may not be useful to those who prefer hardcopies or do not consider themselves tech-savvy enough to navigate the guide online. The CRA may also need to make this option better known so that those who prefer the online questionnaire over the flowchart know it is available and also know where they can find it.
Additional feedback received concerning this document included the following:
Many were pleasantly surprised by the list of alternate formats on the second page of the document, including the means by which they could obtain them. None of the participants were aware of these options prior to this study.
A total of 11 different CRA forms were presented across the interviews. Feedback was fairly consistent across the different forms, as well as across the different disability segments.
However, some participants had varying feedback depending on the format they selected. Specifically, participants who either received large print versions of the forms or were presented large print versions by the moderator via the Zoom share-screen feature were impressed with that format. They all indicated that this format would not only be more accessible for them, but for some, it would also enable them to complete forms more independently. There was strong interest in accessing fillable and saveable versions of the large print format.
"I actually found that this had more definitions and explanations than I'm used to with the CRA documents which was nice." – female, 43, British Columbia, person with a neurodiverse disability, referring to form 2C
"It's not full of like very specific and technical terms. It's very layman's terms, easy to understand, and very…it's not overcomplicated at all." – male, 19, Alberta, person with a neurodiverse disability, referring to form 2C
As with the accessibility review of the guide, participants also differed in their preferences for certain features of the forms. For example, when discussing the amount of "white space" in a form, some felt this was a strength while others felt it was a weakness. This was either due to their disability type , or may have been due to the different forms they received. For form 4A - at least one participant felt there was not enough space, while another participant commented that there was a good amount of white space in form 1C. Both of these participants had regular font.
Some of the key strengths of the various forms reviewed included:
Sections that are clearly delineated and separated one from the other by using boxes and large font section titles (for example, Form 1A).
Some of the key weaknesses of the various forms reviewed included:
"Let's say people want to write big like for me, because I'm doing it paper format. Like it just seems constricting" - female, 60, Ontario, physical disability, referring to form 2B
To improve the user experience, the moderator needed to remind several participants that it would be best to open fillable and saveable PDF files in Adobe. Despite these efforts, some participants could not complete a fillable and saveable PDF because of the software or application they were using. This research revealed that many participants will open these types of documents in their browser or in PDF "readers" other than in Adobe (such as "Foxit" or "Lumin") with varying degrees of success. For example, one participant was not able to fill out the postal code in form 4B using the program "Foxit".
Although the CRA website might indicate that the best user experience is with Adobe, not all form users obtain these forms directly from the site, and even if they do, they might not notice that recommendation or they might believe that if they are able to open a PDF in a given application, they should also be able to complete and save it. Not being able to do so, irrespective of the application, would be seen as an accessibility barrier by the end-user.
By the end of each interview, participants were impressed by the CRA's ability to provide forms and guides in alternate formats.
"What I like is the options that you can do the forms online, you can get it in a print, you can get it in a large print. So, that's great." – female, 51, Ontario, physical disability
As many were previously unaware that they could access forms and guides in alternate formats, they felt that this is something that should be better promoted/marketed by the CRA. When asked how they would go about accessing these types of files, the vast majority would do a search on the CRA website or perhaps Google. Some also felt a part of the CRA home page or a part of the website reserved for forms and guides should be specifically dedicated to accessibility-related resources, tips and documents.
"It would have to be on the very front page and very clear that if you have any disabilities or vision problems, et cetera…I don't know what words you'd use but it can be, ‘Click here'." – female, 55, Ontario, hearing and visual disability
There is a high level of expectation of being able to download their preferred format directly from the CRA website – needing to email or call the CRA to "order" a document seemed like an antiquated approach.
Time permitting, participant were asked how the CRA’s forms and guides compare to those from other government levels, such as provincial and municipal governments, in terms of accessibility. Participants did not access forms and guides from other levels of government too often. For the most part, participants felt that the CRA was a leader in this area, especially once they found out that forms and guides were available in alternate formats.
“Well, they definitely adapted… They're better than my municipality.” Female, 46, Ontario, a physical disability, referring to documents 2C, 3C, 4C
Overview: The research methodology consisted of 44 depth interviews with individuals from across the country. Research was conducted between August 23 and October 4, 2022.
Quorus was responsible for coordinating all aspects of the research project including working with CRA in designing and translating the recruitment screener and the interview guide, coordinating all aspects of data collection logistics, including participant recruitment, providing forms and guides in the participant’s preferred format, providing the interview platform, moderating all interviews and delivering required reports at the end of data collection. The research approach is outlined in greater detail below.
The target audience for this research study consisted of members of the general population, ages 18+, who identify as having one of the following:
Regional representation was achieved, with interviews distributed fairly evenly across the following four regions: Atlantic Canada, Quebec, Ontario, and Western Canada. Interviews were conducted in both English and French.
On a best-effort basis, the research recruitment also aimed to include a good representation by age, gender, ethnicity, education, income, professional status, etc.
In addition to the general participant profiling criteria noted above, additional screening measures to ensure quality respondents include the following:
Data collection consisted of one-on-one telephone and online depth interviews. Each interview lasted approximately one hour.
For each target audience, Quorus aimed to complete 12 interviews, with the goal of achieving a total of 48 interviews. All interview participants were offered an honorarium of $250 after their session. In instances where an individual required a caregiver to assist with their participation, an additional incentive of $125 was provided.
Participants invited to participate in the interviews were recruited by telephone through random digit dialing of the general public, through social media outreach, through the use of a proprietary opt -in database and by contacting various organizations and groups directly involved with specific population segments targeted for this study.
Despite these efforts and the opportunity to conduct their interview via a range of accessible approaches (including the option to use an interpreter), the initial target of 48 interviews could not be achieved. The shortfall was exclusively among participants for Audience 2 (hearing and visual disability). Not only were individuals in this segment extremely difficult to find, but given their disabilities, it was particularly difficult to convince them to participate in research sponsored by the CRA. Many potential participants were very skeptical of the legitimacy of the study despite being able to verify its authenticity directly on the CRA website. Some from this segment also explained that all their tax matters are handled by someone else, which in turn limited the relevance and ability to provide feedback based on direct experience of form and guide accessibility for these individuals.
The recruitment of participants followed the screening, recruiting and privacy considerations as set out in the Standards for the Conduct of Government of Canada Public Opinion Research-Qualitative Research. Furthermore, recruitment respected the following requirements:
At the recruitment stage and at the beginning of each interview, participants were informed that the research was being conducted for the Government of Canada and the CRA. Participants were informed of the audio/video recording of their session and of the presence of CRA observers. Quorus ensured that prior consent was obtained at the recruitment stage and before participants began their interview session.
During the screening process, if a participant explained that they required a caregiver, family member or friend to assist them with participating in the interview, they were informed that they could have this person attend the interview with them. An additional incentive of $125 was offered for these instances.
All sessions were conducted during regular business hours or during evenings (whatever suited the respondent's availability and preferences). Participants were presented with the option of completing the interview either by telephone, or online via the web-platform, Zoom. For participants who requested the interview to be conducted online, the research team used the Zoom platform to host and record the session (through microphones and webcams connected to the moderator and the participant's electronic devices, such as their laptop, tablet or smartphone) enabling client remote viewing. For telephone interviews, an audio recording was captured using the moderator's smartphone.
In total, twelve different CRA documents (one guide and eleven forms) were selected for review by the CRA client team. Each participant was provided three documents ahead of their interview in their preferred format. The distribution of documents across the list of participants can be found in Tables 3 through Table 6.
Participants chose between having the documents sent to them by mail or email. Additionally, those who requested for the documents to be mailed for them could choose between regular print, braille, large print, or audio format.
Prior to their session, measures were taken to ensure that the participants did not experience any barrier to participation.
The full list of documents tested in the study is presented in the following table.
A total of 44 one-on-one depth interviews were conducted with participants across Canada. Among the 44 completed interviews, 9 were in French and the remaining 35 were in English.
The participant breakdown for each target audience is detailed in the table below:
| Target audience | Number of interviews |
|---|---|
| Audience 1: Very low or no vision | 12 |
| Audience 2: Hearing and visual disability | 8 |
| Audience 3: A physical disability | 12 |
| Audience 4: Persons with a neurodiverse disability | 12 |
| TOTAL: | 44 |
| Date | Region | Interview language | Documents Reviewed |
|---|---|---|---|
| August 23, 2022 | New Brunswick | English | 1A, 2A, 3A |
| August 25, 2022 | Quebec | French | 3B, 4B, 1C |
| August 25, 2022 | Ontario | English | 2C, 3C, 4C |
| August 26, 2022 | Alberta | English | 1A, 2A, 3A |
| August 31, 2022 | Ontario | English | 2C, 3C, 4C |
| September 2, 2022 | New Brunswick | English | 3B, 4B, 1C |
| September 6, 2022 | Quebec | French | 1A, 2A, 3A |
| September 7, 2022 | New Brunswick | English | 4A, 1B, 2B |
| September 7, 2022 | Ontario | English | 4A, 1B, 2B |
| September 9, 2022 | Alberta | English | 3B, 4B, 1C |
| September 14, 2022 | Quebec | French | 1A, 2A, 3A |
| September 14, 2022 | Alberta | English | 2C, 3C, 4C |
| Date | Region | Interview language | Documents Reviewed |
|---|---|---|---|
| August 26, 2022 | Nova Scotia | English | 4A, 1B, 2B |
| August 26, 2022 | Ontario | English | 1A, 2A, 3A |
| August 31, 2022 | British Columbia | English | 3B, 4B, 1C |
| September 6, 2022 | New Brunswick | English | 2C, 3C, 4C |
| September 7, 2022 | Ontario | English | 1A, 2A, 3A |
| September 7, 2022 | British Columbia | English | 4A, 1B, 2B |
| September 14, 2022 | Ontario | English | 3B, 4B, 1C |
| September 23, 2022 | Ontario | English | 1A, 2A, 3A |
| Date | Region | Interview language | Documents Reviewed |
|---|---|---|---|
| August 22, 2022 | Ontario | English | 1A, 2A, 3A |
| August 22, 2022 | Quebec | French | 4A, 1B, 2B |
| August 29, 2022 | New Brunswick | English | 1A, 2A, 3A |
| August 30, 2022 | Alberta | English | 4A, 1B, 2B |
| September 1, 2022 | New Brunswick | English | 3B, 4B, 1C |
| September 2, 2022 | Quebec | French | 3B, 4B, 1C |
| September 8, 2022 | Ontario | English | 2C, 3C, 4C |
| September 12, 2022 | Nova Scotia | English | 1A, 2A, 3A |
| September 13, 2022 | Ontario | English | 2C, 3C, 4C |
| September 16, 2022 | British Columbia | English | 3B, 4B, 1C |
| September 29, 2022 | Ontario | English | 4A, 1B, 2B |
| October 4, 2022 | Quebec | French | 2C, 3C, 4C |
| Date | Region | Interview language | - |
|---|---|---|---|
| August 23, 2022 | Ontario | English | 1A, 2A, 3A |
| August 25, 2022 | Manitoba | English | 4A, 1B, 2B |
| August 25, 2022 | Quebec | French | 4A, 1B, 2B |
| August 29, 2022 | British Columbia | English | 2C, 3C, 4C |
| August 29, 2022 | Quebec | French | 4A, 1B, 2B |
| August 29, 2022 | Ontario | English | 3B, 4B, 1C |
| August 30, 2022 | Newfoundland | English | 3B, 4B, 1C |
| September 1, 2022 | Quebec | French | 1A, 2A, 3A |
| September 1, 2022 | Alberta | English | 2C, 3C, 4C |
| September 9, 2022 | Alberta | English | 2C, 3C, 4C |
| September 12, 2022 | Nova Scotia | English | 1A, 2A, 3A |
| September 15 | New Brunswick | English | 3B, 4B, 1C |
Qualitative research seeks to develop insight and direction rather than quantitatively projectable measures. The purpose is not to generate “statistics” but to hear the full range of opinions on a topic, understand the language participants use, gauge degrees of passion and engagement and to leverage the power of the group to inspire ideas. Participants are encouraged to voice their opinions, irrespective of whether or not that view is shared by others.
Due to the sample size, the special recruitment methods used, and the study objectives themselves, it is clearly understood that the work under discussion is exploratory in nature. The findings are not, nor were they intended to be, projectable to a larger population.
Specifically, it is inappropriate to suggest or to infer that few (or many) real world users would behave in one way simply because few (or many) participants behaved in this way during the sessions. This kind of projection is strictly the prerogative of quantitative research.
| Region | # of interviews | Language | Dates |
|---|---|---|---|
| Atlantic Canada | 12 | English | TBD |
| Ontario | 12 | English | TBD |
| Western Canada | 12 | English | TBD |
| Quebec | 12 | French | TBD |
| Total: | 48 |
Hello/ Bonjour, my name is [NAME]. I am calling from Quorus Consulting Group, a national public opinion research company. We're conducting a research study on behalf of the Government of Canada with people in your area. Would you prefer to continue in English or French? / Préférez-vous continuer en anglais ou en français?
As I was saying - we are conducting research on behalf of the Government of Canada to get a better understanding of the service experience of people with different types of disabilities or medical conditions when they access various forms and guides.
Participation is completely voluntary and your decision to participate or not will not affect any relationship you may have with the Government of Canada. We are interested in your opinions and your experiences. No attempt will be made to sell you anything and at no point will you be asked to share personal information. We are not asking you to participate right now - at this stage we are simply reaching out to invite Canadians to participate in interviews scheduled from August to September. People who take part will receive a cash gift to thank them for their time. All your opinions will remain anonymous and confidential and will be used for research purposes only in accordance with laws designed to protect your privacy.
*IF AN INDIVIDUAL QUESTIONS THE VALIDITY OF THE RESEARCH, INVITE HIM/HER TO CALL/EMAIL OR GO ONLINE:
1. INSTRUCT THEM TO CALL OR EMAIL KRISTA HOMES AT KRISTA.HOLMES@CRA-ARC.GC.CA OR 343-551-6112.
2. TO VERIFY IT IS A GOVERNMENT OF CANADA PROJECT, THEY CAN SEARCH THE FOLLOWING WEBSITE TO SEE LIST OF CURRENT PUBLIC OPINION RESEARCH PROJECTS - INCLUDING THIS ONE: "ACCESSIBILITY TESTING OF CRA'S FORMS & GUIDES" EN - http://canada.ca/por-cra FR - http://canada.ca/arc-por
**IF ASKED, SAY: The information collected through the research is subject to the provisions of the Privacy Act, legislation of the Government of Canada, and to the provisions of relevant provincial privacy legislation. In addition to protecting your personal information, the Privacy Act gives you the right to request access to and correction of your personal information. You also have the right to file a complaint with the Privacy Commissioner of Canada if you think your personal information has been handled improperly.
1. EXPLAIN INTERVIEWS. We will be conducting a series of 1-on-1 interviews, by telephone or online, all of them randomly recruited just like you. The discussion will last no more than 60 minutes and can be scheduled at a time convenient for you. Participants in the interviews will receive an honorarium of $250 for their time. But before we invite you to attend, we need to ask you a few questions to ensure that we get a good mix and variety of people. May I ask you a few questions? This will take about 5 minutes.
| Yes | 1 | |
| No | 2 | ASK IF ANYONE ELSE IN THE HOUSEHOLD MIGHT BE INTERESTED IF NOT, THANK/ DISCONTINUE |
2. Do you or does anyone in your immediate family or household work in any of the following areas? [READ LIST]
| - | Yes | No |
|---|---|---|
| a) A marketing research firm | 1 | 2 |
| b) A magazine or newspaper, online or print | 1 | 2 |
| c) A radio or television station | 1 | 2 |
| d) A public relations company | 1 | 2 |
| e) An advertising agency or graphic design firm | 1 | 2 |
| f) An online media company or as a blog writer | 1 | 2 |
| g) The government, whether federal, provincial or municipal | 1 | 2 |
| h) A tax preparation company | 1 | 2 |
| i) Accounting | 1 | 2 |
IF “YES” TO ANY OF THE ABOVE, THANK AND TERMINATE
3. Through this research, the Canada Revenue Agency wants to get a better understanding of the service experience of people with different types of disabilities or medical conditions when they access various forms and guides. Do you have any of the following…
| - | Yes | No |
|---|---|---|
| a ) A physical disability which may make it difficult for you to read text or complete forms without assistance - [READ AS NEEDED: This may include mobility, flexibility and dexterity and (prolonged) pain issues, such as (but not limited to) the following: Absent limb/reduced limb function, Arthritis, osteoarthritis, rheumatoid arthritis, Cerebral palsy, Epilepsy, Multiple Sclerosis (MS), Muscular Dystrophy (MD), Motor Neurone Disease (MND), Spinal cord injury, Paraplegia] | 1 | 2 |
| b) A seeing disability - [READ AS NEEDED: also known as visual impairment, it affects a person's ability to see - even when wearing glasses or contact lenses] | 1 | 2 |
| c) A hearing disability - [READ AS NEEDED: also known as deaf or hard of hearing, it affects a person's ability to hear - even when choosing to use devices like hearing aids or cochlear implants] | 1 | 2 |
| d) A cognitive disability or neurodiversity which may make it difficult for you to read text or complete forms without assistance - [READ AS NEEDED: This may include learning, developmental, memory or mental health-related disabilities, such as (but not limited to) the following: Aphasia (difficulty finding words, writing, or understanding language), Autism, Attention deficit disorder, Brain injury, Dementia, Developmental (down syndrome), Dyslexia, Intellectual loss, Learning disabilities] | 1 | 2 |
| e) Do you have any other disabilities or medical reasons which may make it difficult for you to read text or complete forms without assistance: [READ AS NEEDED: This may include injuries, cancer, a concussion, mental health, etc.] | 1 | 2 |
IF “NO” TO ALL, THANK AND TERMINATE
IF ASKED WHY THEY ARE BEING ASKED ABOUT THEIR MEDICAL CONDITION OR DISABILITY, SAY: Through this research, the Canada Revenue Agency wants to get a better understanding of the service experience of people with different types of disabilities or medical conditions when they access various forms and guides. Your feedback will help support the development of better accessible products.
RECRUIT THOSE WITH A DISABILITY OR MEDICAL CONDITION IN THE BELOW CATEGORIES.
4. Have you ever requested a Canada Revenue Agency form or guide in any of the following alternate formats for your own personal use?
| Digital audio (MP3) – Content is put onto a digital storage media and can be listened to as an audio file | 1 |
| Electronic text (Etext) – Content is put onto a digital storage media and can be read to you with screen reading software. | 2 |
| Braille – Content is printed on paper with raised dots. | 3 |
| Large print – Content is printed in large text (18-point type) | 4 |
| None of the above | 9 |
5. In order to use or complete a Canada Revenue Agency form or guide on a computer or in hardcopy format, would you require any of the following for assistance…?
| - | Yes | No |
|---|---|---|
| a) A screen reader | 1 | 2 |
| b) Voice recognition software | 1 | 2 |
| c) A hearing aid | 1 | 2 |
| d) Adapted equipment such as an adapted mouse, joystick, keyboard or other type of equipment adapted to your disability which would help you navigate websites, complete online forms, etc. | 1 | 2 |
| e) A cognitive aid [READ AS NEEDED: This may include computer or electrical assistive devices, to help people with memory, attention, or other challenges in their thinking skills] | 1 | 2 |
| f) Assistance from a caregiver | 1 | 2 |
| g) Assistance from a family member or friend | 1 | 2 |
| h) Other adaptation, assistive device or technology (please specify _______) | 1 | 2 |
IF “NO” TO ALL, THANK AND TERMINATE
6. The next few questions will ensure that we get a good mix and variety of people in terms of age, gender, so on and so forth. What is your gender identity? [If you do not feel comfortable disclosing, you do not need to do so] [DO NOT READ LIST]
| Male | 1 |
| Female | 2 |
| Non-binary | 3 |
| Other gender identity – please describe: _____ | 4 |
| Prefer not to say | 9 |
AIM FOR 50/ 50 SPLIT OF MALE AND FEMALE, WHILE RECRUITING OTHER GENDER IDENTITIES AS THEY FALL
7. We are looking to include people of various ages in this study. May I have your age please?
RECORD AGE: ______________
RECRUIT A MIX ACROSS INTERVIEWS
8. Do you currently live in… [READ LIST]
| A city or metropolitan area with a population of at least 100,000 | 1 |
| A city with a population of 30,000 to just under 100,000 | 2 |
| A city or town with a population of 10,000 to just under 30,000 | 3 |
| A town or rural area with a population of less than 10,000 | 4 |
RECRUIT ~2 PARTICIPANTS PER REGION WHO LIVE IN A CITY OR TOWN WITH A POPULATION OF NO MORE THAN 30,000 (Q8 = 3 or 4)
9. In which city/town/village and in which province/territory do you currently reside in?
RECORD: ___________
10. We want to make sure we speak to a diversity of people. Do you identify as any of the following?
| An Indigenous person (First Nations, Inuit or Métis) | 1 |
| A member of an ethnocultural or a visible minority group other than an Indigenous person [IF NEEDED: Ethnocultural means you identify as a member of a particular ethnic group] |
2 |
| None of the above | 3 |
TRY TO RECRUIT MEMBERS OF VISIBLE MINORITIES AND APPROXIMATELY 3 MEMBERS OF INDIGENOUS COMMUNITIES ACROSS ALL INTERVIEWS
11. [ASK ONLY IF Q10 = 2] What is your ethnic background? RECORD ETHNICITY: ________
12. What is your current employment status?
| Working full-time | 1 |
| Working part-time | 2 |
| Self-employed | 3 |
| Retired | 4 |
| Currently not working | 5 |
| Student | 6 |
| Other | 7 |
| Prefer not to say | 8 |
RECRUIT A MIX
13. What was your approximate household income before tax for 2021? Was it…[READ LIST]
| Under $20,000 | 1 |
| Between $20,000 and just under $60,000 | 2 |
| Between $60,000 and just under $100,000 | 3 |
| Over $100,000 | 4 |
| Prefer not to say | 9 |
RECRUIT A MIX
14. Could you please tell me what is the highest level of education that you have attained?
| Some high school only | 1 |
| Completed high school | 2 |
| Some college/university | 3 |
| Completed college/university | 4 |
| Prefer not to say | 5 |
RECRUIT A MIX
15. For this study, we are inviting Canadians to an interview to share their experiences and opinions. The format is a one-on-one interview which can be conducted either by telephone or online using an online application called Zoom. The interview would be led by a Quorus research professional. The session would last 1 hour, and participants would receive $250 for their time and effort. Would you be able to participate in this type of interview?
IF NEEDED: If you would typically need the help or support of someone else for this type of activity, they are welcomed to join you. An additional incentive of $125 would be offered.
| Yes | 1 |
| No | 2 |
16. Would you prefer doing this type of interview over the telephone or would you prefer using an online video conference platform like Zoom?
| Telephone | 1 |
| Web-based | 2 |
17. Do you require any other accommodations or assistance in order to participate in your interview? Please note that you will be asked to read text or write out answers during the discussion.
| Yes – please specify: ____________________________ | 1 |
| No | 2 |
| Don’t know/Prefer not to say | 9 |
18. I would like to invite you to participate in a one-on-one interview with a researcher from the national public opinion research firm, Quorus Consulting. The session will be audio-recorded but your participation will be confidential. We have openings on the following dates and times [PROVIDE OPTIONS AND SELECT ONE OPTION]. It will last 60 minutes, and you will receive $250 to thank you for your time.
Would you still be interested and available to take part in this study?
| Yes | 1 | |
| No | 2 | THANK & TERMINATE |
19. The interview will involve sharing with you three CRA forms or guides so that during the interview, you can let the researcher know which aspects of those forms and guides meet your accessibility needs and which ones do not. There will be no need for you to complete the entire forms or read the entire guides ahead of time. As well, you can use fictitious information when completing the forms. Given your usual approach to CRA forms and guides, would you prefer we email these to you or would you prefer hardcopies. We recommend you select the approach you would prefer to use under normal circumstances.
| 1 | |
| 2 |
20. [ASK IF “MAIL” SELECTED IN Q19] Materials are available in the following formats: regular print, in Braille, in large print or in audio format. If you prefer an audio format, we can either email you an audio file or we can mail you a CD. Which format would you prefer we send you?
| Regular print | 1 |
| Braille | 2 |
| Large print | 3 |
| Audio format - email | 4 |
| Audio format – CD | 5 |
21. The interview will be recorded. These recordings are used to help with analyzing the findings and writing the report. The results from the interviews will be grouped together in the research report, which means that individuals will not be identified in anyway. Is this acceptable?
| Yes | 1 | |
| No | 2 | THANK & TERMINATE |
22. Individuals from the Canada Revenue Agency (CRA) and/ or the Government of Canada involved in this research project may be observing the session. They will not take part in the discussion, and they will not know your full name. Is this acceptable?
| Yes | 1 | |
| No | 2 | THANK & TERMINATE |
23. Thank you. Just to make sure, your interview will take place on [DAY OF WEEK], [DATE], at [TIME] and it will last 60 minutes. Following your participation, you will receive $250 to thank you for your time. You will need to be in a place that is quiet and free of distractions for the duration of the interview. This includes ensuring to the extent possible, you are able to be on your own, without pets, children or other people nearby, and in a room in your home or office that is as quiet as possible. You will not be able to take part in the interview from an outdoor area, a vehicle, or a public place, as these locations are too noisy. Are you interested and available to attend?
| Yes | 1 | |
| No | 2 | THANK & TERMINATE |
As we are only inviting a small number of people to this research study, your participation is very important to us. If for some reason you are unable to participate or you have technical issues preventing your participation, please call so that we may either help you, or get someone to replace you – you cannot choose your own replacement if you cannot attend. You can reach us at [INSERT NUMBER] at our office. Please ask for [INSERT NAME].
So that we can contact you to remind you about your interview or in case there are any changes, can you please confirm your name and contact information for me? [READ INFO AND CHANGE AS NECESSARY.]
[IF PARTICIPANT HAS REQUESTED HARDCOPIES OF MATERIALS] We will also need your mailing address in order to send you the hardcopy materials prior to your interview.
First name___________________________________________
Last name____________________________________________
Email (OPTIONAL)_____________________________________
Day time phone number________________________________
Night time phone number______________________________
Mailing address_____________________________________
_______________________________________
Thank you!
If the respondent refuses to give his/ her first or last name or phone number please assure them that this information will be kept strictly confidential in accordance with the privacy law and that it is used strictly to contact them to confirm their attendance and to inform them of any changes to the interview. If they still refuse THANK & TERMINATE.
BEFORE STARTING THE INTERVIEW, THE MODERATOR WILL CONFIRM THE FOLLOWING WITH THE PARTICIPANT:
Thank you for joining this interview!
INTRODUCTION: Please tell me your name and a little bit about yourself, such as where you live, who lives with you, what you do for a living, etc. and since this research is all about accessibility, briefly describe the disabilities or medical conditions you have to the extent you feel comfortable sharing.
Our discussion today will only focus on CRA forms and guides, so we stay focused on the key research objectives. Generally speaking, the most common types of CRA “forms” are what you would need to fill out to apply for certain types of benefits or tax credits. Forms are also used when completing a tax return. CRA “guides” provide information, definitions and explanations to help those who are filling out CRA forms.
We will be talking about your experiences completing CRA forms and guides.
[SET UP]
Let's now shift gears to review a few CRA forms and guides so we can discuss whether they meet your accessibility requirements.
You will recall that you were sent some forms and guides prior to this interview.
[MODERATOR TO INTRODUCE FIRST FORM/GUIDE]
Let's start with the form/guide titled [INSERT DOCUMENT TITLE].
| FOR GUIDES | FOR FORMS |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
[REPEAT QUESTIONS WITH FOR ALL FORMS/GUIDES]
Thanks again! The team that invited you to participate in this session will contact you regarding how you can receive the incentive we promised you.
Thank you – have a nice evening!