Prepared for the Administrative Tribunals Support Service of Canada, Social Security Tribunal Secretariat
Supplier Name: Kelly Sears Consulting Group and Narrative Research
Contract Number: CW2307292
Contract Value: $ 38 950.49
Contract award date: May 30, 2023
Delivery date: November 13, 2023
Registration Number: POR 011-23
For more information on this report please contact the Social Security Tribunal Secretariat at:
NC-SST-TSS-OUTREACH-RELATIONS-GD@canada.gc.ca
Ce rapport est aussi disponible en français.
End-User Interviews on the Navigator Service in the Appeal Division
Final Report
Prepared for the Administrative Tribunals Support Service of Canada, Social Security Tribunal Secretariat
Supplier Name: Kelly Sears Consulting Group and Narrative Research
November 2023
This public opinion research report presents the results of in-depth interviews conducted with end-users of the Navigator Service at the Appeal Division of the Social Security Tribunal of Canada. An evaluation of end-user experience was sought to identify and address any performance issues there may be. The research involved a total of 20 in-depth interviews, conducted by telephone, with a cross-section of types of former end-users of the Navigator Service. The fieldwork was conducted between July 28 and August 28, 2023.
Cette publication est aussi disponible en français sous le titre:
Entrevues avec les utilisatrices et les utilisateurs finaux du service d’accompagnement à la division d’appel
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from the Administrative Tribunals Support Service of Canada. For more information on this report, please contact the Social Security Tribunal of Canada at: info.sst-tss@canada.gc.ca.
Catalogue Number:
Em27-4/2024E-PDF
International Standard Book Number (ISBN):
978-0-660-69442-9
Related publications (registration number:
Catalogue Number Em27-4/2024F-PDF (Final Report, French)
ISBN 978-0-660-69443-6
© His Majesty the King in Right of Canada, 2023
Narrative Research Inc. and Kelly Sears Consulting
Contract Number: CW2307292
Contract Award Date: 2023-05-30
Contracted Cost: $38,950.49
The Administrative Tribunals Support Service of Canada (ATSSC) provides support services and facilities to twelve federal administrative tribunals through a single, integrated organization. The Social Security Tribunal of Canada (SST) is one of the tribunals supported by the ATSSC. The SST makes decisions on appeals related to employment insurance benefits and income security benefits (e.g., Canada Pension Plan, Old Age Security). The SST has two levels of appeal: the General Division (GD) and the Appeal Division (AD).
The SST Secretariat launched the Navigator Service in the Appeal Division in November 2020. After more than two years in operation, the SST wished to study the performance of the service by collecting feedback from key players, including end-users. This research will provide the SST with the user feedback needed to identify and address performance issues from the perspective of key actors in the appeal system. The SST will apply the results to inform any changes to service delivery and internal processes.
More specifically, areas of investigation that were part of the study included:
The investigation included 20 in-depth interviews conducted by telephone from July 28 to August 28, 2023 with a cross-section of types of end-users. The SST conducted the initial outreach to end-users in the spring of 2023 to assess their interest in taking part. The resulting list included 32 contacts, from which the research team scheduled interviews with 20 end-users. The selection was made to ensure a good cross-section of regions, official languages, and navigators. While a preference was given to include those with most recent interactions for best recall, interviews covered the last two years of the service to minimize recency biases .
This report presents the findings from the end-user interviews. Caution must be exercised when interpreting the results, as qualitative research is intended to be directional only. Results cannot be attributed to the overall population under study, with any degree of confidence.
I hereby certify as a Representative of Narrative Research that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Directive on the Management of Communications. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.
Signed
Margaret Brigley, CEO & Partner | Narrative Research
Date: September 11, 2023
The Navigator Service is well understood by end-users as a service that provides direction or instructions to assist them in preparing their appeal. Specifically, end-users have a clear understanding that the navigator is able to explain the appeal process, describe what will happen during the hearing, guide claimants to online resources that provide information or support, and remain available to answer questions. There is also a clear understanding of the Navigator Service’s limitations, namely, that the navigator is unable to provide legal advice or personal opinions, assist in preparing the arguments, attend the hearing and represent claimants, or provide a referral for representatives, organizations, or documents to be used.
Nonetheless, claimants’ needs go beyond what the Navigator Service is currently offering, which leads many end-users to be unhappy with their experience. When asked to rate their satisfaction with the Navigator Service, about half of those interviewed gave a negative score, while the other half rated their experience positively. Satisfaction primarily stems from the personal interactions with the navigator, whose professionalism and personable approach often exceeded end-users’ expectations. In addition, participants attributed satisfaction to the Navigator Service having delivered on its promise.
That said, the Navigator Service fails to meet claimants’ needs for guidance and support to ensure they are well prepared for the hearing. Specifically, they would like the navigator to provide a more in-depth description of the appeals process, including what will happen during the hearing. Claimants’ experience with the informality of their hearing at the General Division does not prepare them for the formality and legal requirements of the Appeal Division hearing. As such, there is a desire for better explanation of the legal terminology and the inclusion of what to expect at the hearing, instructions on how to prepare and present a strong argument, and guidance in choosing the most relevant court cases, laws and regulations to prepare the appeal. The inability for the navigator to act as an advisor, and claimants feeling ill-prepared for the hearing (especially when finding this out at the hearing), are by far the main sources of dissatisfaction that participants have with the Navigator Service.
The level of contact with the navigator varied across end-users interviewed. That said, all received an introductory call from the navigator explaining their role, and all were called again a couple of weeks prior to the hearing with reminders. Some were in more frequent contact with the navigator during the preparation of their appeal, as needed. This level of contact was viewed as adequate given the scope of the Navigator Service, and end-users understood that they could contact the navigator at any point in time during the process.
Navigator accessibility is also adequate to meet the needs of service users. With a few exceptions, the navigator was easy to reach, and offered flexible communication methods to meet the needs of end-users. While returned calls were promptly made, a couple of claimants would have liked the ability to immediately access a navigator for assistance when reviewing information. As such, the SST could explore the possibility to offer a complementary service for immediate response to process questions (e.g., live chat) thus allowing end-users to keep their momentum when preparing their appeal.
Navigator Service users generally felt that sufficient time was provided for them to prepare their appeal. They generally appreciated that the navigator provided information to help direct their efforts, including basic information about the appeal process, reference for documentation or support (e.g., court decisions, legal aid organizations, acts and legislation), and reminders of important milestones (e.g., notifications of documents being sent, reminders about the hearing).
While a general description of the appeal process was provided by the navigator (broad description of the hearing process, parties attending and their role, and documents required, and the type of questions that may be asked), there appears to be a gap in helping claimants to effectively prepare, structure and present their arguments. As such, providing service-users with a more in-depth explanation of the legal process and terminology may improve the end-user experience, and should be considered by the SST. This could include guiding claimants to the relevant court cases and legislation, or at least helping them search the database, and providing instructions on how to structure and present their arguments during the hearing. In fact, identifying the most relevant court cases and legislation was consistently mentioned by end-users as time-consuming and difficult.
Service users are very complimentary of the navigators. Throughout the process, they generally felt respected, listened to, and supported within the scope of the program. The navigators were often lauded for their professionalism, honesty, helpfulness, personable approach, politeness, patience, friendliness, and for being open and honest, empathetic and easy to talk to. They were also seen as well-spoken, but somewhat guarded in their assistance, due to the limitation of their role. Although accommodations were not required by service users interviewed, all felt that the navigator would have been flexible and accommodating if needed. The navigators’ abilities to make end-users feel cared for was consistently the most appreciated aspect of the interactions. By contrast, the limitation in the level of support provided by the navigator given their role, was most often mentioned as the least appreciated aspect of interactions.
One of the goals of the Navigator Service is to ensure that claimants feel prepared and confident for the hearing. About half of claimants felt prepared and confident on the day of the hearing, based on the information they had collected and the strength of their argument. Those who had a favourable decision at the General Division hearing, but then the decision was challenged by the Minister, were also most confident about the Appeal Division hearing. The other half of claimants felt nervous and insecure, primarily as they did not know what to expect from the hearing, or for not having been able to find required information (e.g., relevant court cases, laws and regulations that apply to their situation).
Despite their initial sentiment prior to the hearing, claimants were surprised with the formality and structure that defined the Appeal Division’s hearing, which significantly affected their level of confidence during the hearing itself. As such, there is a need for the Navigator Service to provide more information about the hearing to raise claimants’ level of confidence, not only in the hearing, but also in ensuring their perception that the tribunal provided an opportunity to be well represented, and a fair chance to win their appeal. More time spent on preparing claimants for the hearing, through discussion and practices, may also provide a greater sense of confidence among claimants.
Findings from this research suggest that there would be value in the SST reviewing the Navigator Service to see how it could improve the level of support provided. This could include the development of additional tools to describe the appeal process, as well as tips on how to prepare an argument. Attention should also be afforded to describing the purpose of the Appeal Division hearing, clearly outlining the hearing process, as well as informing claimants as to how tribunal decisions are made. Consideration should also be given to expanding the role of navigators to provide individualized support, helping claimants source the right information and documents, and guiding them on the preparation of their appeal based on each individual case. Further, there is merit in exploring the possibility of making the Appeal Division less formal and more accessible to the claimant. Finally, while the service provided by navigators is well received, there is merit in implementing a service that could offer immediate response to questions, in instances where the navigator is not available (e.g., live chat service).
The Administrative Tribunals Support Service of Canada (ATSSC) provides support services and facilities to twelve federal administrative tribunals through a single, integrated organization. Examples of these support services include registry, research, human resources, and evaluation. The Social Security Tribunal of Canada (SST) is one of the tribunals supported by the ATSSC. The SST makes decisions on appeals related to employment insurance benefits and income security benefits (e.g., Canada Pension Plan, Old Age Security). The SST has two levels of appeal: the General Division (GD) and the Appeal Division (AD).
Recent independent, external studies conducted by the Faculty of Law, University of Windsor, and from the Department of Political Science at the Université Laval, highlighted the fact that feedback from end-users can provide important information that other approaches (e.g., internal research) may miss. The independent study focused on the use of the Navigator Service for Canada Pension Plan-Disability appeals heard by the Income Security – General Division of the SST. The SST Secretariat launched this service in the Appeal Division in November 2020. This is the first attempt to study the performance of the service after over 24 months of operation. The purpose of this study was to understand the effectiveness of the SST Navigator Service in enhancing end-users’ access to justice. The SST will apply the results to inform any changes to service delivery and internal processes.
Interviewing former end-users will provide the SST with the user feedback needed to identify and address performance issues from the perspective of key actors in the appeal system. The interviews were required to capture information on the following research issues:
From a database of all navigator service recipients, the SST created a list, selected primarily based on caseload characteristics, including those who received a Tribunal decision in the last two years, and those who did not have any outstanding matter before the SST. The initial outreach to end-users was conducted by the SST to determine interest in taking part in the research and obtain consent for passing on the end-user contact information to Kelly Sears Consulting Group and Narrative Research. The list provided by SST included the names of 32 contacts. For each claimant, information provided to the consultants was limited to their name, email address, telephone number, province/region, preferred language, decision date, and the name of the navigator they dealt with.
Kelly Sears Consulting Group and Narrative Research scheduled and conducted a total of 20 in-depth semi-structured interviews by telephone, with each lasting approximately 45 minutes. In choosing who to approach, a random selection was made based on achieving a good cross-section of regions (providing coverage for Ontario, British Columbia, Quebec, and the rest of Canada), to include diversity in Canada’s official languages (within the limits of the sample list), and to ensure a diversity of navigators were represented, where possible. While the recency of end-users’ experience was also a selection factor for best recall, interviews covered the last two years of the service to minimize recency biases. Note that due to the non-probability sampling approach, the limited sample size, and the use of a qualitative methodology, the correlation between satisfaction and the type of claim, hearing decision, or navigator cannot be inferred.
Based on the sample file and end-users’ preferences, 17 interviews were conducted in English, while 3 interviews were in French. Interview scheduling was done by email and telephone contacts and interviews were scheduled at a time convenient to each end-user. An incentive of $100 was offered to each participant in appreciation for their time, paid by cheque or e-transfer, as preferred.
In consultation with the SST, Kelly Sears Consulting Group and Narrative Research designed a recruitment script to invite participants and schedule interviews (provided in Appendix A), in addition to a discussion guide that addressed the research objectives (provided in Appendix B). An outline of discussion topics was provided to end-users in advance of the interview to allow them to prepare. Only personal end-user data deemed pertinent for the study was collected.
The following provides an overview of the demographic profile of those interviewed, as self-identified during the interview.
| Age group | Results |
|---|---|
| 18 to 24 years old | 1 |
| 25 to 34 years old | 1 |
| 35 to 44 years old | 7 |
| 45 to 54 years old | 4 |
| 55 to 64 years old | 1 |
| 65 years old or older | 4 |
| Prefer not to answer | 2 |
| Gender | Results |
|---|---|
| Female | 9 |
| Male | 8 |
| Gender diverse | 0 |
| Prefer not to answer | 3 |
| Form of education | Results |
|---|---|
| Less than a high school diploma or the equivalent | 0 |
| High school diploma or the equivalent | 1 |
| Registered apprenticeship or other trades certificate or diploma | 3 |
| College, CEGEP, or other non-university certificate or diploma | 6 |
| University certificate or diploma below bachelor’s level | 2 |
| Bachelor's degree | 3 |
| Postgraduate degree above bachelor’s level | 1 |
| Unsure/Prefer not to answer | 4 |
| Total income | Results |
|---|---|
| Under $20,000 | 2 |
| $20,000 to under $40,000 | 2 |
| $40,000 to Under $60,000 | 2 |
| $60,000 to under $80,000 | 0 |
| $80,000 to under $100,000 | 1 |
| $100,000 to under $150,000 | 3 |
| $150,000 or more | 2 |
| Unsure/Prefer not to answer | 8 |
| Identity | Yes | No | Prefer not to answer |
|---|---|---|---|
| First Nations | 1 | 14 | 5 |
| Inuk (Inuit) | 0 | 15 | 5 |
| Métis | 0 | 15 | 5 |
| Indigenous – Other | 0 | 15 | 5 |
| a member of an ethno-cultural or visible minority group | 2 | 13 | 5 |
| a member of the LGBTQ2+ community | 0 | 15 | 5 |
| a person living with a disability | 4 | 11 | 5 |
| a member of an official language minority community (French-speaking communities outside Quebec and English-speaking communities in Quebec) | 0 | 15 | 5 |
Qualitative discussions are intended as moderator-directed, informal, non-threatening discussions with participants whose characteristics, habits, and attitudes are considered relevant to the topic of discussion. The primary benefits of individual (or group) qualitative discussions are that they allow for in-depth probing with qualified participants on behavioural habits, usage patterns, perceptions and attitudes related to the subject matter. This type of discussion allows for flexibility in exploring other areas that may be pertinent to the investigation. Qualitative research allows for more complete understanding of the segment in that the thoughts or feelings are expressed in the participants’ “own language” and at their “own levels of passion.” Qualitative techniques are used in evaluation research as a means of developing insight and direction, rather than collecting quantitatively precise data or absolute measures. As such, results are directional only and cannot be projected to the overall population under study.
Expectations of the Navigator Service are clear based on the description provided, although the service, as currently structured, does not entirely meet the needs of claimants. As such, satisfaction with the Navigator Service is mixed, although end-users are highly pleased with their interactions with the navigator.
When asked how the Navigator Service was explained to them, nearly all participants felt that the description provided a clear understanding of the service. The program is initially introduced in an email received from the SST advising claimants of the navigator assigned to their appeal. The navigator then follows up with an introductory telephone call, during which the service is explained, including any limitations in the role of the navigator.
“It was pretty clear in terms of what they could and couldn't do.”
Based on the information provided by the SST, claimants’ first impression of the Navigator Service is that it provides support to individuals who do not have legal representation by guiding them through the process of preparing for their appeal. Most notably, claimants were initially left with the understanding that the navigator would:
“I felt like [the navigator] wasn’t there to really help, but just to explain things. [They were] good at giving me information but [they] did not help as much as I wanted. I was more on my own with that kind of things. [They] did do a great job providing the information though.”
Nearly all claimants recalled having been informed of the limitations of the Navigator Service, understanding that it needs to remain neutral. Specifically, they were left with the impression that the navigator could not:
“[The navigator] gave me a list of the documentation that was sent to me and the date they were sent to me and what I needed to bring to the hearing. That is what I expected of them because they outlined their role at the time.”
That said, a couple of claimants were left with the impression that the navigator would review their file, once finalized, and provide feedback on their arguments. While one of those claimants recalled the navigator having offered the help, the other appellant was left with this impression from the Navigator Service being described as offering ‘help’ to claimants. Of note, neither appellant was actually advised by the navigator (according to those interviewed), which suggests there may have been a misunderstanding on the part of the end-users.
Despite understanding the Service’s limitations, many of the claimants would have liked to have received additional support and counsel, to help them prepare for the appeal. While they realize that the SST is unable to provide legal representation, and that it is not the role of the navigator to prepare the arguments, there is an expressed need for assistance in better understanding the legal structure of the appeal process. Claimants mentioned having been informed of the appeals process, although they would have liked the navigator to have provided more detailed information in some cases, or counsel and personal advice to assist in preparing for the appeal. As such, many claimants would have liked the navigator to:
“I was expecting that [the navigator’s] role was to tell me if I was doing the right thing or not, and to provide assistance or judgements on what was worth bringing up [at the hearing] and what wasn’t. And [I expected] more information about how the trial was going to go.”
“I was expecting the navigator to really hold my hand. I am just a civilian and a lot of these things I do not understand. I was looking for someone that would explain things to me, someone that knows the system and would help me navigate through it with my information. Things like how to present my case and what to do to be successful. I had to figure that out myself. The information was presented to me, but I had to read through and analyse and prepare. I did not have the understanding of how to navigate my way around it. [I would have liked the navigator to] point me to what’s important and what’s not important in terms of the type of information we need.”
It should be further mentioned that several claimants we spoke to mentioned that, as part of their role to support claimants in preparing for the hearing, they would have liked the navigator to more clearly explain to them how to prepare for the hearing, including identifying the information specific to their situation they would need to present. Some, as respondents, were under the impression that the purpose of the Appeal Division’s hearing was to provide a second opinion on their appeal presented at the General Division, rather than having to respond to formal arguments of the appellant at the Appeal Division. As such, many came to the hearing ill-prepared.
The difference in what is expected of the appellant at the Appeal Division hearing, compared to the General Division hearing was unclear to them. Further, the lack of proper explanation by the navigator regarding the format of the Appeal Division’s hearing left many claimants to expect a much more informal process than what they experienced.
“Their approach saying, they do not give advice or answer questions disappointed me. I asked [the navigator] if [my argument] was good enough and they could not tell me. I found out at the hearing that the information I prepared had nothing to do with my appeal.”
“The navigator did not play a major role with this process. They gave me the broad strokes to ‘this and that’, and ‘stay within these’, and told me that if I have any issues to get legal advice. They were limited in what they could do and even in instructing me. Again, as a self-represented person, it was problematic because I felt that I did not need legal advice but once I came to trial, I found out how poorly informed I was of the process.”
Four claimants recalled having dealt with a navigator to prepare their appeal at the General Division. Although having interacted with two different navigators (one at the General Division and one at the Appeal Division) did not pose any challenge, all four would have preferred to keep the same navigator for consistency and better rapport building.
As a summary of their experience, and after an in-depth discussion of various areas of the Navigator Service, claimants were asked to summarize their experience by providing a rating on a scale from 1 (not at all satisfied with the experience) to 10 (completely satisfied with the experience). While qualitative research does not provide the ability to measure opinions, it is interesting to note that the average score is just above the mid-point on the scale. Opinions are quite varied, with scores ranging from 1 to 10. One-half of the participants gave a score from 1 to 5, while the other half provided a score between 6 and 10. While this is not intended to measure opinions, it illustrates the broad range of end-user experiences.
For the most part, satisfaction stems from the availability of support for claimants who do not have legal representation. Just the fact that the service is offered is appreciated, regardless of end-users’ personal experience. Favourable ratings were also supported by having received courteous and efficient assistance (within the scope of the Navigator Service), and for the navigator’s professionalism, helpfulness, responsiveness, and attentiveness. In fact, the personable nature of the navigators was the only aspect of the Navigator Service that exceeded some claimants’ expectations.
“I am happy there were resources and help available that I was able to reach out to.”
“The general idea of it is good; the idea of being helped, because [the appeal] is a lot of money and a lot to deal with. Having the support was awesome and having [the navigator] check in once in a while to see how I am doing is great. Especially for people who can’t figure any of it out very easily.”
“[The navigator] was trying to help. They were trying to do their best and trying to be as helpful as possible. That is what I remember; that they were courteous and helpful or trying to be.”
“[The navigator’s] personality [exceeded my expectations]. They were great to deal with. They helped me feel less overwhelmed about the process. They were reassuring and they helped me understand.”
By contrast, dissatisfaction among those interviewed stems from the perception that the Navigator Service, as designed, provided little value in actually preparing for the hearing. This speaks to claimants’ expectations that advice and guidance should be provided, in addition to information and instructions.
“I don’t think that the navigator had the authority or the access to do more than what was done for me. My [negative] opinion is not a personal reflection on the navigator, but more so on their position.”
“[The navigator] did not provide enough clarity about the process to have me be as prepared as I should have been [for the hearing].”
“I did not get anything out of [the Navigator Service] in terms of advice or guidance.”
The level of contact with the navigator, their availability, and responsiveness are generally considered adequate given the purpose of the Navigator Service.
While the level of contact with the navigator varied, it was generally considered sufficient given the scope of the Navigator Service. Consistently, the navigator contacted the appellant by telephone at the beginning of the process to introduce themselves, describe their role, and provide contact information. The navigator also contacted the appellant within a couple of weeks of the hearing, to check-in on the preparation, and provide reminders regarding the hearing. Other communications varied based on the need of each appellant, with some having been in contact two or three other times in between. All end-users were aware that they could contact the navigator as needed, while preparing their appeal.
“It was just the right amount [of contact] but it could have been more if [the navigator was] able to guide me. They sent the information when needed and told me how to go about finding the documents and they said I could call them with questions.”
For the most part, the navigator was considered accessible and easy to reach. Where required, telephone meeting times were scheduled to align with the needs of claimants. Returned calls were promptly made, either within 24 or 48 hours, within service users’ expectations. That said, a couple of claimants were displeased with the inability to immediately access the navigator when needed, noting that being able to speak with someone for clarification while reviewing documentation is important. Given the complexity of the information or documents reviewed, claimants felt that having a response to their questions as they are reviewing documents would help them keep the momentum, and avoid having to refamiliarize themselves with the documents later when the navigator returns their call.
“They would get back to you eventually, but it was not timely enough. I would have expected a same-day response. That is important when you go through paperwork and that you are in it at the time. The momentum I was on at that time is lost and sometimes I don’t have the paperwork in front of me when they call back.”
One service user we spoke to mentioned that the navigator they dealt with did not have good command of the English language which caused communications and comprehension issues.
While there is enough time to prepare the appeal and basic information is provided by the navigator, there is an expressed desire for the navigator to provide more guided advice on documentation and resources available to claimants, despite that being outside of the scope of service.
During the time that their appeals took place and before recent legislative change, claimants who made an Employment Insurance claim or an income security claim had 45 days to prepare their arguments for the Appeal Division’s hearing. There is a consensus among claimants we spoke to that this is a reasonable amount of time to search for, assemble, and organize the required information and documentation for the appeal. While service-users generally felt that being allotted more time would not have made a difference to them, being provided with clearer instructions on how to prepare their appeal would have helped.
“It was lots of time, but I did not know what to prepare for.”
Claimants were asked what information they recalled the navigator provided to them. While some had a vivid recall of specific information to inform their feedback, others had a more vague recollection. Information provided could be grouped into three broad categories: information about the appeal process, references for documentation or support, and warnings or reminders of important milestones.
While a general description of the appeal process was provided, there appeared to be a gap in explaining how the hearing would unfold, which would help claimants understand how they could prepare for it. Although an explanation of the hearing was provided, the way the information was conveyed by the navigator, or understood by the appellant, appears to have differed. While some felt that the explanation was adequate to prepare them, others believed that it was insufficient.
In general, the overall process was explained in broad terms, including an indication of who would be in attendance and their respective role, what documents and information claimants needed to have on hand, and the types of questions that could potentially be asked to them. That said, it was suggested that the navigator spend more time guiding claimants through the process of preparing and presenting arguments, including how to structure them and how to present them (i.e., which one to introduce at which point during the hearing). It should be noted that two claimants we spoke to did not look for additional information when preparing their appeal, and relied on the file they had presented at the General Division hearing. Both were under the impression that their initial arguments would be sufficient.
“The navigator should explain what to expect of the process at the trial itself and also say, ‘hey this is where you bring up these certain things that are the main topics’ and ‘this is why’. I did not understand that this part of the trial was going to be judging the verdict of the last [appeal] and the reasoning for the verdict and not [discussing] my case. That was part of the confusion. I assumed this was an opportunity to go through the motions and explain my case again and not just fight the original verdict. I was expecting a re-trial and that is what I was preparing for.”
Moreover, claimants felt there is a gap in the navigator providing a more detailed explanation of what is expected of them, as well as practical advice on how to prepare for the hearing (e.g., what documentation to bring, how to present an argument, what information to reference, how to present the information, what questions might be asked). Further, while some felt that the navigator provided valuable assistance in understanding the legal terminology used throughout the process, others were left without sufficient explanation.
“I did get the information which was useful, but I did not fully understand to the extent of what was needed. The legal stuff was confusing; what arguments to put forward and how to present the information at the hearing.”
In terms of references, it was reported that the navigator consistently provided online links to resources that could be useful to claimants in preparing their hearing, including past court decisions available on the SST website, a list of legal aid organizations, and relevant laws and regulations. Based on their personal experience, most claimants believed that the role of the navigator should be broadened to provide more guided instructions on preparing an appeal, without necessarily providing legal advice.
“[The navigator] should have directed me to similar cases I could look at. And they should have been able to tell me what my chances are, or at least that I should be positive or not positive. At the Tribunal, they are full of tricks and I was not prepared.”
“The navigator should be helping in locating the right resources, not just telling me what the potential resources are. They should have been able and capable to understand which way I should be going. Not tell me, ‘go there and we will see’.”
Finally, many claimants recalled having been notified by the navigator in advance of documents being mailed or emailed to them, something that end-users found to be helpful. They also appreciated the reminder call to ensure they had received the information needed to join the hearing, tips on how to join, and the confirmation that they had received the SST email specifying the date, time and login information for the hearing. In a couple of instances, the navigator mailed printed documents to claimants who did not have access to a printer. One appellant also appreciated that the navigator was available to provide login assistance at the hearing when technology issues were experienced.
For the most part, claimants looked online for information to prepare their hearing, both on the SST website and by conducting general online searches. They reported that the navigator guided them to the SST website for a listing of previous decisions, in addition to guiding them to online information on legal aid services available to them. The navigator sometimes provided telephone numbers of resources within the Government of Canada, and in other instances, links to laws and regulations available online.
Although claimants appreciated having access to court decisions on the SST website, they consistently struggled to effectively search for past cases that resembled their own situation. Indeed, finding the relevant Tribunal cases proved problematic for many, as they needed to sift through the listings, one by one, without a clear understanding of what they were looking for. In only a few instances, the navigator provided direction on which key words to search for, but most claimants were left to determine how to search the database on their own. Likewise, a few experienced difficulties in identifying the relevant information from the Income Tax Act and would have appreciated guidance in that regard.
“I had to find everything on my own. I had to produce a number of documents. And they let the judge eat me up. I am an ordinary citizen and I tried to fight but there is no fight to have. I am sure other people went through this process but surely they would know where the links are if I asked [the navigator]. Instead of me spending days looking for the stuff, they could say I will give you several links [to specific court cases], here is where you could go and look for stuff.”
Navigators were seen as caring, professional, attentive, trusted, and personable, and their interactions were often described as one of the best aspects of the Navigator Service.
Claimants were quite complimentary of the way the navigator interacted with them. They generally felt respected, listened to, and supported within the guidelines of the programs. The navigators were consistently described as professional, honest, direct, open, helpful, personable, polite, patient, friendly, empathetic, and easy to talk to. They were also seen as well-spoken, but somewhat guarded in their assistance, due to the limitation of the role.
“I found that [the navigator] was nice but I felt a little brushed off kind of coming back to the one liner. Personality-wise they were nice. Very personable but not very helpful.”
“Polite, soft spoken, willing, never frustrated. However long it took, they took that time.”
“They were so nice, friendly, right on the ball. Zero complaint. Super friendly, super nice.”
While most felt that their navigator supported them to the best of their abilities, others felt the lack of advice or direction (due to the navigator’s restricted role) led them to perceive that the navigator did not provide adequate support. This perception was more a function of the limitations of the service, than the willingness of the navigator. Navigators generally elicited a sense of trust through their interactions with the claimants. That said, a couple of people were suspicious of the navigator’s guidance, given their inability to go beyond the ‘official line’ or guide end-users in their choices.
“I did not feel supported because they did not provide any direction.”
The claimants we spoke to did not feel they needed accommodations, and as such were unable to comment on the navigator’s willingness and ability to tailor their approach to specific needs. That said, many believed that if required, accommodations would have been provided. One appellant who did not have access to a printer appreciated that the navigator printed and mailed documentation to them. Another end-user who was on call at work appreciated the navigator’s flexibility in scheduling meeting calls accordingly.
Claimants often described the feeling that their navigator cared about their situation, despite their inability to provide more tailored assistance. This was consistently one of the most appreciated aspects of the relationship.
“They were just so friendly and easy going. You could ask anything. They were professional but understanding. They were amazing and very helpful and very quick.”
“Very friendly, they were easy to speak to and very personable. From the get-go it was like someone I knew for a while even if I never met them in my life.”
The inability to provide more personalized advice or assistance was consistently mentioned by claimants as what they liked least about the navigator.
“No direction from [the navigator] except to tell me to scroll through the information. They should have provided me with keywords or instructions on how to find the information quickly rather than me having to scan through the documents.”
Despite some level of confidence and preparedness for the hearing, the formality and structure of the hearing leaves the impression that claimants do not have a fair opportunity to present their appeal, pointing to a perceived gap in the Navigator Service.
One of the goals of the Navigator Service is to ensure that claimants feel prepared and confident for the hearing. About half of those we spoke to felt prepared and confident on the day of the hearing, based on the information they had collected and the strength of their argument. Those who had a favourable decision at the General Division hearing were also most confident about the Appeal Division hearing.
“I had my notes; I was ready and I felt prepared.”
“I felt very confident. I had gone through all of their documents and all of their bylaws, I had done my research and submitted several pages of documentation with case studies, and precedents where precedence had been set.”
The ability to participate in the hearing from the appellant’s home was appreciated by many, notably for being able to be in a comfortable and familiar environment, and the ability to easily access notes and documents they needed to reference during the discussion.
The other half of end-users we spoke to felt nervous and insecure during the hearing, primarily as they did not know what to expect from the hearing, or for not having been able to find the required information. In most cases, claimants did not feel well prepared for the hearing, as they expected a much more casual process like what they experienced at the General Division’s hearing, rather than the more formal discussion during the Appeal Division hearing. Many did not come prepared with legal references, and were surprised to be asked to quote parts of the law relevant to their appeal.
“There was sufficient time to prepare but [the hearing] completely blind-sided me with the technical jargon. I thought there was a requirement at the tribunal to ‘dumb it down’ so the average person could understand. It seemed like the vice chair or tribunal member was looking for me to hit specific points and I was under the understanding from the earlier tribunal [at the General Division] and the information from the navigator that I was just supposed to speak my piece. I was not able to reference whatever act or supporting cases and so forth. I did not think that it would have been such a big deal.”
“I was nervous about the tribunal situation so maybe the navigator could have gone a bit more into that. Telling me things like what will transpire at the hearing; when and how I am to present my arguments and how the Commission will present their argument and how much time is given, …things like that.”
“Throughout my whole process I never felt prepared. I did my research and I felt like I was alone in this. The information is given to me and you do what you want with him [sic] it. My navigator did not help me see different views or alternatives. It could be that it is not their role, but it would have been a huge help if they were able to do that.”
Regardless of how prepared and confident claimants felt, many were apprehensive about their ability to represent themselves during the hearing, especially if they had received an unfavourable decision at the General Division, or if they felt alone and without resources when preparing for the Appeal Division hearing. Moreover, many were ill prepared for the formality of the hearing, and thus were left with the impression that they did not get a fair opportunity to present their appeal.
“When it comes to dealing with the law, one word can make a difference as to whether you would win or lose. Somehow, somewhere these navigators have the information that someone is looking for but they are not willing to reveal it.”
“I felt like ‘cool I am bringing good stuff forward’, until the judge told me how things were going to go and then my stomach dropped and I realized that I was not prepared at all.”
“The legal language and requirements were not properly explained to me. I felt lost.”
“During the hearing I was surprised … they were asking me things that I did not understand. I had to ask them to repeat and explain more. I was confused. They did not talk in layman’s term. I am a non- legal person and there are two of them talking to each other and they are talking their language and then they turn to me asking me if I agree with them or not. The General Division was so casual, and it was like having a casual conversation. Very different.”
When asked what the navigator should have done differently to help them feel more prepared and confident on the day of the hearing, most claimants reiterated the importance of providing advice or guidance in preparing the arguments, as well as explain the process in greater details.
“It would have been nice to have someone say, ‘here are some cases similar to yours in the past and here are the outcomes and this is what EI is looking for’.”
“It would have been nice to know that I had to bring forward the majority of the evidence I thought was pertinent and that the judge did not have the information. I did not know that. I got ready to discuss information I thought they had. We are talking 300 pages of information, so it was difficult to summarize into a 30 min hearing when you are not prepared for it.”
“[The navigator] should be an advisor; they should know what works and what does not. Not do the work for me but let me know where I should focus my efforts and where I would waste my time. The navigator should be there to fill in the gaps in my thought process, in my preparation.”
Someone also mentioned that the navigator could assist in understanding the Tribunal decision by explaining the rationale supporting it.
The following recommendations are drawn from the analysis of findings and are presented for the SST’s consideration.
Email Direct Invite & Telephone Invitation Script for Scheduling of Interviews
Specification summary:
Subject: SST Navigator Service – tell us about your experience / Service d’accommpagnateur du TSS – parlez-nous de votre expérience
(UN MESSAGE EN FRANÇAIS SUIVRA)
Dear [NAME],
Earlier this year, you expressed interest in giving feedback on your experience with the Navigator Service provided to you by the Social Security Tribunal’s Appeal Division. I am reaching out today to schedule a time that is convenient to you for a 45-minute interview that will be conducted by telephone. You will receive $100 in appreciation for your time.
As you may recall, the Social Security Tribunal commissioned Kelly Sears Consulting Group and Narrative Research to conduct these interviews. We’d like to remind you that your participation in this process is voluntary and that any feedback you provide will remain anonymous and will not affect your dealings with the Social Security Tribunal, or with any other federal department or agency.
Please kindly reply to this email and suggest a few times when you are available for the 45-minute telephone interview within the next three weeks, and specify what accommodations, if any, you would require. Your participation is important and by telling us about your experience, you will help with improving the Navigator Service.
Thank you kindly for your interest; we look forward to hearing from you.
All the best,
Plus tôt cette année, vous avez exprimé votre intérêt à partager vos commentaires sur votre expérience avec le service d’accompagnateur fourni par la division d’appel du Tribunal de la sécurité sociale. Je communique avec vous aujourd’hui pour prendre rendez-vous à un moment qui vous convient pour une entrevue de 45 minutes par téléphone. Vous recevrez 100 $ en reconnaissance de votre temps.
Comme vous vous en souviendrez peut-être, le Tribunal de la sécurité sociale a mandaté Kelly Sears Consulting Group et Narrative Research afin d’effectuer ces entrevues. Nous vous rappelons que votre participation à ce processus est volontaire et que vos commentaires demeureront anonymes et votre participation n’aura aucun impact sur vos rapports avec le Tribunal de la sécurité sociale ou avec tout autre ministère ou agence fédéral.
Veuillez répondre à ce courriel et suggérer quelques disponibilités pour l’entrevue téléphonique de 45 minutes au cours des trois prochaines semaines, et préciser les mesures d’adaptation dont vous auriez besoin. Votre participation est importante et en nous racontant votre expérience, vous aiderez à améliorer le service d’accompagnateur.
Merci de votre intérêt; nous sommes impatients d’avoir de vos nouvelles.
Cordialement,
Hello, my name is _______________ with Narrative Research and I am calling on behalf of the Social Security Tribunal. I am following up on an email we recently sent you inviting you regarding an evaluation of the Navigator Service. The Social Security Tribunal commissioned Kelly Sears Consulting Group and Narrative Research to conduct these interviews, with the goal of improving the service.
I am looking to schedule a 45-minute telephone interview with you within the next two weeks. In appreciation for your time, you would receive a $100 compensation. Your comments will remain anonymous, and your participation will not affect your relations with the Social Security Tribunal or any other federal government department or agency.
Would you like us to schedule this interview?
[IF YES] We have the following times available… [CONSULT SCHEDULES; RECORD DATE AND TIME FOR THE INTERVIEW]
And what is the best telephone number to reach you for the interview? [RECORD PHONE NUMBER]
Would you like to receive an email confirmation of the interview date and time?
[IF YES] Please provide your email address where we should send you this information. [RECORD AND CONFIRM EMAIL ADDRESS]
Are there any accommodations you may require to ensure you are able to participate in this interview? [IF YES, RECORD RESPONSE] We will get back to you about the possibility of providing those accommodations.
Thank you for your interest in our study. We look forward to discussing your opinion during the follow-up interview.
Bonjour, je m’appelle ________ et je travaille pour la firme Narrative Research et j’appelle au nom du Tribunal de la sécurité sociale. Je fais suite à un courriel que nous vous avons envoyé pour vous inviter à participer à une évaluation du service d’accompagnateur. Le Tribunal de la sécurité sociale a demandé à Kelly Sears Consulting Group et à Narrative Research de mener ces entrevues, dans le but d’améliorer le service.
J’aimerais prendre rendez-vous pour une entrevue téléphonique de 45 minutes au cours des deux prochaines semaines. En reconnaissance de votre temps, vous recevrez une compensation de 100 $. Vos commentaires demeureront anonymes et votre participation n’aura aucun impact sur vos relations avec le Tribunal de la sécurité sociale ou tout autre ministère ou agence du gouvernement fédéral.
Souhaitez-vous que nous planifiions cette entrevue?
[SI OUI] Nous avons les horaires suivants disponibles... [CONSULTER LES HORAIRES; ENREGISTRER LA DATE ET L’HEURE DE L’ENTREVUE]
Et quel est le meilleur numéro de téléphone pour vous joindre pour l’entrevue? [ENREGISTRER LENUMÉRO DE TÉLÉPHONE]
Souhaitez-vous recevoir un courriel pour confirmer la date et l’heure de l’entrevue?
[SI OUI] Veuillez fournir votre adresse e-mail où nous devrions vous envoyer ces informations. [ENREGISTRER ET CONFIRMER L’ADRESSE COURRIEL]
Y a-t-il des mesures d’adaptation dont vous pourriez avoir besoin pour pouvoir participer à cette entrevue? [SI OUI, CONSIGNEZ LA RÉPONSE] Nous vous reviendrons concernant la possibilité de fournir ces mesures d’adaptation.
Nous vous remercions de l’intérêt que vous portez à notre étude. Nous sommes impatients de discuter de votre opinion lors de l’entretien de suivi.
Profile information (from sample file)
Name:
Email:
Telephone:
Date and Time of Interview:
Decision Date:
Name of Navigator:
Contact Period:
Type of appellant (EI or IS):
Follow this script to ask people about their experience of using the navigator service.
The script wording is indented. Wording in grey is for you to fill in or follow as a direction.
Thanks for taking the time to speak with me today. My name is [name] and I work for Kelly Sears and Narrative Research. Our discussion should take about 45 minutes. We’ll be talking about your experience with the navigator service at the Social Security Tribunal. You had a navigator to help you prepare for your appeal. Your answers will help the Social Security Tribunal make the service better.
You may have been in touch with many people when preparing for your appeal. But today we’ll focus on your interactions with your navigator, [name of navigator], around [contact period with navigator].
Your participation is voluntary, your comments will remain anonymous, and the information you provide will be administered in accordance with the requirements under the Privacy Act, Access to Information Act, and any other pertinent legislation. Your participation won’t affect the services you get from the Social Security Tribunal or any other federal department or agency.
One of my colleagues is on this call and will be taking notes. But, anything you say will remain anonymous.
Any questions before we begin?
When the government denies you Employment Insurance, Canada Pension Plan, or Old Age Security benefits, challenging that decision generally involves a few steps. For you, it went like this:
I’d like to better understand your experience with this navigator.
Questions about the process and what they expected (8 min.)
Questions about time and the level of contact (5 min.)
Questions about documents and information (7 min.)
Questions about interacting with their navigator (10 min.)
Questions about how prepared and confident they felt (5 min.)
Questions about their overall experience (5 min.)
Based on everything we’ve discussed today …
Before we finish up, I have a few questions that will help the Social Security Tribunal better understand the people it serves. You don’t have to answer any of these questions, and as I said, anything you say will remain anonymous. To begin …
[Optional answers if they mention them]
That concludes my questions today. On behalf of the Social Security Tribunal, thank you for your participation! You’ll get an email in the coming days with details about the $100 you’ll receive for your time today.
Thank you for taking the time to share feedback on your experience with the navigator service at the Social Security Tribunal during our upcoming interview. The navigator service provided you support to help prepare for your appeal and you dealt with a navigator as part of this process, prior to your last hearing. We will not discuss the hearing itself, but rather how you were supported to prepare for that hearing. Your feedback will help the Social Security Tribunal make the navigator service better.
The upcoming interview will be an informal discussion with a professional facilitator from Narrative Research. While no preparation is required, we wanted to let you know the types of topics that will be discussed:
Again, as the discussion will be information, there is no need to prepare for the upcoming interview. We look forward to hearing your thoughts and opinions!