Prepared for the Administrative Tribunals Support Service of Canada, Social Security Tribunal Secretariat
Supplier Name: Kelly Sears Consulting Group and Narrative Research
Contract Number: CW2307292
Contract Value: $ 38 950.49
Contract award date: May 30, 2023
Delivery date: November 13, 2023
Registration Number: POR 011-23
For more information on this report please contact the Social Security Tribunal Secretariat at:
NC-SST-TSS-OUTREACH-RELATIONS-GD@canada.gc.ca
Ce rapport est aussi disponible en français.
End-User Interviews on the Navigator Service in the Appeal Division
Final Report
Prepared for the Administrative Tribunals Support Service of Canada, Social Security Tribunal Secretariat
Supplier Name: Kelly Sears Consulting Group and Narrative Research
November 2023
This public opinion research report presents the results of in-depth interviews conducted with end-users of the Navigator Service at the Appeal Division of the Social Security Tribunal of Canada. An evaluation of end-user experience was sought to identify and address any performance issues there may be. The research involved a total of 20 in-depth interviews, conducted by telephone, with a cross-section of types of former end-users of the Navigator Service. The fieldwork was conducted between July 28 and August 28, 2023.
Cette publication est aussi disponible en français sous le titre:
Entrevues avec les utilisatrices et les utilisateurs finaux du service d’accompagnement à la division d’appel
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from the Administrative Tribunals Support Service of Canada. For more information on this report, please contact the Social Security Tribunal of Canada at: info.sst-tss@canada.gc.ca.
Catalogue Number:
Em27-4/2024E-PDF
International Standard Book Number (ISBN):
978-0-660-69442-9
Related publications (registration number:
Catalogue Number Em27-4/2024F-PDF (Final Report, French)
ISBN 978-0-660-69443-6
© His Majesty the King in Right of Canada, 2023
Narrative Research Inc. and Kelly Sears Consulting
Contract Number: CW2307292
Contract Award Date: 2023-05-30
Contracted Cost: $38,950.49
The Administrative Tribunals Support Service of Canada (ATSSC) provides support services and facilities to twelve federal administrative tribunals through a single, integrated organization. The Social Security Tribunal of Canada (SST) is one of the tribunals supported by the ATSSC. The SST makes decisions on appeals related to employment insurance benefits and income security benefits (e.g., Canada Pension Plan, Old Age Security). The SST has two levels of appeal: the General Division (GD) and the Appeal Division (AD).
The SST Secretariat launched the Navigator Service in the Appeal Division in November 2020. After more than two years in operation, the SST wished to study the performance of the service by collecting feedback from key players, including end-users. This research will provide the SST with the user feedback needed to identify and address performance issues from the perspective of key actors in the appeal system. The SST will apply the results to inform any changes to service delivery and internal processes.
More specifically, areas of investigation that were part of the study included:
The investigation included 20 in-depth interviews conducted by telephone from July 28 to August 28, 2023 with a cross-section of types of end-users. The SST conducted the initial outreach to end-users in the spring of 2023 to assess their interest in taking part. The resulting list included 32 contacts, from which the research team scheduled interviews with 20 end-users. The selection was made to ensure a good cross-section of regions, official languages, and navigators. While a preference was given to include those with most recent interactions for best recall, interviews covered the last two years of the service to minimize recency biases .
This report presents the findings from the end-user interviews. Caution must be exercised when interpreting the results, as qualitative research is intended to be directional only. Results cannot be attributed to the overall population under study, with any degree of confidence.
I hereby certify as a Representative of Narrative Research that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Directive on the Management of Communications. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.
Signed
Margaret Brigley, CEO & Partner | Narrative Research
Date: September 11, 2023
The Navigator Service is well understood by end-users as a service that provides direction or instructions to assist them in preparing their appeal. Specifically, end-users have a clear understanding that the navigator is able to explain the appeal process, describe what will happen during the hearing, guide claimants to online resources that provide information or support, and remain available to answer questions. There is also a clear understanding of the Navigator Service’s limitations, namely, that the navigator is unable to provide legal advice or personal opinions, assist in preparing the arguments, attend the hearing and represent claimants, or provide a referral for representatives, organizations, or documents to be used.
Nonetheless, claimants’ needs go beyond what the Navigator Service is currently offering, which leads many end-users to be unhappy with their experience. When asked to rate their satisfaction with the Navigator Service, about half of those interviewed gave a negative score, while the other half rated their experience positively. Satisfaction primarily stems from the personal interactions with the navigator, whose professionalism and personable approach often exceeded end-users’ expectations. In addition, participants attributed satisfaction to the Navigator Service having delivered on its promise.
That said, the Navigator Service fails to meet claimants’ needs for guidance and support to ensure they are well prepared for the hearing. Specifically, they would like the navigator to provide a more in-depth description of the appeals process, including what will happen during the hearing. Claimants’ experience with the informality of their hearing at the General Division does not prepare them for the formality and legal requirements of the Appeal Division hearing. As such, there is a desire for better explanation of the legal terminology and the inclusion of what to expect at the hearing, instructions on how to prepare and present a strong argument, and guidance in choosing the most relevant court cases, laws and regulations to prepare the appeal. The inability for the navigator to act as an advisor, and claimants feeling ill-prepared for the hearing (especially when finding this out at the hearing), are by far the main sources of dissatisfaction that participants have with the Navigator Service.
The level of contact with the navigator varied across end-users interviewed. That said, all received an introductory call from the navigator explaining their role, and all were called again a couple of weeks prior to the hearing with reminders. Some were in more frequent contact with the navigator during the preparation of their appeal, as needed. This level of contact was viewed as adequate given the scope of the Navigator Service, and end-users understood that they could contact the navigator at any point in time during the process.
Navigator accessibility is also adequate to meet the needs of service users. With a few exceptions, the navigator was easy to reach, and offered flexible communication methods to meet the needs of end-users. While returned calls were promptly made, a couple of claimants would have liked the ability to immediately access a navigator for assistance when reviewing information. As such, the SST could explore the possibility to offer a complementary service for immediate response to process questions (e.g., live chat) thus allowing end-users to keep their momentum when preparing their appeal.
Navigator Service users generally felt that sufficient time was provided for them to prepare their appeal. They generally appreciated that the navigator provided information to help direct their efforts, including basic information about the appeal process, reference for documentation or support (e.g., court decisions, legal aid organizations, acts and legislation), and reminders of important milestones (e.g., notifications of documents being sent, reminders about the hearing).
While a general description of the appeal process was provided by the navigator (broad description of the hearing process, parties attending and their role, and documents required, and the type of questions that may be asked), there appears to be a gap in helping claimants to effectively prepare, structure and present their arguments. As such, providing service-users with a more in-depth explanation of the legal process and terminology may improve the end-user experience, and should be considered by the SST. This could include guiding claimants to the relevant court cases and legislation, or at least helping them search the database, and providing instructions on how to structure and present their arguments during the hearing. In fact, identifying the most relevant court cases and legislation was consistently mentioned by end-users as time-consuming and difficult.
Service users are very complimentary of the navigators. Throughout the process, they generally felt respected, listened to, and supported within the scope of the program. The navigators were often lauded for their professionalism, honesty, helpfulness, personable approach, politeness, patience, friendliness, and for being open and honest, empathetic and easy to talk to. They were also seen as well-spoken, but somewhat guarded in their assistance, due to the limitation of their role. Although accommodations were not required by service users interviewed, all felt that the navigator would have been flexible and accommodating if needed. The navigators’ abilities to make end-users feel cared for was consistently the most appreciated aspect of the interactions. By contrast, the limitation in the level of support provided by the navigator given their role, was most often mentioned as the least appreciated aspect of interactions.
One of the goals of the Navigator Service is to ensure that claimants feel prepared and confident for the hearing. About half of claimants felt prepared and confident on the day of the hearing, based on the information they had collected and the strength of their argument. Those who had a favourable decision at the General Division hearing, but then the decision was challenged by the Minister, were also most confident about the Appeal Division hearing. The other half of claimants felt nervous and insecure, primarily as they did not know what to expect from the hearing, or for not having been able to find required information (e.g., relevant court cases, laws and regulations that apply to their situation).
Despite their initial sentiment prior to the hearing, claimants were surprised with the formality and structure that defined the Appeal Division’s hearing, which significantly affected their level of confidence during the hearing itself. As such, there is a need for the Navigator Service to provide more information about the hearing to raise claimants’ level of confidence, not only in the hearing, but also in ensuring their perception that the tribunal provided an opportunity to be well represented, and a fair chance to win their appeal. More time spent on preparing claimants for the hearing, through discussion and practices, may also provide a greater sense of confidence among claimants.
Findings from this research suggest that there would be value in the SST reviewing the Navigator Service to see how it could improve the level of support provided. This could include the development of additional tools to describe the appeal process, as well as tips on how to prepare an argument. Attention should also be afforded to describing the purpose of the Appeal Division hearing, clearly outlining the hearing process, as well as informing claimants as to how tribunal decisions are made. Consideration should also be given to expanding the role of navigators to provide individualized support, helping claimants source the right information and documents, and guiding them on the preparation of their appeal based on each individual case. Further, there is merit in exploring the possibility of making the Appeal Division less formal and more accessible to the claimant. Finally, while the service provided by navigators is well received, there is merit in implementing a service that could offer immediate response to questions, in instances where the navigator is not available (e.g., live chat service).