Client Experience Survey 2018-19

Executive Summary

Service Canada CX Survey 2018-19 — Results at a Glance

4,401 interviews conducted (between 600-1100 per program) 

Methodology: Telephone survey

Fieldwork: March 6 to 30 2019

Overall Service  Experience

Satisfaction by Channel

Satisfaction Among Selected Vulnerable Groups

Satisfaction by Program

Top Drivers of satisfaction

Strengths

Areas of Improvement

Background and Objectives

• The Client Experience Measurement Project was conducted in two phases.  The first phase involved executing the Client Experience (CX) Survey while the second phase included a series of online focus groups and in-depth interviews with clients identified through the quantitative survey.
• The Client Experience (CX) Survey is intended to track the service experience delivered by Service Canada.  In 2017-18, the Citizen Service Branch launched the annual CX Survey to obtain a baseline measurement. 
• In 2018-19, the survey has been used to collect data to contribute to monitoring the service delivery performance of Service Canada, and to inform service transformation.
• Note that for this wave of the survey considerable space was devoted to gathering information to support transformation decision-making over tracking detailed aspects of the service experience.  This change was made for this wave of the survey since the impact of service transformation was not expected to be evident until at least two years of service transformation implementation.
• The second qualitative phase explored why clients who are used to transacting online choose to use the in-person channel and the extent to which they would or would not shift to using the online channel in response to proposed innovations.
• The qualitative portion was also used to better understand the service experience among those who encountered a problem or issue or expressed dissatisfaction with their overall experience.

Methodology- Quantitative Phase

• A telephone survey was conducted with a sample of 4,401 Service Canada clients across the six major programs. For the 2018-19 survey, recipients of OAS and GIS have been combined into one client group because the applications were combined.
  • Employment Insurance (EI)
  • Canada Pension Plan (CPP)
  • Canada Pension Plan Disability Benefit (CPP-D)
  • Old Age Security (OAS)/ Guaranteed Income Supplement (GIS) and
  • Social Insurance Number (SIN).  
• Oversamples were conducted with two vulnerable client groups: those living in remote areas and Indigenous clients.
• The interviews were conducted from March 6 and March 31, 2019.
• Clients who had received a service outcome during August, September or October 2018 were sampled (for CPP-D and OAS/GIS the period was extended to December 2018 to gather sufficient sample).
• The survey sample size has a margin of error of +/-1.5%, with between 600-1100 for each program with a margin of error of between +/-4% to +/-2.9%. 
• Results were weighted by age, gender, region, program and benefit receipt (approved/ denied) using administrative data on clients who completed a client journey (initial decision) from August to October 2018. Program weights were held constant with 2017-18 to allow the results to highlight any change due to the service experience.
• The 2018-19 Client Experience Survey Detailed Methodology document is available under separate cover.

Methodology- Qualitative Phase

• A series of 6 qualitative online focus group discussions were convened between May 7 and May 9, 2019.
• 4 groups for Attracting Clients to Self-Service:
  • 3 groups with 17 participants in total: 1 among those who applied for EI in person but are not e-vulnerable, 1 among those who applied for CPP in person but are not e-vulnerable and 1 among those who applied for CPP-D in person;
  • 1 control group with 8 participants who applied for EI or CPP online.
• 2 Groups for Dissatisfaction / Issue Resolution:
  • 2 groups with 11 participants in total
• All sessions were 90 minutes in length and hosted on Ipsos’ Ideation Exchange platform, and all participants also dialed into a teleconference line. All were in English.
• A series of 6 telephone interviews were conducted between May 23 – 28, 2019 in French. 5 were with EI applicants and 1 was with an OAS applicant.

Key Findings: Performance

• Satisfaction with the service experience remained stable compared to 2017-18 and continued to be lower for EI and CPP-D clients. Satisfaction is strongly related to trust in Service Canada.
  • The vast majority of Service Canada’s clientele remain satisfied with the service experience (85%) and found it easy (85%) and effective (84%).  Improvements have been made to effectiveness compared to last year and a greater proportion of clients provide the highest ratings for ease and effectiveness, while fewer provide the highest rating for overall satisfaction.
  • Clients also express a high degree of trust (83%) in Service Canada to deliver services effectively to Canadians (a new measure for 2018-19).  Trust is highly correlated to overall satisfaction in the service experience.
  • Over nine in ten (92%) SIN clients express satisfaction with the service experience, the highest of any program and consistent with 2017-18.  Eight in ten EI clients (80%) and closer to six in ten CPP-D clients (62%) are satisfied overall, lower than other programs and statistically unchanged year over year.
• Clients remain most satisfied with the in person experience, while satisfaction is lowest for specialized call centres. Challenges have emerged with the specialized call centres experience year over year.
  • The large majority of clients remain satisfied with each service channel.  Satisfaction with in person service remains the highest (87%), followed by online (79%), and specialized call centres (75%).
  • A decline in satisfaction among those who used specialized call centres has been observed in this survey period compared to the last survey period.  Further, fewer clients provide the highest ratings for all service channels than last year.
• Overall, Service Canada clients rate the service most highly on helpfulness of staff, confidence in having everything needed when submitting an application and confidence in information security.
  • Nearly nine in ten clients find Service Canada staff helpful (89%), are confident they had everything they needed when they submitted their application (88%) and confident their personal information was protected (87%).
• Service attributes with lower ratings are timeliness of service, ease of getting assistance, needing to explain your situation only once and clarity of process.
  • Fewer than eight in ten provided high ratings for the amount of time it took for the entire client journey was reasonable (76%), ease of getting assistance when needed (77%), needing to explain your situation only once (77%) and that it was clear through the process what would happen next and when (77%).
• Certain aspects of service were rated consistently high or low across all programs
  • Confidence they had everything they needed when they submitted their applications and helpfulness of Service Canada Agents are consistently the highest rated areas.  Confidence in issue resolution is consistently among the lowest rated area.
• SIN clients continue to provide the highest ratings across all service attributes.
  • The vast majority of SIN clients provide positive ratings for all attributes and found the service experience easy and effective. Relatively speaking, lower scoring areas (which still achieved ratings of greater than 80%) include clarity of what to do when you have a problem and clarity in what would happen next and when.
• CPP and OAS/ GIS clients provide similar ratings across service attributes and most rate each area highly.
  • The majority CPP and OAS/GIS clients provide high ratings across most service attributes.  The strongest performing areas include helpfulness of Service Canada Agents, ease of applying and ease of understanding the requirements to apply.   The consistently lowest rated areas are ease of getting assistance and clarity in what would happen next and when.  CPP clients also experienced more difficulty in understanding what to do if they had a problem/question, while OAS/GIS clients provided lower ratings for needing to explain their situation only once.
• The majority of EI clients feel positive about most aspects of service, however ratings are slightly lower than other programs.
  • The EI service experience performs most strongly for confidence in protection of personal information, ease of applying, information being easy to understand and the helpfulness of Service Canada Agents. Lower performing areas include the timeliness of service, explaining situation only once, ease of getting assistance, and clarity of process.
• CPP-D clients continue to experience the most difficulty during the service experience. 
  • CPP-D clients provide considerably poorer ratings across most service attributes.  The CPP-D service experience is rated highest for confidence in protection of personal information and helpfulness of Service Canada Agents. Lowest rated service attributes include timeliness of service, clarity in what would happen next and when and explaining your situation only once.
• Overall, positive changes have been observed for a number of service attributes related to ease and confidence, while fewer rate the helpfulness of Service Canada Agents highly.
  • Clients are more likely to agree information was easy to understand, to be confident they had everything they needed when submitting their application, that it was easy to understand the requirements to apply, information about how to apply was easy to find, they were able to move through all steps smoothly, and were confident that any issues would be easily resolved.
  • Helpfulness of Service Canada Agents has declined compared to last year.

Key Findings: Drivers of Satisfaction

• The most important drivers of satisfaction are: timeliness of service, helpfulness of staff, ease of getting assistance, receiving consistent information and ease of issue resolution, though satisfaction has not improved in these areas. 
  • Common areas for potential improvement for the Service Canada clientele remain consistent year over year.  To improve the service experience for the greatest number of clients accessing ESDC programs, focus should be placed primarily on improving the timeliness of service, ease of getting assistance, and ease of issue resolution which represent service attributes most strongly related to overall satisfaction where performance is lower relative to other areas.
  • Timeliness of service has taken on increased prominence in driving satisfaction. Providing a mechanism to better manage expectations regarding wait-times and the timeline for a decision is likely to help to improve satisfaction among clients.
  • The ease of receiving assistance throughout the client journey is strongly tied to confidence in issue resolution as well as overall satisfaction and trust.  It provides clients with the reassurance they desire throughout the different stages of the application process.
• While fewer clients experienced an issue this year, challenges arose regarding the ease of resolution.  Improvements to online functionality, courtesy when explaining denials, proactively setting expectations for wait times and centralization of application information is likely to lead to more satisfactory outcomes.
  • Sixteen percent (16%) of clients report having experienced a problem in 2018-19, lower than last year.  However, only one-third (33%) of those who did feel it was easily resolved, significantly lower than in 2017-18. The most common problems experienced are long/complicated applications, confusing online information, too long to provide benefit/decision and complaints about staff not being knowledgeable. 
  • CPP-D and EI clients are most likely to have experienced a problem (32% and 24% respectively) and fewer CPP-D, OAS/GIS or SIN clients experienced a problem this year.  Ease of problem resolution is highest for CPP clients (46%) and lowest for CPP-D clients (28%) and has declined for EI and SIN clients year over year.
  • As identified through the qualitative research, suggested improvements relate to greater online functionality, improved courtesy when explaining denials, proactively setting expectations for wait times and centralization of application   information for better issue resolution.

Key Findings: Access to Service

• Most vulnerable client groups continue to provide high ratings of the service experience. Satisfaction among Indigenous clients has improved year over year.
  • A strong majority of clients in vulnerable groups Service Canada serves had a positive service experience. 
  • Compared to 2017-18, satisfaction with the service experience has improved substantially among Indigenous clients.  Clients with less formal education are less satisfied year over year, however the vast majority provide a positive rating and ratings remain consistent with other clients.
• Clients with restrictions that make it more difficult to access service, and clients with disabilities need improvements to their service experience to bring their satisfaction up to the levels of other clients. Improving the simplicity of information, clarity of process and ease of accessing assistance, particularly regarding unique and complex situations, is likely to help to improve their experience.
  • Clients have restrictions are more likely to have experienced a problem, have a longer client journey (specifically more than 8 weeks), and rate satisfaction with all channels lower compared to all clients.  They are also less satisfied with several service attributes with the largest gaps for ease of understanding information, needing to explain your situation only once, ease of completing online, confidence in issue resolution and ease of getting assistance.
  • Clients with disabilities are significantly more likely to have experienced a problem, have a longer client journey (specifically 4 weeks or longer), and rate channel satisfaction lower for in-person and online.  They are also less satisfied with several service attributes with the largest gaps for ease of completing online, needing to explain your situation only once, moving smoothly through all steps and ease of understanding information.

Key Findings: Service Transformation

• Clients who visited an office to apply, including those who had the option to complete online, did so due to the confidence and ease that they find through person-to-person contact.  Having an officer explain the process, and provide assistance improves clients’ confidence that they applied correctly.
  • The main reasons clients prefer to visit a Service Canada Office are the confidence things are done properly, and to get assistance/make it easier.  Further, OAS/GIS, CPP-D and CPP clients are more likely to say they are not comfortable applying online, while CPP-D and OAS/GIS clients are also more likely to report they don’t have access to the internet / to a computer.
  • Qualitatively, applying in-person was perceived to have a number of advantages over applying online.  These include the confidence that their application was accurate and submitted, that it would be faster or easier than calling to get assistance and the desire to be “coached” through the application, to ensure greater chance of approval and/or to prevent delays.
  • The need for in-person channels will continue to exist particularly for more complex cases such as clients with restrictions to access service or those with a disability who rely more heavily on in-person channels.
• Nearly half of clients who have the option to self-serve would be influenced to do so through improved assistance, either through quick help by phone or chat, or through video link.  These changes would help to provide clients with confidence they are following the process properly, which is a strong driver for choosing the in-person channel.
  • Quick help by phone during business hours and being able to talk to an agent by video link are most likely to drive clients to apply from home.  If steps online were simpler and clearly explained or if quick help could be provided through online chat were motivating to some. Notably, two in ten said none of the proposed solutions would motivate them to apply from home.
  • As learnt through the qualitative research, attracting clients to self-service needs to fulfill their need for detailed information and instruction and providing some of the best aspects of the in-person experience. These include reassurance the application is accurate and actively being processed, confirmation that any supporting documents have been received, and ease of having their questions answered. Concerns with self-service would also need to be addressed including better communication of expectations for each step of the process and providing reassurance that their application matters. 
• My Service Canada Account (MSCA) is helping to promote self-service by diverting clients away from phone channels.  MSCA users value the platform for providing simple and easy access to information and improved clarity about the status of their application.  Non-users lacked awareness of the platform and its functionality but found it appealing.
  • More than seven in ten EI and CPP-RTR clients said they registered for a My Service Canada Account (MSCA) during their application, while three in ten OAS/GIS clients said they did so. Three-quarters of those who said they registered for a MSCA found it easy to use, and ratings are consistent across EI, CPP-RTR and OAS/GIS clients.
  • Roughly one quarter of clients in each program required assistance setting up their MSCAs, with half of those clients going into an office for help while one-third called an office.
  • Nearly eight in ten used MSCA to check the status of their application and two-thirds of those who did got the information they needed.  EI clients are far more likely to have checked the status of their application. CPP-RTR clients are more likely to have received the information they needed without calling Service Canada for assistance.
  • Qualitatively, the functionality offered by MSCA is universally appealing.  However, many are unaware of the platform and/or about the range of functions and information available.
  • Clients say that use of MSCA would eliminate confusion and uncertainty about their application, and provide confidence their application was accurate, and actively being processed.  The idea of a centralized “case management” system that they could access was extremely appealing.  A few had issues with registering for an account and having a PIN number sent via email was appealing to most.
• The main barriers to applying online from home are psychological and emotional.  Clients are often applying for benefits for the first time and the importance of the support increases the need for reassurance and confidence.
  • Almost all clients are actively using other online services on a regular basis. However, clients feel that the situations are not parallel in that they very rarely apply for government benefits – for many, they are applying for the first time.
  • Further, the gravity and importance of what they are applying for means that they have a heightened sense of needing reassurance and feeling confident in the process.
• Findings indicate that the following can help to increase uptake of self-service among clients:
  • MANAGE TIMELINESS EXPECTATIONS – better communicate the steps in the process including how long each step can take up, the client’s role throughout including key milestones, and of ways to receive assistance or information without going in-person (i.e. 1 800 O Canada, MSCA);
  • AVAILABILITY OF ASSISTANCE – nearly half of clients who have the option to self-serve could be influenced to do so through improved assistance, either through quick help by phone or chat, or through video link; these changes could help to provide clients with confidence they are following the process properly, which is a strong driver for choosing the in-person channel.
  • CONFIDENCE – clients want to know that their application is important and being actively considered.  Service Canada staff excel at making clients feel valued and this represents a shortcoming of the self-service experience at present:
    • CERTAINTY – providing assurance that their application has been filled out correctly is a strong desired outcome and represents a primary reason clients choose to access services in-person for assistance;
    • CLARITY – explaining each question in the application, and how to navigate the process overall are key benefits of the in-person channel that could be integrated into other channels;