Client Experience Survey 2019-20

Executive Summary

Service Canada CX Survey 2019-20 — Results at a Glance

Background and Objectives

• The annual Service Canada Client Experience (CX) Survey tracks the impact of service delivery change on clients’ ability to access federal programs, particularly as delivery was increasingly e-enabled over time.
• The 2019-20 survey was the third wave of the survey established by the Citizen Service Branch in 2017-18. It tracks client experience performance measures associated with the ease, effectiveness, confidence and satisfaction with the service experience by program, client group and service channels used.
• It also tracks reported self-service and use of assistance among Service Canada clients to inform Service Transformation over time.
• Due to Covid-19, fieldwork for the 2019-20 CX Survey was ended midway through execution. As such, the original sample size target could not be reached. The Client Experience Measurement Project was also intended to include a second phase of qualitative research, however this phase was cancelled due to concerns around contacting clients during the health crisis.

Methodology

• A telephone survey was conducted with a sample of 2,431 Service Canada clients across the six major programs.
  • Employment Insurance (EI): (n=701) +/- 3.7 percentage points
  • Canada Pension Plan (CPP): (n=389) +/- 5.0 percentage points
  • Canada Pension Plan Disability Benefit (CPP-D): (n=417) +/- 4.8 percentage points
  • Old Age Security (OAS)/ Guaranteed Income Supplement (GIS): (n=562) +/- 4.1 percentage points;  and
  • Social Insurance Number (SIN): (n=362) +/- 5.1 percentage points
• Oversamples were collected with two client groups: those living in remote areas and Indigenous clients.
• The interviews were conducted between Feb 25 and March 16, 2020 (Fieldwork was ended early due to COVID-19).
• Clients who completed a client journey, that was received an initial decision, benefit or Social Insurance Number, in August, September or October 2019 were sampled. 
• The survey sample size has a margin of error of +/-2.0%.
• Results were weighted by age, gender, region, program and benefit receipt (approved/ denied) using administrative data on clients who completed a client journey from August to October 2019. Program weights were held constant with 2017-18 to allow the results to highlight any change due to the service experience.
• Comparisons to 2018-19 results reflect service delivery changes made after October 2018, and comparisons to 2017-18 results reflect changes relative to roughly April/May/June 2017.
• Due to Covid-19, fieldwork for the 2019-20 CX Survey was ended midway through execution. As such, the original sample size target of n=4000 could not be reached.
• The 2019-20 Client Experience Survey Detailed Methodology document was available under separate cover.

Executive Summary: Change in Channel Use

• The proportion of clients who self-served online with no assistance was slightly lower in the August-October 2019 period relative to April-June 2017. Self-service online without assistance increased among CPP clients at the aware and apply stages, and among OAS/GIS clients applying and following up on an expected benefit.  At the same time, fewer EI clients self-served online without assistance at both the aware and apply stages; more applied in-person and more followed up on an application to check the status (41%) or provide missing information (21%), most often turning to the in-person and phone channels rather than My Service Canada Account alone.
  • The majority of clients continued to utilize in-person service (62%) at some point during the client journey, while nearly two in ten (17%) self-served online only, one in ten (10%) self-served online with assistance, four percent (4%) were auto-enrolled and did not contact Service Canada, and one percent accessed service by mail only.
  • Overall, clients were more likely to use in-person service during the apply stage and less likely during the aware stage compared to the baseline wave of the survey.  They were also more likely to self-serve with assistance at the follow-up stage.
    • EI clients less likely to self-serve with no assistance at the aware or apply stages compared to 2017-18 and more likely to get help through the in-person service when applying. They were also less likely to utilize assisted self-service during the apply stage and more likely during the follow-up stage.
    • CPP clients were more likely to self-serve and less likely to rely on in-person service at the aware or apply stages compared to the baseline wave.  They were also less likely to only receive mail service at the apply stage.
    • OAS/GIS clients were more likely to self-serve at the apply or follow-up stages than they were in the baseline wave. They were also less likely to rely on in-person service at the awareness stage.
    • CPP-D clients show no significant increases or decreases since the baseline wave.

Executive Summary: Number of Contacts with Service Canada

• Clients had a higher number of contacts with Service Canada than in previous years, due primarily to an increase in the number of times clients contacted specialized call centres.  Satisfaction with the service experience declines with the number of times the client contacted Service Canada and was lower than last year among those with contact 3 or more times.
  • As in previous years, the level of satisfaction with the service experience declined by the number of times the client contacted Service Canada. Satisfaction was notably lower among those who had five or more contacts with Service Canada through any channel during the client journey.
  • Compared to 2018-19, satisfaction improved among those who had contact once or twice and declined among those who had a greater number of contacts during their experience.
• EI and CPP-D clients continued to have more contact with Service Canada during the client journey compared to all clients.  Year over year, EI and CPP clients were in contact with Service Canada more times during their experience.
  • Number of contacts continued to differ by program with EI and CPP-D clients being most likely to have contacted Service Canada 5+ times, while SIN and OAS/GIS were most likely to have contacted Service Canada 1 or 2 times.
  • Compared to 2018-19, EI and CPP clients were more likely to have been in contact with Service Canada 5+ times, while CPP-D clients were less likely to have had contact twice.

Executive Summary: Channel Use By Stage

• Online and in-person channels continued to be used by clients most often for the aware and apply stages, while telephone and online were more common for follow up.  Use of online increased during the aware stage and decreased for apply, while in-person use increased across both the aware and apply stages.
  • During the aware stage, clients continued to be most likely to use online government sources (58%), followed by in-person service (47%) while around one in ten used either telephone (15%) or mail (10%). Use of online and in-person channels increased compared to the baseline wave, while telephone and mail have decreased.
  • During the apply stage, clients were most likely to use in-person service (59%), followed by online (48%) while around one in ten used either telephone (13%) or mail (10%).  Use of in-person service increased compared to the baseline wave, while online, telephone and mail have decreased.
  • Among those clients who followed-up, they continued to be most likely to use the telephone channel (68%), followed by online (56%) and in-person (40%) service while around one in ten used mail (15%). Use of online and mail increased compared to the baseline wave.
• Being able to complete steps online made the process easier for the vast majority of clients and progress was made improving ease of use among CPP, CPP-D and OAS/GIS clients.
  • Consistent with last year, three quarters (75%) of clients agreed that being able to complete steps online made the process easier. EI clients (82%) were more likely to agree compared to all clients and while CPP (60%), CPP-D (37%) and OAS/GIS  (48%) clients were less likely to agree, their ratings of ease have improved year over year.

Executive Summary: Multi-Channel Use By Stage

• The vast majority of clients used only one channel during each stage of the client journey. Clients continued to rely more heavily on online and in-person as their first point of contact for all stages except follow-up where telephone was the most used primary channel.
  • Clients most often used the online channel first at the aware and follow-up stage of the client journey and most used in-person first for the apply stage (followed closely by online). Telephone continued to be the least used channel for the first point of contact at the aware and apply stages and was the second most used at the follow-up stage.
  • Clients who used in-person first were less likely to use a second channel compared to clients who used online first. Clients were generally more likely to go in-person or online after beginning over the phone at the aware and apply stages, but were no more likely at the follow-up stage.
  • Year over year, clients were more likely to have used in-person first during the apply stage and less likely for the aware or follow-up stages. They were more likely to have used online first at the follow-up stage and less likely at the apply stage. Clients were also less likely to have used the telephone channel first at the apply stage.
• Vulnerable clients varied in their channel preferences. Certain groups relied more heavily on in-person while others relied more heavily on mail service during the client journey.
  • Youth and newcomers were both significantly more likely to choose an in-person channel at both the aware and apply stages.
  • Seniors were significantly less likely to use in-person service and more likely to choose the mail channel at both aware and apply while clients with disabilities and those who were e-vulnerable were less likely to use in-person at the apply stage and more likely to use mail.
  • All other vulnerable groups showed no statistically significant variance in their channel preference compared to all clients.

Executive Summary: Overall Satisfaction

• Satisfaction with the overall service experience was consistent with 2018-19 but has declined compared to 2017-18. Satisfaction continues to be lower for EI and CPP-D clients. Satisfaction among EI clients has declined compared to the baseline wave.
  • The vast majority of Service Canada’s clientele continue to be satisfied with the service experience (84%) and found it easy (84%) and effective (82%). Year over year, effectiveness has declined (82% vs. 84%), while overall satisfaction was lower than in the baseline wave in 2017-18 (86%).
  • Clients expressed a high degree of trust (83%) in Service Canada to deliver services effectively to Canadians, confidence in the issue resolution process (78%) and felt the timeliness of service was reasonable (77%).
  • Over nine in ten (94%) SIN clients expressed satisfaction with the service experience, the highest of any program and consistent with previous waves. Close to eight in ten EI clients (77%) and six in ten CPP-D clients (60%) were satisfied overall, lower than other programs, and EI clients were less satisfied than they were in 2017-18 (83%).
  • SIN clients were more likely to feel the process was effective (91%) and to have confidence in issue resolution (87%), while OAS/GIS clients were more likely to feel the process was easy (92%) and effective (88%). SIN, OAS/GIS and CPP clients were the most likely to rate the timeliness of service as reasonable (89%, 85%, 83% respectively).
  • EI and CPP-D clients were less likely to find the process was effective (76%, 57%), to have confidence in issue resolution (72%, 51%) or agree the timeliness of service was reasonable (68%, 49%). CPP-D clients were also less likely to feel the process was easy (55%).
  • Compared to 2018-19, OAS/GIS clients were more likely to feel the process was effective (88% vs. 84%) and to feel the timeliness of service was reasonable (85% vs. 80%).  EI clients were less likely to feel the process was effective (76% vs. 81%), while CPP clients were more likely to have confidence in the issue resolution process (81% vs. 76%) and express trust in Service Canada (86% vs. 81%).

Executive Summary: Problems Encountered

• More clients experienced a problem this year due to an increase among EI and SIN clients, however the ease of resolution among those who experienced an issue improved.
  • Two in ten (20%) clients reported having experienced a problem in 2019-20, higher than in 2018-19 (16%). Among those who experienced a problem, nearly half (45%) of clients felt the issue was easily resolved; this was higher than in the previous wave (33%), but stable with levels in the baseline wave.
  • The most common problems/issues experienced were that it took too long to provide benefits/ decision, long/ complicated applications and that online information was confusing. Fewer clients mentioned that the application form was long/ complicated compared to last year.
  • CPP-D and EI clients were most likely to have experienced a problem (33% and 29% respectively), while OAS/GIS and SIN were least likely (9% and 8%).  A greater number of EI (29% vs. 24%) and SIN (8% vs. 5%) clients experienced a problem compared to last year.
  • Ease of problem/issue resolution was highest for SIN clients (57%) and lowest for CPP-D (25%) and OAS/GIS clients (33%). Ease of resolution increased significantly for EI (45% vs. 32%) and SIN clients (57% vs. 32%) this year.

Executive Summary: Service Attribute Performance

• Overall, Service Canada clients provided the highest ratings for the helpfulness of in-person staff, confidence in information security, the process being easy and effective.
  • The vast majority found Service Canada in-person staff helpful (92%), were confident their personal information was protected (87%), found it easy to apply (84%) and were able to move smoothly through all steps (82%).
• Service attributes with lower ratings were ease of follow-up, confidence the application would be processed in a reasonable amount of time, ease of figuring out eligibility, clarity of the process and helpfulness of call centre phone staff.
  • Fewer than three quarters provided high ratings for the ease of following-up on their application (61%), confidence the application would be processed in a reasonable amount of time (64%), ease of figuring out eligibility (71%) or the best time to start your pension (72%), that it was clear through the process what would happen next and when (73%) and the helpfulness of Service Canada call centre staff (73%).
• Certain aspects of service were rated consistently high, while lower rated areas differed considerably by program.
  • The helpfulness of Service Canada in-person staff and confidence their personal information was protected were consistently the highest rated areas.
• Overall, there have been a number of negative shifts for service attributes related to ease and effectiveness year over year.
  • Clients were less likely to agree that it was clear through the process what would happen next and when, that they were able to move smoothly through all steps, received consistent information, found it easy to figure out eligibility, were able to find the information they needed during the aware stage in a reasonable time and found it easy to follow-up.

Executive Summary: Service Attribute Performance By Program

• SIN clients continue to provide the highest ratings across all service attributes.
  • The vast majority of SIN clients provided positive ratings for all attributes and found the service experience easy and effective. Lower scoring areas include the reasonableness of distance travelled to access service and the helpfulness of call centre staff.
• CPP and OAS/GIS clients continued to provide similar ratings across several service attributes and strong majorities rate most areas highly.
  • The strongest performing areas included the helpfulness of Service Canada in-person staff, ease of applying, moving smoothly through all steps and the timeliness of service. The consistently lowest rated areas were the ease of getting assistance when needed and being able to complete steps online made it easier. CPP clients also experienced more difficulty with clarity of the issue resolution process and helpfulness of Service Canada call centre staff. OAS/GIS clients provided lower ratings for confidence in the issue resolution process.
• EI clients provided positive ratings across most areas of service, however ratings were lower than other programs particularly for aspects of effectiveness.
  • The EI service experience performed most strongly for confidence in protection of personal information, ease of applying, and the helpfulness of Service Canada in-person staff. Lower performing areas included the timeliness of service, explaining situation only once, ease of getting assistance, clarity of process, confidence in issue resolution, completed the application in a reasonable time and distance travelled to access service was reasonable.

Key Findings: Service Attribute Performance By Program

• Consistent with previous years, CPP-D clients continued to experience the most difficulty during the service experience.
  • CPP-D clients provided considerably poorer ratings across nearly all service attributes. The CPP-D service experience was rated highest for confidence in protection of personal information and helpfulness of Service Canada in-person staff. Lowest rated service attributes included timeliness of service, clarity of process, effectiveness, confidence in issue resolution, completing application in a reasonable amount of time, understanding information about the program, ease of figuring out eligibility and ease of completing the form.
• Non Auto-Enroll clients were more likely to be satisfied with their experience compared to all clients while satisfaction among Auto-Enroll clients was comparable with the aggregate. 
  • Similar trends were observed when comparing clients receiving both OAS and GIS and those receiving only OAS. Clients receiving both OAS and GIS were more likely to be satisfied with their experience, while satisfaction among those receiving only OAS was consistent with all clients. 
  • OAS/GIS clients were predominantly non auto-enroll while the vast majority of OAS clients were auto-enrolled.

Key Findings: Change in Service Experience By Program

• EI clients were less satisfied with a number of aspects of service year over year.
  • Compared to 2018-19, EI clients were less likely to be satisfied with the effectiveness (76% vs. 81%) and clarity of the process (65% vs. 75%) and compared to the baseline wave had more difficulty figuring out eligibility (66% vs. 72%) and being able to complete the application in reasonable time (59% vs. 66%).
• CPP clients expressed higher trust in Service Canada and rated aspects of ease and confidence higher than previous years.
  • Compared to 2018-19, CPP clients were more likely to express trust in Service Canada (86% vs. 81%) and to agree that being able to complete steps online made it easer (60% vs. 52%). Compared to the baseline wave, CPP clients found it easier to understand information about the program and to be confident their application would be processed in a reasonable time (79% vs. 70%).
• OAS/GIS clients were more satisfied with a number of aspects of service year over year and considerable progress was made across several areas since the baseline wave.
  • Compared to 2018-19, OAS/GIS clients were more likely to agree they only had to explain their situation only once (80% vs. 75%), completing steps online made it easier (48% vs. 36%) and the amount of time from start to finish was reasonable (85% vs. 80%).
  • Compared to 2017-18, OAS/GIS clients were more likely to feel the process was easy (92% vs. 84%) and effective (88% vs. 79%). They also found it easier to understand information about the program (82% vs. 69%), figure out eligibility (84% vs. 72%), understand the requirements of application (83% vs. 75%) and to be confident the application would be processed in reasonable time (82% vs. 72%). They were also less likely to have experienced a problem (9% vs. 15%).
  • Improvement has been made among clients receiving OAS and GIS on overall satisfaction with their experience, timeliness of service and trust in Service Canada. We saw similar shifts among the subgroup of non-auto enroll clients due to the high proportion of overlap between the two groups- 69% of the OAS/GIS clients were non-auto-enrolled.

Key Findings: Change in Service Experience By Program

• Key pain points identified by CPP-D clients in the baseline survey have changed little.
  • Compared to the 2018-19, CPP-D clients were more likely to agree that being able to complete steps online made it easier (37% vs. 31%, but were less confident the application would be processed in a reasonable time (34% vs. 42%) when compared to the baseline wave.
  • Little progress was made on improving satisfaction around key pain points identified in the baseline study. There has been no statistically significant shifts since the 2017-18, and notably several measures have softened including ease of issue resolution, ease of figuring out eligibility, ease of finding steps to apply and overall satisfaction.
• No significant movement among SIN clients.

Key Findings: Overall Drivers of Satisfaction

• The most important drivers of satisfaction were: the amount of time it took from start and to finish was reasonable, the helpfulness of Service Canada call centre phone representatives and ease of getting assistance when needed. 
  • To improve the service experience for Service Canada clientele as a whole, focus should be placed primarily on improving the timeliness of service, ease of getting assistance, and the helpfulness of Service Canada call centre phone representatives. These represent service attributes most strongly related to overall satisfaction where performance was lower relative to other areas.
  • This year the impact of whether the application was approved or denied took on increased importance in driving satisfaction. Results indicate that satisfaction among those denied the benefit for which they applied has declined, particularly for CPP and EI clients, and has a bigger impact on their impressions of their experience than in previous years. While there were select drivers that have a stronger impact on satisfaction, improving the experience for those denied a benefit will also be important to helping improve overall satisfaction.
• Drivers of satisfaction differ significantly by program. The most common top drivers were consistent with Service Canada clientele as a whole and include the helpfulness of call centre representatives for EI, CPP and CPP-D clients, the timeliness of service for EI and CPP-D clients and the ease of getting assistance for EI and SIN clients.
  • Current areas that were performing strongly and were correlated to satisfaction include service in choice of official language, completing application in a reasonable amount of time, and confident personal information was protected. These attributes should be maintained moving forward to protect these strengths.
  • The greatest opportunities to improve service across programs which represent areas strongly correlated to satisfaction where performance was lower to other areas include improving the helpfulness of call centre representatives (for EI, CPP and CPP-D), the timelessness of services (for EI and CPP-D), the ease of getting assistance (for EI and OAS/GIS) and the amount of travel required to access service (for EI and SIN).
  • For CPP-D clients, it will also be important to improve the clarity of requirements for the application.
  • For OAS/GIS clients, it will also be important to focus on improving the ease of finding information about the program, the ease of following up and confidence in issue resolution.

Key Findings: Service Channel Assessment

• Clients continued to be most satisfied with the in-person experience, while satisfaction remained lowest for specialized call centres. Satisfaction with the specialized call centres and Government of Canada websites declined for the second consecutive year.
  • As in previous years, the large majority of clients remained satisfied with each service channel.  Satisfaction with in-person service continued to be the highest (86%), followed by MSCA (75%), online (73%), 1 800 O-Canada (68%) and specialized call centres (60%).
  • For the second consecutive year, satisfaction among those who used specialized call centres declined with fewer clients providing the highest ratings. Fewer clients also expressed satisfaction with online compared to the previous year due to a reduction in those providing a rating of 4 out of 5.
• Satisfaction with service channels differed by program. CPP-D clients rated their satisfaction with in-person service and online lower compared to all clients.
  • Compared to 2018-19, EI clients provided lower ratings for online (71% vs. 79%) and specialized call centres (59% vs. 74%).
  • CPP (67% vs. 80%) and OAS/GIS (64% vs. 79%) clients also provided lower ratings for specialized call centres, while SIN clients provided lower ratings for in-person service (90% vs. 94%).
• Nearly all clients agreed they were provided service in their choice of English or French, and that it was easy to access service in a language they could speak and understand well. 
• Self-service clients continued to be well served by the online application process however EI clients experienced more difficulty understanding requirements of the application than in 2017-18.
  • The vast majority of self-service clients found it easy to understand the requirements of the application, put together the information needed and to complete the application in a reasonable amount of time.  EI clients who self-served were less likely to find it easy to understand the requirements of the application than in the baseline wave.
• MSCA was utilized by the vast majority of EI clients and half of CPP clients, while CPP-D or OAS/GIS clients were much less likely.  Notably, three in ten CPP-D clients (31%) and four in ten OAS/GIS clients (40%) had followed-up through MSCA. EI clients found the registration process easy, while clients from other programs had more difficulty.
  • Over three-quarters of EI clients (77%) and half of CPP clients (50%) used MSCA during their experience. Four in ten CPP-D clients (39%) and one-third OAS/GIS (32%) clients also did so. The vast majority of those registered for MSCA for the first time felt the process was easy (69%).  EI clients were more likely to have found the process easy, while CPP, CPP-D and OAS/GIS clients felt it was more difficult.
  • Overall, three-quarters (75%) of clients who utilized MSCA were satisfied. Satisfaction in Quebec (87%) was higher compared to all clients, while ratings were lowest in the West (70%).
• 1 800 O-Canada was utilized by a limited proportion of clients at the awareness stage and usage did not differ significantly across most vulnerable client groups. Satisfaction was lower among Indigenous clients and those with restrictions.
  • Fewer than one in ten (8%) clients used 1 800 O-Canada at the aware stage. Usage was consistent among most vulnerable groups but was higher among clients with disabilities and those with no devices (no computer, smartphone or tablet).  At seven in ten (69%), a strong majority of clients were satisfied with their experience with 1 800 O-Canada. Satisfaction with the channel was lower among Indigenous clients (52%) and clients with restrictions (55%).

Key Findings: Barriers to Accessing Service

• Clients with restrictions that make it more difficult to access service, and clients with disabilities continued to have lower satisfaction than other clients. Use of the online channel remained particularly challenging for both client groups.
  • Clients who experienced a restriction to accessing service (35% of the sample population) were more likely to have experienced a problem and have lower satisfaction with service provided in-person, through 1 800 O-Canada or MSCA. They were also less satisfied with several service attributes with the largest gaps for ease of registering for MSCA, ease of completing the application form, overall ease of applying, being able to complete steps online made the process easier for you and confidence in issue resolution.
  • Restrictions to accessing service were more prevalent among several vulnerable client groups, in particular clients with disabilities, those with a language barrier, non-English for French speaking clients. Incidence of restrictions were also higher among Indigenous clients, mobile-only clients, clients with no devices and e-vulnerable clients.
  • Clients who self identify as having a disability (7% of the sample population) were significantly more likely to have experienced a problem and rate channel satisfaction lower for in-person service. They were also less satisfied with several service attributes with the largest gaps for being able to complete steps online made the process easier for you, ease of finding the information needed to provide when applying, ease of understanding the requirements of the application and ease of putting together the information needed.
  • Compared to 2018-19, there were improvements across a number of service attributes including needing to explain your situation only once, being able to complete steps online made the process easier for you and confidence your personal information was protected.

Key Findings: Vulnerable Groups

• Among the 16 vulnerable client groupings, most provided high ratings of the service experience. Satisfaction among Indigenous clients and those living in remote areas declined year over year.
  • Satisfaction with the service experience remained strong and the vast majority of clients in vulnerable groups rated it positively. Notably, satisfaction among Newcomers was higher than compared to all clients.
  • Overall satisfaction among Indigenous clients (specifically those in remote areas) and clients living in remote (in general) declined compared to 2018-19. Satisfaction among Indigenous clients was also lower this year compared to all clients.
  • Compared to 2018-19, Indigenous clients were more likely to have experienced a problem and have lower satisfaction with service provided online, through the specialized call centre or 1 800 O-Canada. Satisfaction also declined for ease of applying, ease of getting help when needed, moving smoothly through all steps and clarity of the issue resolution process.
  • Indigenous clients were less likely to feel it was easy to figure out if they were eligible for benefits, finding the steps to apply, understanding the requirements of the application, completing the form and moving smoothly through all steps compared to all clients.
  • Among clients living in remote areas, satisfaction declined overall but remained consistent with all clients. Ratings declined for being able to move smoothly through all steps, clarity of process, easy to get help when needed and receiving consistent information. Notably, remote clients were more likely however to feel confident their application would be processed in a reasonable amount of time compared to all clients.