Client Service Feedback Survey 2021-2022

Analysis Report

Contract Information

This report presents the results and methodological details for the Client Service Feedback Survey 2021-2022 conducted by Advanis Inc. on behalf of Public Services and Procurement Canada (PSPC). The survey was administered among 1,622 pension members, between March 30 and May 2, 2022.

Ce rapport est aussi disponible en français sous le titre: Sondage sur la rétroaction du service à la clientèle 2021-2022.

This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Public Services and Procurement Canada (PSPC). For more information on this report, please contact PSPC at:

TPSGC.Politiquesdepension-PensionPolicies.PWGSC@tpsgc-pwgsc.gc.ca

• Catalogue Number: P54-3/2022E-PDF
• International Standard Book Number (ISBN): 978-0-660-44295-2

• Related publications (registration number: POR 130-20):

  Catalogue number: P54-3/2022F-PDF

  International Standard Book Number (ISBN): 978-0-660-44296-9

• © His Majesty the King in Right of Canada, as represented by Public Services and Procurement Canada (PSPC), 2022

1. Summary

1.1 Background and objectives

Public Services and Procurement Canada (PSPC) provides day-to-day pension services through two service channels, the Government of Canada Pension Centre, and the Pension Program secure employee portal for pension plan members belonging to the Public Service, Canadian Forces, and Royal Canadian Mounted Police.

PSPC, in creating a culture of client service excellence, wishes to evolve service delivery based on feedback from its members. To that end, PSPC was seeking to measure member satisfaction with regards to the Pension Program’s service delivery channels.

The primary objectives of the research were to:

• Assess members’ (pension members) satisfaction of services offered by the Government of Canada Pension Centre during service calls or with any services used online by members.
• Assess pension members' ease of use, understanding and experience with pension online tools or content.
• Assess the importance that pension members place on different online web applications and future applications.

The probability-based study was conducted in English and French online and over the phone. Overall, 1,622 people completed the survey between March 30, 2022, and May 2, 2022. The data was weighted according to the pension group of the respondents (PSSA, CFSA, or RCMP) and service line (online vs phone) from the population of pension plan members that were in contact with the Government of Canada Pension Centre or accessed the Online Portal between February 21, 2022, and March 23, 2022.

The results will be used by PSPC to identify opportunities for improvements in the way they deliver services.

The total cost of this research was $28,773.00 (before taxes).

1.2 Key findings

Most pension members who answered the survey were satisfied with the service they received (78.9%). It was the case for 88.1% of those who contacted Pension Centre and 75.0% of those who used the Online Portal.

Requesting general pension information, a form or documentation was the main reason for contacting the Pension Centre (42.7%). Those who contacted the Pension Centre were especially satisfied with the courteousness (96.0%), the helpfulness (91.1%) and the knowledge of the staff (88.7%).

Getting a pension estimate was the main use of the Online Portal (69.8%). Online Portal users were satisfied with the ease of login (80.2%), the ease of landing page navigation (71.3%) and the ease of self-service options (71.3%).

More than two thirds (68.0%) reported the information was easy to understand (score of at least 8 on a scale of 1 to 10). This proportion was 81.7% for those who called the Pension Centre and of 62.1% for Online Portal users.

Eight respondents out of ten (80.0%) reported they were very likely (score of at least 8 on the 10-point likeliness scale) to access personal pension information from a home, personal laptop if this was available.

Among Pension Centre respondents, 56.2% liked that the agents were helpful, knowledgeable, or were able to provide answers the most. However, 46.1% mentioned that the information provided could be improved and 31.1% thought delays could be shortened.

What Online Portal users liked the most was its user-friendly aspect (33.9%) and having the ability to estimate their pension (31.1%). More than half (52.5%) thought that the information provided could be most improved.

1.3 Extrapolating the results to a broader audience

Since the target population only includes respondents who had recently called the Pension Centre or visited the Online Portal during a specific period, results cannot be extrapolated to another period or for the broader population

1.4 Political Neutrality Certification

Political neutrality certification

I hereby certify as Senior Officer of Advanis that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Policy on Communications and Federal Identity and the Directive on the Management of Communications.

Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.

Signed

Nicolas Toutant

Vice-President, Research and Evaluation

Advanis

2. Methodology

2.1 Sample planning and data collection

The target population for the survey was both retired and active pension members, more specifically:

• Public Service Superannuation Plan active employees and retired members (PSSA), including Crown Corporation active employees and retired members.
• RCMP Superannuation Plan active officers and retired members (RCMPSA).
• Canadian Forces Superannuation Plan active regular force and reservist and retired members (CFSA).

This survey was sent to members of the target population who had recently called the Government of Canada Pension Centre or visited the Pension Program secure employee portal. To reach this population, Advanis used a multimodal approach. Potential respondents who recently visited the Online Portal were invited by email to participate in the survey. Potential respondents who recently called the Government of Canada Pension Centre were also invited to participate in the web survey and some were called using a computer-assisted telephone interviewing (CATI) methodology.

Two different samples were sent by PSPC. The first contained a list of active and retired members who recently called the Government of Canada Pension Centre and included information to reach them by phone and by email. The second contained a list of active members who recently visited the Online Portal and included information to reach them by email only.

Data collection started on March 30, 2022 and ended on May 2, 2022. A pretest took place from December 15, 2021, to December 22, 2021. The Government of Canada’s standards for pre-testing were adhered to, with pretests being conducted in both English and French. In total, 96 surveys were completed during the pretest (23 by phone and 73 on the web). The pretest data was not retained for the analysis because a significant amount of time had passed between the pretest and data collection.

Email invitations were sent for Online Portal potential respondents and Pension Centre potential respondents who had not answered the survey on the phone or had not stated their refusal. In total, 3,206 members were invited via email to participate in the study. On March 30, 2022, 1,000 Online Portal respondents were invited and 2,206 Pension Centre respondents were invited on April 11, 2022. Up to two email reminders were sent to those who had not yet completed the survey and had not stated their refusal to participate.

Data collection on the phone was performed from March 31, 2022, to April 5, 2022, with members who had recently called the Pension Centre. A total of 2,965 Pension Centre potential respondents were called. Of those, 97 had a number that was not in service, a wrong number, or could not be reached for reasons related to their phone number. It is to be noted that 502 respondents agreed to participate in the survey, however, 14 were not eligible.

In total, 1,622 respondents participated in the Client Service Feedback Survey 2021-2022. Of those 1,622, a total of 1,255 were respondents who recently called the Pension Centre and 367 as respondents who recently visited the Online Portal. All Online Portal respondents answered the web survey. A total of 488 Pension Centre respondents answered the survey on the phone and 767 answered on the web.

2.2 Questionnaire

The Client Service Feedback Survey 2021-2022 was designed by PSPC. The questionnaire was programmed by Advanis in both English and French. An electronic version of the survey was provided to PSPC for review at several steps. After incorporating changes from PSPC, a final version in both official languages for both CATI and web was created.

The average survey length was 5.6 minutes for completing the web survey and 6.9 minutes for completing the survey over the phone.

2.3 Calls Monitoring

The data collection period started after the questionnaire had been validated and tested.

The interviewers’ work schedule extended from 4:00 p.m. to 9:00 p.m. Monday to Friday, and from 10:00 a.m. to 6:00 p.m. on Saturdays and Sundays. Occasionally, interviews began calling earlier if requested by a respondent (appointments monitoring).

The call-back plan distributed calls during the day and over the weekend at different hours.

2.4 Weighting

The data was weighted according to the pension group of the respondents (PSSA members, CFSA members or RCMP members) and whether the respondent recently contacted the Pension Centre on the phone or recently consulted the Online Portal (Pension Centre respondent vs. Online Portal respondent), using the population files sent by PSPC. The first list contained all members who had called the Pension Centre between February 21, 2022, and March 23, 2022. The second list contained all members who had visited the Online Portal between February 21, 2022, and March 22, 2022.

3. Note to readers

The term "respondent" will be used throughout this report to refer to all participants in the study. All survey results are presented excluding "Don’t know" answers from the base of valid responses.

In this report, Pension Centre respondents refers to the respondents who had recently called the Pension Centre and Online Portal respondents refers to those who had recently visited the Pension Program secure employee portal.

Readers should also keep in mind that the total for percentages presented in tables and charts may not sum to 100.0% due to rounding. Indeed, a maximum variation of more or less 1.0% is to be expected.

4. Results

4.1 Respondents’ profile (unweighted data)

More than 3 quarters (77.4%) of respondents had recently called the Pension Centre. The others had recently visited the Online Portal (22.6%).

Most respondents were PSSA members (77.6%), 18.9% were CFSA members and 3.5% were RCMP members. Among Online Portal respondents, the vast majority was a PSSA member (95.1%). Almost three quarters (72.5%) were PSSA members among Pension Centre respondents.

Among all respondents, 61.8% were currently employed by the Federal government, more than a third (35.0%) were retired, and only 0.4% were survivors. Online Portal respondents were almost all currently employed (99.5%) and Pension Centre respondents were more evenly split between currently employed (50.8%) and retired (45.3%) members.

More than half (53.8%) of respondents were between 40 and 59 years old (49.1% of Pension Centre respondents and 69.7% of Online Portal respondents). More Pension Center respondents were 60 years old or older compared to Online Portal respondents (38.2% compared to 15.8%).

Most respondents (68.0%) were not members of a visible minority, living with a disability, LGBTQ2+ or Indigenous.

Among those who answered the survey, 50.8% were female, 49.1% were male and 0.1% were non-binary.

4.2 Reasons for contacting the Pension Centre or visiting the Online Portal (weighted data)

The main reason for calling the Pension Centre was to request general pension information, a form or documentation (42.7%). More than a fifth (20.4%) called to request a buyback estimate or to apply to buy back service. Almost a fifth called regarding their pension payments or to get an estimate of their pension (18.7%) or to update their profile (18.1%).

Getting a pension estimate was by far the main reason for using the Online Portal (69.8%). More than a fifth (22.9%) used the Online Portal for the buyback estimator or to get their payment status.

4.3 Satisfaction (weighted data)

Overall, most respondents were satisfied with the service they received (78.9%). Those who called the Pension Centre were satisfied in a proportion of 88.1%. It was the case for 75.0% of Online Portal respondents.

The courteousness of the staff was the item Pension Centre respondents were the most satisfied with (96.0%), followed by the helpfulness of the staff (91.1%) and the knowledge level of the staff (88.7%). The ease of navigating the automated phone system was the item Pension Centre respondents were the least satisfied with, even though more than three quarters were satisfied (76.9%).

For their part, 8 Online Portal respondents out of 10 (80.2%) were satisfied with the ease of login on the Online Portal. A little more than 7 respondents out of 10 were satisfied with the ease of landing page navigation and the ease of self-service options (71.3% and 70.4%, respectively).

4.4 Understanding the information provided (weighted data)

More than two thirds (68.0%) considered that the information they were provided was easy to understand by giving a score of at least 8 on a 10-point scale. Pension Centre respondents were 81.7% to give a score of at least 8, whereas it was the case for 62.1% of Online Portal respondents. Respondents who were 60 years old or older were more likely to give a score of at least 8 (76.3%) than those who were 40 to 59 years old (68,4%) and less than 40 years old (53.8%).

4.5 Contact options

Accessing personal pension information from a personal laptop is the contact option respondents would be most likely to use, if available. Indeed, 80.0% reported they would use it by giving a score of at least 8 on a 10-point likeliness scale. This proportion was 63.0% for virtual meetings with the Pension Centre.

Among Pension Centre respondents only, accessing personal pension information is followed by virtual meetings with the Pension Centre as well. The contact option Pension Centre respondents would be the least likely to use is a webchat.

Online Portal respondents also had accessing personal pension information from a personal laptop and virtual meetings as contact options they would be most likely to use. Receiving SMS notices or reminders is the contact option they would be the least likely to use.

4.6 Strengths and things to improve (weighted data)

When asked what respondents liked most about their interaction with the Pension Center was, 56.2% of Pension Centre respondents reported that agents they have interacted with were helpful, knowledgeable, or were able to provide answers. A proportion of 37.9% mentioned the agent’s soft skills and 31.8% praised the efficiency of the Pension Centre.

Even though many respondents mentioned they like the fact that agents were helpful or knowledgeable, 46.1% of those who mentioned things that could be improved also think that the information provided by the Pension Centre could be improved. Delays were also mentioned as something that could be improved by a high proportion of respondents to this question (31.1%). When mentioning delays, a lot of respondents were referring to delays on the phone when waiting to receive a service from the Pension Centre. Some also mentioned delays to get the answer to a question, to receive documents or to process a request.

For Online Portal respondents, the most liked aspects of their interaction with the Online Portal were finding it user-friendly (33.9%) and having the ability to estimate their pension (31.1%). More than a quarter (26.4%) also mentioned enjoying the ability to access information.

Among responses obtained, the aspect of the Online Portal that could be improved was the information provided. Indeed, more than half (52.5%) reported that it could either contain additional information or give more detailed information. Almost a quarter (23.0%) mentioned that the calculator or the method for getting a pension statement could be improved.

5. Conclusion

In general, most respondents were satisfied with the service they received either when contacting the Pension Centre (88.1%) or using the Online Portal (75.0%).

Respondents who contacted the Pension Centre were especially satisfied with the courteousness (96.0%), the helpfulness (91.1%) and the knowledge of the staff (88.7%). Those who used the Online Portal were mostly satisfied with the ease of login (80.2%).

The main reason for contacting the Pension Centre was to request general pension information, a form or documentation (42.7%). For the Online Portal, its main use was to get a pension estimate (69.8%).

Overall, more than two thirds (68.0%) reported the information was easy to understand by giving a score of at least 8 on a scale of 1 to 10. This proportion was 81.7% for those who called the Pension Centre and of 62.1% for those who used the Online Portal.

A proportion of 80.0% reported they were very likely (score of at least 8 on the 10-point likeliness scale) to access personal pension information from a home, personal laptop if this was available. This contact option was followed by the ability to have a virtual meeting with the Pension Centre (63.0%).

When asked what they liked the most, Most Pension Centre respondents (56.2%) reported they liked that the agents were helpful, knowledgeable, or were able to provide answers. However, among those who mentioned things that could be improved, 46.1% mentioned that the information provided could be improved.

More than a third (33.9%) of those who used the Online Portal reported that they like the user-friendly aspect and 31.1% reported that they liked having the ability to estimate their pension. Similar to Pension Centre respondents, 52.5% of Online Portal users who mentioned things that could be improved mentioned the information provided.

6. Appendix

6.1 Appendix A: Methodology

The Client Service Feedback Survey 2021-2022 was designed by PSPC. The questionnaire was programmed by Advanis in both official languages.

The survey was administered among members of the target population who had recently called the Government of Canada Pension Centre or visited the Pension Program secure employee portal.

The target population was both retired and active pension members, more specifically:

• Public Service Superannuation Plan active employees and retired members (PSSA), including Crown Corporation active employees and retired members.
• RCMP Superannuation Plan active officers and retired members (RCMPSA).
• Canadian Forces Superannuation Plan active regular force and reservist and retired members (CFSA)

Online portal users were sent an email inviting them to participate in a web survey. Potential respondents who recently called the Government of Canada Pension Centre were invited to participate in the web survey as well. Some were also called using a computer-assisted telephone interviewing (CATI) methodology.

Two different samples were sent by PSPC. The first contained a list of 3,450 active and retired members who recently called the Government of Canada Pension Centre and included information to reach them by phone and by email. The second contained a list of 15,925 active members who recently visited the Online Portal and included information to reach them by email only. Data collection started on March 30, 2022, and ended on May 2, 2022.

In total, 3,206 members were invited via email to participate in the study. On March 30, 2022, 1,000 Online Portal respondents were invited and 2,206 Pension Centre respondents were invited on April 11, 2022. Up to two email reminders were sent to those who had not yet completed the survey and had not stated their refusal to participate. A total of 2,965 Pension Centre potential respondents were called. Data collection on the phone was performed from March 31, 2022, to April 5, 2022.

Appendix B: Email invitations sent

• bounced: Invalid email address
• clicked: Respondent clicked on the link included in the email
• opened: Respondent read the email but did not click on the link
• sent: Valid email address but no action was taken

Appendix C: Call statistics (Pension Centre)

Appendix D: Response rate

For the consultation among those who had recently visited the Pension Centre, the response rate was calculated by dividing the number of respondents (1255) by the number of eligible members in the sample (3,003) for a response rate of 41.8%.

For those who visited the online portal, the response rate was calculated by dividing the number of respondents (367) by the number of initial email invitations who reached potential respondents (908). Hence the response rate for the Online portal portion of this study was 40.4%.

Appendix E: The weights

As previously mentioned, the data was weighted according to the pension group of the respondents (PSSA members, CFSA members or RCMP members) and service line (online vs phone), using the population files sent by PSPC. The first list contained all members who had called the Pension Centre between February 21, 2022, and March 23, 2022. The second list contained all members who had visited the Online Portal between February 21, 2022, and March 22, 2022.

Appendix F: Additional notes

Since the target population only includes respondents who had recently called the Pension Centre or visited the Online Portal during a specific period, results cannot be extrapolated to another period or for the broader population.

When surveying only a sample of a broader population, there are always risks that results suffer from a non-response bias. This happens when characteristics of those who answered the survey differ from those who did not answer. For this study, several strategies were employed to increase response rates and reduce the effects of non-response bias. This includes communicating the purpose and importance of research at the beginning of the survey as well as reassuring respondents on the confidentiality of their responses and on the legitimacy of the survey.

Appendix G: English questionnaire

Canada Pension Members

Government of Canada

1

Intro

The Minister of Public Services and Procurement of Canada (PSPC) is conducting a survey to learn more about your experience as a pension plan member when accessing our services. Feedback from you is essential to improve the services.

Note: If you need to leave the survey and come back to it later, you may use the same link and the survey will resume where you left off.

Please click on the right arrow to continue

Your participation is voluntary and your answers will remain strictly confidential.

© 2022 Advanis Privacy Policy CRIC Pledge

Q1

Was your most recent interaction with the Government of Canada Pension Center or using the Employee Pension Portal online?

• ○ (1) Government of Canada Pension Centre
• ○ (2) Online Portal

Q2a

What was the purpose of your call to the Pension Centre?

Select all that apply

• □ (1) Pension payments / Request an estimate of my pension
• □ (2) Obtain an update of the status of a transfer payment (transfer value, pension transfer agreement)
• □ (3) Request a buyback estimate and/or apply to buy back service
• □ (4) Request general pension information, a form or documentation
• □ (5) Update my profile (name, address, banking information, etc.)
• □ (6) Life Events (Enrollments, marriage status, Supplementary Death Benefit)
• □ (97) Other (please specify): 

Q2b

What was the reason for your use of the Online Pension Portal?

Select all that apply

• □ (1) Get a pension estimate
• □ (2) Update personal information (address, contact details, etc.)
• □ (3) Service buyback estimator / payment status
• □ (4) Survivor benefits estimator
• □ (5) Other (please specify):

Q3a

Thinking back on your recent experience with the Government of Canada Pension Centre, how would you rate your satisfaction related to the following?

• 1. Ease of navigating the automated phone system
• 2. The courteousness of staff
• 3. The helpfulness of staff
• 4. The knowledge level of staff
• 5. The clarity of the information provided to you
• 6. The timeliness of the service you received

• ○ (1) Very satisfied
• ○ (2) Satisfied
• ○ (3) Neutral
• ○ (4) Dissatisfied
• ○ (5) Very dissatisfied
• ○ (-8) Not applicable

Q3b

Thinking back on your recent experience using the Online Pension Portal, how would you rate your satisfaction related to the following?

• 1. Ease of login
• 2. Ease of Landing Page navigation
• 3. Ease of self-service options

• ○ (1) Very satisfied
• ○ (2) Satisfied
• ○ (3) Neutral
• ○ (4) Dissatisfied
• ○ (5) Very dissatisfied
• ○ (-8) Not applicable

Q4

On a scale of one to 10, was the information you were provided, easy to understand? [[The information being referred to would be from the pension expert on the phone, or the information found on the portal. ]]

• ○ (10) 10 - Easy to understand
• ○ (9) 9
• ○ (8) 8
• ○ (7) 7
• ○ (6) 6
• ○ (5) 5
• ○ (4) 4
• ○ (3) 3
• ○ (2) 2
• ○ (1) 1 - Did not understand at all
• ○ (-8) Not applicable

Q5

How would you rate your experience with the service you received?

• ○ (5) Very Satisfied
• ○ (4) Somewhat Satisfied
• ○ (3) Neither Satisfied nor Dissatisfied
• ○ (2) Somewhat Dissatisfied
• ○ (1) Very Dissatisfied

Q6

On a scale of 1 to 10, how likely would it be for you to use the following contact options if they were available:

• 1. Accessing personal pension information from a home, personal laptop
• 2. Text (SMS) notices/reminders
• 3. Webchat/IM (Instant Messaging) with Pension Centre
• 4. Virtual meeting (Audio/Video call) with Pension Centre
• 5. Mobile application

• ○ (10) 10 - Very likely
• ○ (9) 9
• ○ (8) 8
• ○ (7) 7
• ○ (6) 6
• ○ (5) 5
• ○ (4) 4
• ○ (3) 3
• ○ (2) 2
• ○ (1) 1 - Not likely at all

3

D1

What is your current status at the Government of Canada Pension Centre?

• ○ (1) Active (currently employed)
• ○ (2) Non-Active (retired)
• ○ (3) Survivor
• ○ (4) Other (please specify):

D2

What age group do you fall under?

• ○ (1) 19 and under
• ○ (2) 20 - 29
• ○ (3) 30 - 39
• ○ (4) 40 - 49
• ○ (5) 50 - 59
• ○ (6) 60 - 69
• ○ (7) 70 or older
• ○ (-8) I prefer not to answer

D3

Do you belong to any of the following groups?

Select all that apply

• □ (2) Indigenous peoples
• □ (3) LGBTQ2+
• □ (4) Persons with disabilities
• □ (5) Members of a visible minority
• □ (1) No, I do not belong to any of the following groups
• □ (7) Prefer not to answer

D4

Please identify your gender :

• ○ (1) Male
• ○ (2) Female
• ○ (3) Non-Binary
• ○ (4) Other (please specify): 
• ○ (-8) Prefer not to answer

Q7a

Overall, what is the one thing you liked most about your interaction with the Government of Canada Pension Centre?

____________________

Q7b

Overall, what is the one thing you liked most about your interaction with the Online Pension Portal?

____________________

Q8a

Overall, what is the one thing you believe could be improved on most to make your interaction with the Government of Canada Pension Centre a more positive experience?

____________________

Q8b

Overall, what is the one thing you believe could be improved on most to make your interaction with the Online Pension Portal a more positive experience?

____________________

PretestQs [FALSE]

P1

We would now like your opinion on this survey itself.

Overall, how easy were the questions in the survey to understand? Please use a scale from 0 to 10 where 0 means very difficult, and 10 means very easy.

• ○ (0) 0 - Very difficult
• ○ (1) 1
• ○ (2) 2
• ○ (3) 3
• ○ (4) 4
• ○ (5) 5
• ○ (6) 6
• ○ (7) 7
• ○ (8) 8
• ○ (9) 9
• ○ (10) 10 - Very easy

P2

Please explain your rating. For example, what specific words or questions did you find difficult to understand?

____________________

• ○ (-8) Prefer not to answer

4

t1

You have now completed the survey. Thank you very much for your time.

6.8 Appendix H: French questionnaire