Client Service Feedback Survey – #005

September 2023

Analysis Report

Contract Information

For more information on this report, please contact Public Services and Procurement Canada at: TPSGC.Politiquesdepension-PensionPolicies.PWGSC@tpsgc-pwgsc.gc.ca

Ce rapport est aussi disponible en français.

This report presents the results and methodological details for the Client Service Feedback Survey #005 conducted by Advanis Inc. on behalf of Public Services and Procurement Canada (PSPC). The survey was administered among 1,780 pension members, between July 6 and August 14, 2023.

Ce rapport est aussi disponible en français sous le titre: Sondage sur la rétroaction du service à la clientèle – #005.

This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Public Services and Procurement Canada (PSPC). For more information on this report, please contact PSPC at: TPSGC.Politiquesdepension-PensionPolicies.PWGSC@tpsgc-pwgsc.gc.ca

• Catalogue Number: P54-3/2023E-3-PDF
• International Standard Book Number (ISBN): 978-0-660-67861-0

• Related publications (registration number: POR 012-23):

  Catalogue number: P54-3/2023F-3-PDF

  International Standard Book Number (ISBN): 978-0-660-67862-7

• © His Majesty the King in Right of Canada, as represented by Public Services and Procurement Canada (PSPC), 2023

1.0 Summary

1.1 Background and objectives

Public Services and Procurement Canada (PSPC) provides day-to-day pension services through two service channels, the Government of Canada Pension Centre and the Pension Program secure employee portal, for pension plan members belonging to the Public Service, Canadian Forces, and Royal Canadian Mounted Police.

PSPC, in creating a culture of client service excellence, wishes to evolve service delivery based on feedback from its members. To that end, PSPC was seeking to measure member satisfaction with regards to the Pension Program’s service delivery channels.

The primary objectives of the research were to:

• Assess members’ (pension members) satisfaction of services offered by the Government of Canada Pension Centre during service calls or with any services used online by members.
• Assess pension members' ease of use, understanding and experience with pension online tools or content.
• Assess the importance that pension members place on different online web applications and future applications.

The probability-based study was conducted in English and French online and over the phone. Overall, 1,780 people completed the survey between July 6th, 2023, and August 14th, 2023. The data was weighted according to the pension group of the respondents (PSSA, CFSA, or RCMP) from the population of pension plan members that were in contact with the Government of Canada Pension Centre or accessed the Online Portal between May 31, 2023, and June 29, 2023. Pension Centre respondents and Online Portal respondents were weighted separately.

A first wave of this study was conducted between March 30th, 2022, and May 2nd, 2022, a second wave between August 3rd, 2022, and September 14th, 2022, a third wave between November 15th, 2022, and December 13th, 2022, and a fourth wave between February 8th, 2023, and March 7th, 2023. For these waves, a report was provided to PSPC. This report presents the results for the fifth wave of the study.

The results will be used by PSPC to identify opportunities for improvements in the way they deliver services.

The total cost of this research was $44,640.65 (before taxes).

1.2 Key findings

Pension Centre respondents

The main reason respondents called the Pension Centre was to request general information (48.3%). Generally, they found that the information provided was easy to understand (80.0% gave a score of at least 8 on a scale of 10).

Respondents who contacted the Pension Centre were for the most part satisfied with the service they received (86.4%). The aspects they were the most satisfied with were related to the staff. Indeed, they found the staff courteous (94.8%), helpful (89.9%) and knowledgeable (88.2%). They were less inclined to find that the automated phone system was easy to navigate, but the majority was still satisfied (75.4%).

When asked about the aspect they liked the most about their call to the Pension Centre, the most common answers were related to the agents’ helpfulness (62.6%) and their soft skills (49.6%).

Having better access to agents (24.2%), the knowledge of the agents and the information provided (18.9%) and delays (14.2%) were the aspects they thought could be improved the most.

Among different online/virtual contact options, Pension Centre respondents would be most likely to access their pension information from a home, personal computer (78.3%).

Online Portal respondents

The main reason for using the Online Pension Portal was to get a pension estimate (71.6%). The majority of those who used the Online Portal were satisfied with the service they received (69.8%). A proportion of 81.0% were satisfied with the ease of login and the majority said the information provided by the Online Portal was easy to understand (59.4% gave a score of at least 8 on a scale of 10). Satisfaction tended to be higher among older respondents compared to younger respondents.

The ability to estimate their pension was the aspect respondents liked the most (42.0%) and a proportion of 33.2% liked the fact that it was user-friendly the most.

Furthermore, the information (quality or quantity) was the element respondents would wish to see improved the most (27.6%) followed by the user-friendliness and modernizing the platform (17.8%) and issues related to the calculator (17.6%).

Online Portal respondents would be most likely to access their pension information from a home, personal computer (80.0%), among online/virtual contact options.

1.3 Extrapolating the results to a broader audience

Since the target population only includes respondents who had recently called the Pension Centre or visited the Online Portal during a specific period, results cannot be extrapolated to another period or for the broader population.

1.4 Political Neutrality Certification

Political neutrality certification

I hereby certify as Senior Officer of Advanis that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Policy on Communications and Federal Identity and the Directive on the Management of Communications.

Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.

Signed:

Nicolas Toutant

Vice-President, Research and Evaluation

2.0 Methodology

2.1 Sample planning and data collection

The target population for the survey was both retired and active pension members, more specifically:

• Public Service Superannuation Plan active employees and retired members (PSSA), including Crown Corporation active employees and retired members.
• RCMP Superannuation Plan active officers and retired members (RCMPSA).
• Canadian Forces Superannuation Plan active regular force and reservist and retired members (CFSA)

This survey was sent to members of the target population who had recently called the Government of Canada Pension Centre or visited the Pension Program secure employee portal. To reach this population, Advanis used a multimodal approach. Potential Online Portal respondents were invited by email to participate in the survey. Potential Pension Centre respondents were also invited to participate in the web survey, and some were called using a computer-assisted telephone interviewing (CATI) methodology.

Two different samples were sent by PSPC. The first contained a list of active and retired members who recently called the Government of Canada Pension Centre and accepted to be contacted. It included information to reach them by phone and by email. The second contained a list of active members who recently visited the Online Portal and included information to reach them by email only. For privacy reasons, members with only a personal email address listed were removed from the Online Portal list. Only those with an institutional email address at the Government of Canada were kept.

Data collection started on July 6, 2023, and ended on August 14, 2023. A pretest took place on the phone on July 6, 2023, and online on July 26, 2023. The Government of Canada’s standards for pre-testing were adhered to, with pretests being conducted in both English and French. In total, 46 surveys were completed during the pretest (26 by phone and 20 online). The pretest data was retained for the analysis.

Email invitations were sent for Online Portal potential respondents and Pension Centre potential respondents who had not answered the survey on the phone and had not stated their refusal. In total, 4,811 members were invited via email to participate in the study. From July 28 to August 9, 2023, 2,571 Online Portal potential respondents were invited. On July 28, 2023, 2,240 potential Pension Centre respondents were invited. For potential Pension Centre respondents, up to two email reminders were sent to those who had not yet completed the survey and had not stated their refusal to participate. Only one reminder was necessary for potential Online Portal respondents.

Data collection on the phone was performed from July 6, 2023, to July 21, 2023. A total of 3,001 members were called. Of those, 144 had a number that was no longer in service, a wrong number, or could not be reached for reasons related to their phone number. It is to be noted that 507 agreed to participate in the survey, however, 7 were not eligible.

In total, 1,780 respondents participated in the Client Service Feedback Survey – #005. Of those 1,780, a total of 1,084 was coming from the Pension Centre potential respondents’ sample file and 696 were coming from the Online Portal potential respondents’ sample file. All respondents invited from the Online Portal sample file answered the web survey. A total of 500 members invited as Pension Centre respondents answered the survey on the phone and 584 answered on the web.

However, in the questionnaire, respondents were first asked if they had called the Government of Canada Pension Centre or used the Online Portal most recently. This allowed them to confirm if they should answer the survey as members who called the Pension Centre or visited the Online portal. Hence, a few respondents answered the Online Portal survey even if they were invited as Pension Centre respondents, and vice versa, as they likely had accessed both services recently. We then ended up with 1,160 Pension Centre respondents and 620 Online Portal respondents.

2.2 Questionnaire

The Client Service Feedback Survey – #005 was designed by PSPC. The questionnaire was programmed by Advanis in both English and French. An electronic version of the survey was provided to PSPC for review at several steps. A final version in both official languages for both CATI and web was created and sent to PSPC for approval.

The questionnaire was the same used during the fourth wave, with one question added asking about the perception on whether the pension supports the financial wellbeing of the respondent (Q10). The average survey length was 4 minutes and 44 seconds for completing the web survey and 8 minutes 22 seconds for completing the survey over the phone.

2.3 Calls Monitoring

The data collection period started after the questionnaire had been validated and tested.

The interviewers’ work schedule extended from 4:00 p.m. to 9:00 p.m. Monday to Friday, and from 10:00 a.m. to 6:00 p.m. on Saturdays and Sundays. Occasionally, interviews began calling earlier if requested by a respondent (appointments monitoring).

The call-back plan distributed calls during the day and over the weekend at different hours.

2.4 Weighting

The data was weighted according to the pension group of the respondents (PSSA members, CFSA members or RCMP members). Pension Centre and Online Portal respondents were weighted separately. The weights were calculated using the population files sent by PSPC. The list contained all members who had called the Pension Centre or who have visited the Online Portal and consented to participate between May 31, 2023, and June 30, 2023.

3.0 Note to readers

The respondents answered the survey in light of their experience with either the Pension Centre or the Online Portal. Hence, the survey results section of this report is divided in two subsections. The section referring to Pension Centre respondents presents the survey results for those who had recently called the Pension Centre. The section referring to Online Portal respondents presents results for those who had recently visited the Pension Program secure employee portal.

All survey results are presented excluding “Don’t know” answers from the base of valid responses.

Readers should also keep in mind that the total for percentages presented in tables and charts may not sum to exactly 100.0% due to rounding.

4.0 Results

4.1 Respondents’ profile (unweighted data)

A proportion of 65.2% of respondents had recently called the Pension Centre. The others had recently visited the Online Portal (34.8%).

The majority of Pension Centre respondents were PSSA members (90.9%). A proportion of 5.1% were part of the RCMP pension group and the remaining were CFSA members (4.0%). For Online Portal respondents, most were PSSA members (76.8%), while 11.3% were RCMP and 11.9% were CFSA members.

Pension Centre respondents were still employed in a proportion of 78.9%, while 21.1% were retired or had another status (including survivors and non-retired, but not currently working for the Government of Canada). The vast majority of Online Portal respondents was currently employed (96.6%).

Half (50.0%) of Pension Centre respondents willing to give their age were between 40 and 59 years old, while 25.0% were less than 40 and 24.9% were 60 years old or older. Among Online Portal respondents, 68.5% were between 40 and 59 years old, 16.1% were less than 40 15.4% were 60 or older.

A proportion of 62.1% of Pension Centre respondents were not members of a visible minority, living with a disability, LGBTQ2+ or Indigenous. This was also the case for 63.2% of Online Portal respondents.

Among Pension Centre respondents willing to give their gender, 61.4% were female and 38.4% were male. A proportion of 51.5% of Online Portal respondents identified as female and 48.3% as male.

4.2 Pension Centre results

4.2.1 Reasons for contacting the Pension Centre (weighted data)

The main reasons for calling the Pension Centre were to request general pension information, a form or documentation (48.3%), request a buyback estimate (35.1%) and regarding their pension payments or to request an estimate of their pension (25.1%).

4.2.2 Satisfaction with the Pension Centre (weighted data)

Overall, most Pension Centre respondents were satisfied with the service they received (86.4%). The satisfaction level was lower among employees who are less than 40 years old (83.0%) compared to the other age groups.

The courteousness of the staff was the item respondents were the most satisfied with (94.8%), followed by the helpfulness of the staff (89.9%) and their knowledge level (88.2%). The ease of navigating the automated phone system was the item they were the least satisfied with (75.4%).

The satisfaction levels were higher among respondent 60 or older for the helpfulness of the staff (93.3%) and the clarity of information provided (89.8%) whereas it was lower among respondents under 40 years old for the timeliness of the service (81.8%) received compared to older respondents.

4.2.3 Understanding the information provided by the Pension Centre (weighted data)

Most (80.0%) considered that the information provided was easy to understand by giving a score of at least 8 on a 10-point scale. Respondents who were 60 years old or older gave a score of at least 8 out of 10 in a greater proportion (85.1%) than younger respondents. Respondents who were less than 40 were less inclined to find the information easy to understand (72.7%).

4.2.4 Strengths and things to improve with the Pension Centre (weighted data)

When respondents were asked what they liked most of their interaction with the Pension Center, 62.6% reported that agents they have interacted with were helpful, knowledgeable, or were able to provide answers. Almost half (49.6%) mentioned the agents’ soft skills and 23.7% praised the efficiency of the Pension Centre.

Improving access to agents, which includes wait times on the phone and having different opening hours, is the most mentioned suggestion to improve the interaction with the Pension Centre (24.2%). Having more knowledgeable staff or improving the information provided was suggested by almost a fifth (18.9%).

Having better delays is also mentioned as something that could be improved by 14.2% of respondents as well as improving the phone system by 13.3%.

Having an online access or online services is also mentioned as something that could be improved by 18.3% of respondents as well as improving the phone system by 15.2%.

Other responses include the need to have more direct communication or have the same person assigned to a member. For example, to avoid waiting on the line, it was suggested to have a callback number to call directly to the person the respondent already spoke to. The ability to make video calls was also mentioned.

4.2.5 Contact Options

Among Pension Centre respondents, accessing personal pension information from a personal computer is the contact option they would be the most likely to use (78.3% gave a score of at least 8 on a 10-point likeliness scale). The contact option respondents would be the least likely to use is a webchat or Instant Messaging (44.1%), but this option is more popular with respondents under 40 years old (54.4%). Respondents that are 60 or older are the least likely to be interested in using a webchat or Instant Messaging (32.5%) or a mobile application (35.3%).

4.2.6 Perception of the benefits available

More than two thirds (71.0%) agreed that the benefits available under their pension plan support their wellbeing. This proportion is similar across the different age groups.

4.3 Online portal results

4.3.1 Reasons for visiting the Online Portal (weighted data)

Getting a pension estimate was by far the main reason for using the Online Portal (71.6%) while a fifth (19.8%) used the Online Portal to update personal information and 15.7% for the buyback estimator or to get their payment status.

4.3.2 Satisfaction with the Online Portal (weighted data)

More than two third (69.8%) of Online Portal respondents mentioned they were satisfied with the service they received. Respondents who are less than 40 are the least satisfied (59.0%). Those who are 60 years old or older were satisfied in a proportion of 76.4% and those and those between 40 and 59 were satisfied in a proportion of 71.1%. There is also a higher proportion of satisfaction within the RCMP group (81.4%) compared to the other ones.

Most respondents (81.0%) were satisfied with the ease of login on the Online Portal. A proportion of 73.7% were satisfied with the ease of landing page navigation and 73.1% with the ease of self-service options. Satisfaction tends to increase with age. Indeed, those in the less than 40 age group were the least satisfied age group regarding the ease of landing page navigation and the ease of self-service options. Those 60 or older were the most satisfied with these same elements.

4.3.3 Understanding the information provided by the Online Portal (weighted data)

The information provided by the Online Portal was easy to understand for the majority. Indeed, a proportion of 59.4% gave a score of at least 8 on a 10-point scale, when evaluating how easy to understand the information was (10 meaning the information was easy to understand and 1 meaning they did not understand at all). Those 60 or older were more inclined to consider the information easy to understand (74.3%) and those under 40 years old were less inclined (36.0%).

4.3.4 Strengths and things to improve on the Online Portal (weighted data)

The most liked aspect of respondents’ interaction with the Online Portal was having the ability to estimate their pension (42.0%) followed by finding it user-friendly (33.2%). A proportion of 14.2% also mentioned enjoying the ability to access information. Responses in the “Other” category include, among other things, respondents praising the communication aspect of it, like the fact that messages can be sent from the portal..

Among responses obtained, 27.6% of respondents said they would improve the amount or quality of information provided on the Online Portal. A proportion of 17.8% requested for the online portal to be more user-friendly and about the same number of respondents (17.6%) requested improvements or reported issues with the calculator.

4.3.5 Contact Options

Accessing personal pension information from a personal laptop is also the contact option Online Portal respondents would be the most likely to use if available. Indeed, 80.0% reported they would use it by giving a score of at least 8 on a 10-point likeliness scale. The contact options they would be the least likely to use are SMS notices or reminders (46.7%).

4.3.6 Perception of the benefits available

More than two thirds (69.7%) of respondents agreed that the benefits available under their pension plan support their wellbeing. This proportion is higher for those aged 40 to 59 (73.1%). Those aged less than 40 years old tend to be less in agreement with this statement (56.6%). However, these younger respondents were also more neutral (36.9%) than those in the two other age groups.

5.0 Conclusion

Pension Centre respondents

Respondents were generally satisfied with the service they received when calling the Pension Centre (86.4%). They were especially satisfied with the courteousness (94.8%), helpfulness (89.9%) and knowledge level of the staff (88.2%).

When asked about what they like the most about the Pension Centre, finding that the agents were helpful, knowledgeable, and providing answers was the most recurrent theme about their interaction (62.6%), followed by the agents’ soft skills (49.6%).

Regarding suggestions for improvement, 24.2% of respondents suggested improving access to agents. This includes reducing wait times on the phone and expanding service hours, indicating the importance of accessibility and convenience for users. Also, 18.9% of respondents mentioned the need for more knowledgeable staff or better information. Additionally, 14.2% highlighted the importance to improve delays.

Requesting general pension information, a form or documentation were the main reasons for calling the Pension Centre (48.3%). Overall, respondents also found the information they were provided easy to understand. Indeed, 80.0% gave a score of at least 8 on a 10-point scale regarding this aspect.

The contact option Pension Centre respondents would be the most likely to use was a personal computer (78.3%). The webchat or Instant Messaging was the one they would be the least likely to use (44.1%), especially respondents aged 60 or older (32.5%).

Online Portal respondents

Nearly 70% of respondents expressed satisfaction with the Online Portal's services. Notably, satisfaction levels varied across age groups, with those under 40 being the least satisfied at 59.0%, while those 60 years old or older were the most satisfied at 76.4%. Additionally, the RCMP group showed a notably high satisfaction rate at 81.4%.

Regarding the usability of the Online Portal, a substantial majority of respondents were satisfied with the ease of login (81.0%), landing page navigation (73.7%), and self-service options (73.1%). Satisfaction tended to increase with age, with older respondents being more satisfied with these aspects.

A proportion of 59.4% of respondents gave a score of at least 8 on a 10-point scale about the Online Portal's ability to present information in an understandable manner. However, only 36.0% of those under 40 found the information easy to understand.

A desire for more or higher-quality information on the portal is highlighted with 27.6% of respondents noting this is an area they would like to see improved. Additionally, 17.8% requested improvements in user-friendliness, and a similar percentage (17.6%) reported issues or suggested improvements with the calculator.

Finally, a personal computer is the most preferred contact option, with 80.0% of respondents expressing a likelihood to use it. Conversely, SMS notices or reminders were the least favoured option, with only 46.7% expressing a likelihood to use them.

6.0 Appendix

Appendix A: Methodology

The Client Service Feedback Survey – #005 was designed by PSPC. The questionnaire was programmed by Advanis in both official languages.

The survey was administered among members of the target population who had recently called the Government of Canada Pension Centre or visited the Pension Program secure employee portal.

The target population was both retired and active pension members, more specifically:

• Public Service Superannuation Plan active employees and retired members (PSSA), including Crown Corporation active employees and retired members.
• RCMP Superannuation Plan active officers and retired members (RCMPSA).
• Canadian Forces Superannuation Plan active regular force and reservist and retired members (CFSA)

Online portal users were sent an email inviting them to participate in a web survey. Potential Pension Centre respondents were invited to participate in the web survey as well. Some were also called using a computer-assisted telephone interviewing (CATI) methodology.

Two different samples were sent by PSPC. The first contained a list of 3001 active and retired members who recently called the Government of Canada Pension Centre and included information to reach them by phone and by email. The second contained a list of 8881 active members who recently visited the Online Portal and included information to reach them by email only. Data collection started on July 6th, 2023, and ended on August 14, 2023.

From July 28 to August 9, 2023, we sent 2571 invitations for Online Portal users and 2240 invitations for pension center users (4811 in total). For potential Pension Centre respondents, up to two email reminders were sent to those who had not yet completed the survey and had not stated their refusal to participate. For potential Online Portal, an invite and a single reminder was sent.

Appendix B: Email invitations sent

• bounced: Invalid email address
• clicked: Respondent clicked on the link included in the email
• opened: Respondent read the email but did not click on the link
• sent: Valid email address but no action was taken by the recipient

Appendix C: Data collection statistics (Respondents from the Pension Centre sample)

Appendix D: Response rate

For the consultation among respondents contacted from the Pension Centre sample, the response rate was calculated by dividing the number of respondents (1,084) by the number of eligible members in the sample (2,847) for a response rate of 38.1%.

For respondents contacted from Online Portal sample, the response rate was calculated by dividing the number of respondents (620) by the number of initial email invitations who reached potential respondents (2,571). Hence the response rate for the Online portal portion of this study was 24.1%.

Appendix E: The weights

As previously mentioned, the data was weighted according to the pension group of the respondents (PSSA members, CFSA members or RCMP members). Pension Centre and Online Portal respondents were weighted separately. The weights were calculated using the population files sent by PSPC. The first list contained all members who had called the Pension Centre, and consented to participate in the survey, between May 31, 2023, and June 30, 2023. The second list contained all members who had visited the Online Portal between May 31, 2023, and June 29, 2023.

As mentioned in section 1.3, results cannot be extrapolated to another period or for the broader population, since the target population only includes respondents who had recently called the Pension Centre or visited the Online Portal during a specific period.

Appendix F: Additional notes

When surveying only a sample of a broader population, there are always risks that results suffer from a non-response bias. This happens when characteristics of those who answered the survey differ from those who did not answer. For this study, several strategies were employed to increase response rates and reduce the effects of non-response bias. This includes communicating the purpose and importance of research at the beginning of the survey as well as reassuring respondents on the confidentiality of their responses and on the legitimacy of the survey.

Appendix G: English questionnaire

Canada Pension Members Survey wave 5

CATI

CATI INTRO w RECRUIT TO WEB OPTION

CATIInt

Good afternoon/evening. I would like to speak to <<sample.name>>. [IF THEY HAVE TO GET THE PERSON WE WANT TO TALK TO, WAIT UNTIL THAT PERSON PICKS UP THE PHONE AND RE-READ THE INTRO] My name is ________ of Advanis and I am calling on behalf of Public Services and Procurement of Canada (PSPC). We are contacting members who called the Pension Center or accessed the Online Pension Portal in the last three months to learn more about your experience as a pension plan member when accessing our services. Feedback from you is essential to improve the services. The survey takes less than 10 minutes to complete and your participation is voluntary and confidential. Would you have time to complete this with me now? PLEASE VALIDATE Would you continue in English or in French? [IF ASKED ABOUT THE LEGITIMACY OF THE SURVEY]: If you would like to verify the authenticity of this survey, please visit the Government of Canada Pension Centre web page (https://www.tpsgc-pwgsc.gc.ca/remuneration-compensation/services-pension-services/pension/cn-cu-eng.html) or call the Government of Canada Pension Centre at 1-800-561-7930. [IF ASKED ABOUT PRIVACY] :Your answers will remain anonymous and the information you provide will be administered according to the requirements of the Privacy Act, the Access to Information Act, and any other pertinent legislation. [IF ASKED FOR AN ALTERNATIVE MEANS OF COMPLETING THE SURVEY] :If you experience any difficulties with the survey, or for an alternative means of accessing the survey, please contact Advanis by telephone at 1-866-509-6986 or by e-mail at : sday+pension001@advanis.ca

• ○ (1) Yes, continue on the phone
• ○ (4) Send SMS/text message to current phone number
• ○ (5) Send SMS/text message to a different number
• ○ (6) Send email
• ○ (3) Callback later
• ○ (2) No - Refused (did not agree to email or SMS)

Refcontact

Thank you for your time. Good-bye.

(Status code 1000)

CBcontact

Arrange callback.

(Status code 1001)

RECRUIT SEND

confirmemail

What email address would you like us to send the survey link to?

____________________

• ○ (-8) Refused

SMSReminder

Just in case we have an issue reaching you via email, could we send an invitation by text message to this number?

• ○ (1) Yes
• ○ (2) No

EndEmail

You will be receiving an email message shortly with the survey address and login details. If you do not see the email please check your spam folder. The email will come from INSERT EMAIL ADDRESS. Have a nice day.

(Status code 480)

Refuseemail

Thank you for your time.

(Status code 1000)

confirmsms

What mobile phone number would you like us to send the study link to?

____________________

• ○ (-8) Refused

Refusesms

Thank you for your time.

(Status code 1000)

EndSMS

Thank you for agreeing to participate. We will send you a text message with a link to the study. Goodbye and have a great day/evening!

(Status code 480)

Login page

Login page

wcag

Si vous préférez répondre au sondage en français, veuillez cliquer sur « Français ». Public Services and Procurement of Canada ( PSPC ) is conducting a survey to learn more about your experience as a pension plan member when accessing our services. We are contacting members who called the Pension Center or accessed the Online Pension Portal in the last three months. Feedback from you is essential to improve the services. Note: If you need to leave the survey and come back to it later, you may use the same link and the survey will resume where you left off. Your participation is voluntary and your answers will remain strictly confidential. This survey is registered with the Canadian Research Insights Council's ( CRIC ) Research Verification Service. The project verification number is: 20230628-AD511. Click here (opens in a new window) to verify the legitimacy of this survey. If you experience any difficulties with the survey, or for an alternative means of accessing the survey, please contact Advanis by telephone at 1-866-509-6986 or by e-mail at : sday+pension001@advanis.ca

• ○ (1) Assistive Survey for those with a disability (Screen reader enabled)
• ○ (2) Start Survey

LoginTCH

TEXT HERE You have been invited to participate in this [STUDY] on behalf of [CLIENT] and TellCityHall. Thank you for taking the time to have your say on issues important to all Canadians. Your input is valuable to us, the survey will take less than [LENGTH] minutes to complete. Participation is voluntary and you can withdraw at any time. If you get interrupted while doing the survey, you can click on the same link to pick up right where you left off. [CLIENT] has contracted an independent public opinion research company, Advanis , to conduct the research on [CLIENT] behalf. You've completed a TellCityHall survey for us in the past, and indicated then that you might participate in public policy studies in the future (TellCityHall is one of Advanis' data collection methods). Your personal information is treated with the highest standards of confidentiality by Advanis. Survey answers will be grouped with other participant responses, and your personal information will only be used for the purposes stated in the survey. We will not use this information for any purposes other than those consistent with market research requirements and guidelines. This survey is registered with the Canadian Research Insights Council's (CRIC) Research Verification Service. The project verification number is: XXXX-XXXX. Click here to verify the legitimacy of this survey. © 2023 Privacy Policy CRIC Pledge

record

Please be aware that this call may be recorded for quality assurance purposes and that your participation is voluntary. Your responses are confidential and will be grouped with those from other participants. This project also has been registered with the Canadian Research Insights Council.

Survey Questions

Main questions

Q1

Most recently, did you...?

• ○ (1) call the Government of Canada Pension Centre
• ○ (2) use the Employee Online Pension Portal

Q2a

CATI: For the remainder of this survey, please respond based on your call to the Pension Centre
experience with the Online Pension Portal. What was the purpose of your call to the Pension Centre? Did you call...? [IF DOES NOT REMEMBER CALLING THE PENSION CENTER]: We are surveying people who have contacted the Pension Centre in the last three (3) months.

WEB: For the remainder of this survey, please respond based on your call to the Pension Centre
experience with the Online Pension Portal. What was the purpose of your call to the Pension Centre?

CATI: (READ LIST. RECORD ALL MENTIONS)

WEB: Select all that apply

• □ (1) For pension payments / to request an estimate of your pension
• □ (2) To obtain an update of the status of a transfer payment (transfer value, pension transfer agreement)
• □ (3) To request a buyback estimate and/or apply to buy back service
• □ (4) To request general pension information, a form or documentation
• □ (5) To update your profile (name, address, banking information, etc.)
• □ (6) Because of life Events (Enrollments, marriage status, Supplementary Death Benefit)
• □ (97) Other (please specify):

Q2b

CATI: For the remainder of this survey, please respond based on your call to the Pension Centre
experience with the Online Pension Portal. What was the reason for your use of the Online Pension Portal? Was it...?

WEB: For the remainder of this survey, please respond based on your call to the Pension Centre
experience with the Online Pension Portal. What was the reason for your use of the Online Pension Portal?

CATI: (READ LIST. RECORD ALL MENTIONS)

WEB: Select all that apply

• □ (1) Get a pension estimate
• □ (2) Update personal information (address, contact details, etc.)
• □ (3) Service buyback estimator / to get a payment status
• □ (4) Survivor benefits estimator
• □ (5) Other (please specify):

Q3a

CATI: Thinking back on your recent experience when calling the Government of Canada Pension Centre, how would you rate your satisfaction related to the following? Would you say you are Very satisfied, Satisfied, Neutral, Dissatisfied or Very dissatisfied with...?

WEB: Thinking back on your recent experience when calling the Government of Canada Pension Centre, how would you rate your satisfaction related to the following?

• 1. The ease of navigating the automated phone system
• 2. The courteousness of staff
• 3. The helpfulness of staff
• 4. The knowledge level of the staff
• 5. The clarity of the information provided to you
• 6. The timeliness of the service you received

• ○ (1) Very satisfied
• ○ (2) Satisfied
• ○ (3) Neutral
• ○ (4) Dissatisfied
• ○ (5) Very dissatisfied
• ○ (-8) Not applicable

Q3b

CATI: Thinking back on your recent experience using the Online Pension Portal, how would you rate your satisfaction related to the following? Would you say you are Very satisfied, Satisfied, Neutral, Dissatisfied or Very dissatisfied with...?

WEB: Thinking back on your recent experience using the Online Pension Portal, how would you rate your satisfaction related to the following?

• 1. Ease of login
• 2. Ease of Landing Page navigation
• 3. Ease of self-service options

• ○ (1) Very satisfied
• ○ (2) Satisfied
• ○ (3) Neutral
• ○ (4) Dissatisfied
• ○ (5) Very dissatisfied
• ○ (-8) Not applicable

Q4

CATI: On a scale of one to 10, where 1 means Did not understand at all and 10 means Easy to understand , was the information you were provided, easy to understand?

WEB: On a scale of one to 10, was the information you were provided, easy to understand?

CATI: IF NECESSARY: The information being referred to would be from the pension expert on the phone
the information found on the portal.

WEB: The information being referred to would be from the pension expert on the phone
the information found on the portal.

• ○ (1) 1 - Did not understand at all
• ○ (2) 2
• ○ (3) 3
• ○ (4) 4
• ○ (5) 5
• ○ (6) 6
• ○ (7) 7
• ○ (8) 8
• ○ (9) 9
• ○ (10) 10 - Easy to understand
• ○ (-8) Not applicable

Q5

CATI: How would you rate your experience with the service you received? Were you...?

WEB: How would you rate your experience with the service you received?

• ○ (5) Very satisfied
• ○ (4) Satisfied
• ○ (3) Neutral
• ○ (2) Dissatisfied
• ○ (1) Very dissatisfied

Q6

CATI: On a scale of 1 to 10, where 1 means Not likely at all and 10 means Very likely , how likely would it be for you to use the following contact options if they were available:

WEB: On a scale of 1 to 10, how likely would it be for you to use the following contact options if they were available:

• 1. Accessing personal pension information from a home, personal laptop or desktop computer
• 2. Text (SMS) notices/reminders
• 3. Webchat/IM (Instant Messaging) with Pension Centre
• 4. Virtual meeting (Audio/Video call) with Pension Centre
• 5. Mobile application

• ○ (10) 10 - Very likely
• ○ (9) 9
• ○ (8) 8
• ○ (7) 7
• ○ (6) 6
• ○ (5) 5
• ○ (4) 4
• ○ (3) 3
• ○ (2) 2
• ○ (1) 1 - Not likely at all

Q10

CATI: To what extent do you agree with the following? The benefits available to me under my pension plan support my financial wellbeing. Do you strongly agree, somewhat agree, neither agree nor disagree, somewhat disagree or strongly disagree?

WEB: To what extent do you agree with the following? The benefits available to me under my pension plan support my financial wellbeing.

• ○ (1) Strongly agree
• ○ (2) Somewhat agree
• ○ (3) Neutral
• ○ (4) Somewhat disagree
• ○ (5) Strongly disagree

Demographics

Demographics

D1

What is your current status at the Government of Canada Pension Centre?

• ○ (1) Active (currently employed)
• ○ (2) Non-Active (retired)
• ○ (3) Survivor
• ○ (4) Other (please specify):

D2

What age group do you fall under?

• ○ (1) 19 and under
• ○ (2) 20 - 29
• ○ (3) 30 - 39
• ○ (4) 40 - 49
• ○ (5) 50 - 59
• ○ (6) 60 - 69
• ○ (7) 70 or older
• ○ (-8) I prefer not to answer

D3

Do you belong to any of the following groups?

CATI: [READ LIST. RECORD ALL MENTIONS]

WEB: Select all that apply

• □ (2) Indigenous peoples
• □ (3) LGBTQ2+
• □ (4) Persons with disabilities
• □ (5) Members of a visible minority
• □ (1) No, I do not belong to any of the following groups <<catiDoNotRead>>
• □ (7) Prefer not to answer <<catiDoNotRead>>

D4

CATI: Please identify your gender. Are you...?

WEB: Please identify your gender:

• ○ (1) Male
• ○ (2) Female
• ○ (3) Non-Binary
• ○ (4) Other (please specify):
• ○ (-8) I prefer not to answer

Q7a

Overall, what is the one thing you liked most about your call with the Government of Canada Pension Centre?

____________________

• ○ (-8) I prefer not to answer

Q7b

Overall, what is the one thing you liked most about your use of the Online Pension Portal?

____________________

• ○ (-8) I prefer not to answer

Q8a

Overall, what is the one thing you believe could be improved on most to make your call with the Government of Canada Pension Centre a more positive experience?

____________________

• ○ (-8) I prefer not to answer

Q8b

Overall, what is the one thing you believe could be improved on most to make your use of the Online Pension Portal a more positive experience?

____________________

• ○ (-8) I prefer not to answer

Survey quality

P1

We would now like your opinion on this survey itself. Overall, how easy were the questions in the survey to understand? Please use a scale from 0 to 10 where 0 means very difficult, and 10 means very easy.

• ○ (0) 0 - Very difficult
• ○ (1) 1
• ○ (2) 2
• ○ (3) 3
• ○ (4) 4
• ○ (5) 5
• ○ (6) 6
• ○ (7) 7
• ○ (8) 8
• ○ (9) 9
• ○ (10) 10 - Very easy

P2

Please explain your rating. For example, what specific words or questions did you find difficult to understand?

____________________

• ○ (-8) I prefer not to answer

End of the survey

ENDWEB

This is the end of the survey. On behalf of Public Services and Procurement of Canada, we would like to thank you for your participation. Public Services and Procurement of Canada has contracted an independent public opinion research company, Advanis (opens in a new window) , to conduct the research on its behalf. © 2023 Privacy Policy CRIC Pledge

(Status code -1)

CATIEnd

We have asked you all of our questions. We hope you've found this interesting, and we sincerely thank you for your time.

(Status code -1)

Survey End

End_main

ENDCATI

This is the end of the survey. On behalf of Public Services and Procurement of Canada, we would like to thank you for your participation. Public Services and Procurement of Canada has contracted an independent public opinion research company, Advanis, to conduct the research on its behalf.

(Status code -1)

WebEndTCH

We have asked you all of our questions. We hope you've found this interesting, and we sincerely thank you for your time. [CLIENT] has contracted an independent public opinion research company, Advanis , to conduct the research on [CLIENT] behalf. TellCityHall is one of Advanis' data collection methods. If you'd like to see results from other studies conducted by Advanis via TellCityHall, please visit tellcityhall.ca/surveys . © 2022 Privacy Policy CRIC Pledge

(Status code -1)

Appendix H: French questionnaire