Real-Time Client Experience Data Collection Pilot: Data from Two Collection Mechanisms

1. Executive Summary

1.1. Research Purpose and Objectives

The Citizen Service Branch (CSB) has developed a Client Experience (CX) Survey Measurement Model to understand how clients perceive their experience with Service Canada. This model draws on a Quebec Government model, the Common Measurements Tool from the Institute for Citizen-Centered Service (ICCS), and elements of Forrester’s Client Experience Measurement approach. The draft model has been developed in consultation with stakeholders within Service Canada and was piloted through the Employment Insurance Service Quality Review Survey before being finalized following qualitative research conducted with clients.  The model is now being used as the backbone for CX work across Employment and Social Development Canada (ESDC). It was used to design ESDC’s Annual CX Survey and is also being used to facilitate the design of the questionnaire for the regular collection of data by channel that will be piloted in the fourth quarter of Fiscal Year 2017-18.

In keeping with the Treasury Board Policy on Service, and the ESDC Service Strategy, which call for collecting client feedback to support client-centric service management, and with the Client-Centric Feedback solution within the Service Transformation Plan, CSB is leading a pilot project to generate client experience data that is not currently available for corporate performance monitoring and reporting. Regularly reported client experience data for each channel will complement the annual data collected by the CX Survey. 

The annual CX survey examines the service experience delivered by all channels interacting together for results tracking, while collecting data by channel will provide a pulse check on how well each channel is working for clients for course correction during the year.

CSB is implementing a pilot project to determine the most appropriate mechanism to collect data from clients on their service experience in each channel on an ongoing basis.  In proceeding with the pilot, CSB is following through on the third of four client feedback priority activities that were approved by the Service Management Committee in 2016; the regular collection of client experience data that is aligned and automated across service delivery channels.

The pilot project is an initial step to automating and aligning data collection from clients in each channel to allow the Department to monitor the client experience for service management purposes, beginning in 2018/19.

The results of the pilot project will position ESDC to determine which automated mechanisms for collecting ongoing client experience data in the phone and in-person channels can generate data that is sufficiently representative and cost-effective for Service Canada to use in monitoring the performance of its service delivery system. The portion of the pilot addressed in this report includes:

1) generating the control data for the in-person channel by administering intercept surveys to clients as they leave offices after a service interaction; 2) transferring phone channel clients to a set of client experience questions administered through an interactive voice response (IVR) system after a service interaction.

Service Canada will assess the data collected through the set of mechanisms it is testing in-house in the in-person channel against the data collected by Kantar TNS from the same offices to determine the coverage and reliability of results generated by the in-house mechanisms.  It will use the data collected by Kantar TNS on the service experience delivered by the phone channels to assess the effectiveness of the IVR mechanism of collecting data.

The set of questions used to collect the client experience data conform with Service Canada's Client Experience Survey Measurement Model, and gather data on the ease and effectiveness of, and satisfaction with the service experience, and on the program and service task involved in the service experience.

The specific research objectives are to:

• collect data on the service experience of clients who have completed a service interaction in selected in-person offices, including:
  • Service Canada Centres (SCC)
  • Passport Offices
• collect data on the service experience of clients who have completed a service interaction with the phone channel, consisting of:
  • 1 800 O-Canada
  • Employment Insurance and Pensions Specialized Call Centres

1.2. Summary of Findings

The following results describe the period of March 19th to 29th, 2018 in selected offices, and from April 10th to May 4th, 2018 in the phone channels.  They will be used by Service Canada to assess the response rate and coverage or comparability of results of a range of automated client experience data collection mechanisms tested in each channel.

In-Person Channel

The results of the in-person analyses found no significant differences between the findings for Passport Offices and Service Canada Centres except for education level, where it was found that those who accessed a Service Canada Centre were more likely to have only completed high school or less and those who accesses a Passport Office were more likely to have completed university or above.  For this reason, in-person intercept findings are discussed at a total level only and not broken out by Passport Office and Service Canada Centre. 

In almost all instances, staff were regarded as helpful and respectful, contributing to most respondents finding their task easy to accomplish.  Most respondents were equipped with the information they needed to understand their next step in the process with only a small minority leaving without understanding what they needed to do next.  With such high levels of success in achieving what visitors set out to do, it is not surprising that 9 out of 10 respondents were satisfied with their visit to the Passport Office or SCC.

Four-in-five people visiting the Passport Office or SCC wanted to receive service in English and the majority agreed that services were provided in the official language of their choice. 

Phone Channel

The IVR results of the analyses found no significant differences between the findings for the different services accessed except for ease of service, where 1 800 O-Canada was perceived as being easier to use.  On the flip side, compared with the two other services assessed, respondents accessing 1 800 O-Canada were less likely to accomplish the task they set out to do.  Due to the lack of significant differences between the services, IVR findings will be discussed at a total level only and not broken out by services accessed.

The main reason for calling 1 800 O-Canada was to obtain general information about a program or service.  Calls were made to the EI line mainly to obtain an application status or to get assistance with a two-week report. There were multiple main reasons reported for accessing the pension service, including: obtaining general information about benefits, application status and changing an address or direct deposit. 

As was the case with the in-person findings, in almost all instances, staff were considered helpful and respectful and most respondents found their task easy to accomplish.  While most respondents accessing services via telephone could accomplish the task they set out to do, those accessing 1 800 O-Canada were less likely to do so than those who used the other two services offered. 

More than 9-in-10 respondents were given the information they needed to understand their next step in the process, again resulting in very high satisfaction. 

1.3. Methodology

Two methods of data collection were used.  In-person intercept surveys were conducted among 1421 Canadians who made use of a Service Canada Centre or Passport Canada Office.  In addition to this, 2741 IVR phone surveys were conducted from referrals from both EI and Pension specialized call centres and 1 800 O‑Canada call centres. 

For the in-person intercept survey, a pre-test consisting of 77 interviews was completed on March 15 and regular fieldwork took place on 19-29 March 2018. No changes were made to the survey after the pre-test and these results were included in the final dataset.

For the IVR phone surveys, a pre-test consisting of 24 English interviews and 10 French interviews was completed on April 9^th, 2018 and fieldwork took place between April 10 and May 4, 2018.  No changes were made to the pre-test survey and the results were included in the final dataset.

As this was a non-probability sample, margin of error does not apply and conclusions from these results cannot be generalized to any population. Surveying was conducted in the respondent’s official language of choice and took an average of 3.15 minutes for the in-person intercept and 3.52 for the IVR to complete.  A detailed methodology can be found in Chapter 4.

1.4. Contract Value

The total contract value for this project was $121,038.14 including HST.

1.5. Statement of Political Neutrality

I hereby certify as Senior Research Director & Public-Sector Practice Lead of Kantar TNS that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Policy on Communications and Federal Identity and the Directive on the Management of Communications. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences and standings with the electorate or ratings of the performance of a political party or its leaders.

Tanya Whitehead
Kantar TNS
Senior Director & Public-Sector Practice Lead

2. Foreword

2.1. Background and Objectives

The Citizen Service Branch (CSB) has developed a Client Experience (CX) Survey Measurement Model to guide the collection of data about how clients perceive their experiences with Service Canada. This model draws on a Quebec Government model, the Common Measurements Tool from the Institute for Citizen-Centered Service (ICCS), and elements of Forrester’s Client Experience Measurement approach. The draft model has been developed in consultation with stakeholders within Service Canada and was piloted through the Employment Insurance Service Quality Review Survey before being finalized following qualitative research conducted with clients.  The model is now being used as a framework for CX data collection across Employment and Social Development Canada (ESDC). It was used to design ESDC’s Annual CX Survey and is also being used to facilitate the design of the questionnaire for the regular collection of data by channel that was piloted in the fourth quarter of Fiscal Year 2017-18.

In keeping with the Treasury Board Policy on Service, and the ESDC Service Strategy, which calls for collecting client feedback to support client-centric service management, and with the Client-Centric Feedback solution within the Service Transformation Plan, CSB led a pilot project to generate client experience data that would complement the annual data collected by the Client Experience Survey. 

The annual CX survey examines the service experience delivered by all channels interacting together for results tracking, while collecting data by channel would provide a pulse check on how well each channel is working for clients and allow for course correction during the year.

The pilot project was an initial step to automating and aligning data collection from clients in each channel to allow the Department to monitor the client experience for service management purposes, beginning in 2018/19.

The results of the pilot project allowed ESDC to determine whether low-cost automated mechanisms in the phone and in-person channels, such as referring clients to an e-questionnaire, can generate data that is representative of the service experiences that occur within a given channel and to determine whether this data can be used for performance monitoring, or whether higher cost mechanisms, such as intercept interviews are a more appropriate choice.

Kantar TNS was hired to conduct the data collection for the in-person and IVR portion of the pilot, while ESDC was responsible for the remaining data collection, which entailed web-based surveys and the placement of tablets in SCCs. ESDC was responsible for the analysis of the overall results.

The specific research objectives included:

• collect data on the service experience of clients who have completed a service interaction in selected in-person offices, including:
  • Service Canada Centres (SCC)
  • Passport Offices
• collect data on the service experience of clients who have completed a service interaction with the phone channel, consisting of:
  • 1 800 O-Canada
  • Employment Insurance and Pensions Specialized Call Centres

2.2. Methodological Overview

A detailed methodology can be found in Chapter 3.  The following provides a brief overview of the methodology.

For the in-person intercept survey, a pre-test consisting of 77 interviews was completed on March 15 and regular fieldwork took place via full day interviews at 27 locations on 19-29 March 2018.  No changes were made to the survey after the pre-test and these results were included in the final data set.

In-person intercept surveys were conducted among 1421 Canadians who made use of an in-person Service Canada Centre or Passport Canada Office.  In addition to this, 2741 IVR phone surveys were conducted from referrals from both EI and Pension specialized call centres and 1 800 O-Canada call centres. 

For the IVR phone surveys, a pre-test consisting of 24 English interviews and 10 French interviews were completed on April 9 and regular fieldwork took place between April 10 and May 4, 2018.  No changes were made to the survey and these results were included in the final dataset.

As this was a non-probability sample, margin of error does not apply and conclusions from these results cannot be generalized to any population.  Surveying was conducted in the respondent’s official language of choice. The in-person intercept survey took an average of 3.15 minutes to complete. The IVR phone survey took an average of 3.52 minutes to complete. 

The numbers presented throughout this report are rounded to the closest full number. Due to this rounding, in some cases it may appear that ratings collapsed together are different by a percentage point from when they are presented individually and totals may not add up to 100%.

3. Methodology

3.1. Survey Design

Kantar TNS in collaboration with ESDC, designed the survey instruments.  These were developed to be as consistent as possible across all the different methodologies (i.e. in-person, IVR, tablet and online). 

The draft questionnaires were shared with ESDC for their input and the final, signed-off instruments were translated into French. 

Final questionnaires used for this research can be found in Appendix A and B of this report.

3.2. Programming

Once the questionnaires were finalized, they were programmed into the applicable systems.  This was done by experienced programmers.  After programming, the questionnaires underwent extensive testing to ensure that everything is working exactly as intended.

3.3. Pretest

The surveys went through extensive pilot testing (without a respondent) and then pre-testing in live conditions (with a respondent).

After testing each individual section of the survey, we pilot tested (without a respondent) the entire survey to ensure all aspects of the questionnaires were functioning as intended. The pilot involved ensuring the wording was exactly as intended, that the skip patterns and logic were correct and that the questions were clear and concise. Pilot testing involved reviewing the programmed questionnaires to ensure accuracy and match with the hard copy survey instrument. We also automated the process (autopilot) by applying the questionnaires to dummy records and checking the output to ensure that proper routing through the questionnaire has occurred. Once we have confirmed the verbatim and logic is correct in English, we then reviewed the French questionnaires in the same manner outlined above to ensure it is also correct. By testing the two surveys separately, we ensured quality in both languages. 

The in-person survey pre-test was undertaken on March 15th which involved a full day of interviewing at the Hamilton Passport Office and resulted in 77 completions.  The IVR pretest took place on April 9th obtaining 24 English and 10 French completes. The results were reviewed to ensure the surveys were working as expected and that the questions were being interpreted as expected. Based on the results of the pre-test, no changes were required for either surveys and as such the results of the completes were included in the final data sets.

Both surveys were registered with the MRIA Survey Registration System.

3.4 Sampling

In-person intercept surveys

For the in-person intercept surveys, a number of interviewers were placed in 24 Service Canada Centres and 4 Passport Offices across Canada (including the pre-test location). Interviews were conducted by one interviewer for seven hours at each location.  Although interviewing took place throughout the day interviewers were all conducting interviews during peak times of 11am to 1pm.  Respondents were not screened or selected based on any specific demographic criteria. As Service Canada Centre or Passport Office clients were leaving the office, they were invited to participate. Those who agreed, completed the interview on a tablet provided by the interviewer.  Once the interview was completed, the very next person leaving the office was invited to participate. Service staff also referred respondents and those who volunteered to take part, sought out the on-site interviewer and completed the survey.

Respondents were requested to self-complete the survey, however, those who preferred interviewer assistance, received it. Respondents did not receive incentives for completing the survey.

Interviewers were given a lead contact at the Service Canada Centre or Passport Office. There was also a staff member at each venue who briefed the interviewer on best location to intercept clients and any other pertinent information for the location.

Throughout the fieldwork period, the weather was monitored and the interview schedule changed to maximize anticipated traffic volumes that may have been impacted by poor weather. 

The following table provides the schedule for interviewing at each location.

Table 3.4.a: In-person location schedule

IVR survey

IVR phone surveys were conducted from referrals from Employment Insurance (EI), Pension specialized call centres and 1 800 O-Canada call centres. 

For the 1 800 O-Canada call centres, a selection of 9 agents for all hours that the call centre was open were assigned to invite clients to complete a short set of client experience questions.   If the respondent agreed to participate they were then transferred to the IVR survey.  

For the Specialized call centres, a selection of a maximum of 17 (5 for EI, 12 for pensions) agents at any given time were assigned for the duration of the pilot.  In total, 54 unique pensions agents and 35 unique employment insurance agents referred during the field period.  Again, agents invited clients to complete a short set of client experience questions and if they were willing to participate they were transferred to the IVR survey.

Requirements for Agent selection for the specialized call centre:

The following number of agents referred calls for each of the services accessed:

Table 3.4.b:  Number of agents per language group

In future, Kantar TNS recommend that all agents should refer to the survey to increase the number of completions.

3.5 Data Collection

Kantar TNS was responsible for two of the four methods of data collection: In-person interviews and IVR. ESDC was responsible for the other methods:

• For 1 800 O-Canada, online exit survey accessed via active/passive link referrals or verbal communication of URL to clients.
• For Passport Offices and Service Canada Centres, self-completes on tablets or by using Citizen Access Workstation Services (CAWS) available at in‑person locations or clients could answer the survey at home by entering a URL that was provided to them on a business card. 

In-person intercept surveys were conducted among 1421 Canadians who made use of a Service Canada Centre or Passport Canada Office. In addition to this, 2741 IVR phone surveys were conducted from referrals from Employment Insurance and Pension specialized call centres and 1 800 O-Canada call centres. 

In-person intercept surveys:

The intercept interviews were conducted using Computer Assisted Personal Interviewing (CAPI) technology. CAPI ensured the interview flowed as it should with pre-programmed skip patterns.  It also controlled responses to ensure appropriate ranges and data validity. Surveys were conducted in English or French as chosen by the respondent. All participants were informed of the general purpose of the research, they were informed of the sponsor and the supplier and that all responses would be confidential. 

Regular fieldwork for the intercept survey took place from March 19^th to 29^th, 2018 with a pretest on March 15, 2018.  In total, 2238 people were approached¹ and 1421 surveys were completed at 28 offices across Canada. This included the pilot interviews conducted on March 15^th.  The average survey length was 3.15 minutes with the shortest being 1.55 minutes and the longest being 10.75 minutes.

IVR surveys:

This part of the fieldwork was conducted using IVR technology.  This technology allows a computer to interact with humans using voice and keypad input tones.  This ensured that the interview flowed as it should with pre-programmed skip patterns where necessary. It also controlled responses to ensure appropriate ranges and data validity. 

As discussed previously, agents recruited respondents during their service call. Clients who agreed to participate were forwarded to the IVR survey where they were offered to respond to the survey in their official (English or French) language of their choice.

All participants were informed of the general purpose of the research, they were informed of the sponsor and the supplier and that all their responses would be confidential. 

The IVR surveys were conducted from April 10^h to May 4^th, 2018 and the total number of interviews included the results of the pre-test that took place on April 9^th, 2018. In total, 2741 surveys were completed.  The average survey length was 3.52 minutes with the shortest being 1.4 minutes and the longest being 9.16 minutes.

The IVR system did experience some downtime during the fieldwork period due to a system reboot and a log file filling up.  These are noted below:

• April 16^th: Downtime until 12:22pm EST
• April 25^th: Downtime from 3:45pm to 4:15pm EST
• May 2^nd: Downtime 1:30pm to 2:30 pm EST

3.6 Completed Interviews

In-person intercept interviews:

The following interviews were completed at each of the selected Service Canada Centres or Passport Canada Offices:

Table 3.6.a:  Completed interviews

IVR survey:

The following outlines the number of interviews completed by client from each of the call centres by region.

Table 3.6.b:  IVR completed interviews

3.7 Non-Response Bias

As with all samples, there is a possibility of non-response bias. For example, the in-person survey did not include persons who were not approached to complete a survey due to the interviewer being busy with another interview or those who declined to participate.  For the IVR survey, persons who were too busy or declined to participate in the survey were not included in the survey.  In addition, some groups within the population are systematically less likely to answer surveys. To address the issue of response bias, data were weighted to be representative of the defined populations.  Complete weighting details can be found in the following section.

3.8 Weighting

In-person intercept interviews:

Weighting adjustments were applied to the final edited, clean data to ensure that the data were weighted to the population.  The following table shows the interviewed sample (unweighted) versus the population.  The population consists of everyone who have visited the Service Canada Centres or Passport Offices on the days of the interviews.  The sample are those who participated in the interviews on those days. 

The table shows that the interviewed sample was representative of “Canada Pension Plan - Other” as well as “Social Insurance Number”.  The interviewed sample was over represented for “Apprenticeship Programs or Services” as we interviewed 10 respondents and the weighted population show only 1 respondent had to be from this group to represent the population. “Canada Pension Plan – Disability” (21 vs. 11), “My Service Canada Account” (49 vs. 34), “Other” (109 vs. 91) and “Passport Offices” 274 vs. 252) were also over-represented in our sample.  The sample was under represented for “Employment insurance” (314 vs. 343), “Old Age Security and Guaranteed Income Supplement” (92 vs.119) as well as “Passport (SCC Offices)” (145 vs. 159).  This is a reasonable representation of the population and weighting was applied to correct for relatively small deviations from the population.

Table 3.8.a: Sampled vs. Population data

IVR surveys:

Weighting adjustments were applied to the final edited, clean data.  For the IVR surveys, 5 different weights were applied based on the survey data being analyzed:

1. Pensions only data was weighted by region.
2. 1 800 O-Canada only data was weighted by region.
3. EI only data was weighted by region.
4. EI/Pension combined data was weighted by program within region.
5. Total data was weighted by program within region.

The following tables show the unweighted sample (interviewed sample) versus the weighted sample (population).

Table 3.8.b:  Weighted vs. unweighted data: Pensions

Table 3.8.c: Weighted vs. unweighted data: 1 800 O-Canada

Table 3.8.d: Weighted vs. unweighted data: EI

Table 3.8.e: Weighted vs. unweighted data: EI/Pension

Table 3.8.f: Weighted vs. unweighted data: Total

Tabulated Data

Detailed tables are included under separate cover.

4. Detailed Findings: In-Person Intercept

The results of the analyses found no significant differences between the findings for Passport Offices and Service Canada Centres except for education level where it was found that those who accessed a Service Canada Centre were more likely to have only completed high school or less and those who accesses a Passport Canada Office were more likely to have completed university or above.  For this reason, in-person intercept findings will be discussed at a total level only and not broken out by Passport Office and Service Canada Office.

4.1 Program Accessed & Main Reason for Visit

Table 4.1.a:  Program or service accessed or discussed

Q07: When you came to the office today, which program or service were you here to access or discuss?

The main reasons for visiting a Passport Office were to apply for a passport (34%) or to renew a passport (58%).  Some of the other reasons entailed submitting additional documentation (3%) and to get general information about passports (2%).

Respondents who visited a Service Canada Centre, mainly did so to access employment insurance or employment related services (30%) with the majority (43%) applying for employment insurance.  This was closely followed by Social Insurance Number related service (25%) of which most (72%) respondents visiting for this reason, were applying for a Social Insurance Number.  Fourteen percent of visits to a SCC were also Passport related. One in ten respondents visiting a SCC did so for a Guaranteed Income Supplement or Old Age Security reason with even fewer respondents reporting to visit a SCC for Canada Pension Plan related services (8%).

Table 4.1.b:  Main reason for visit

Q08: What was the main reason for your visit today?

4.2. Staff

In almost all instances, staff at the Passport Office or Service Canada Centre were seen as helpful (93% agree/strongly agree) and respectful (94% agree/strongly agree). 

Table 4.2.a:  Rating Staff

Q04: How much do you agree or disagree with the following statements:  Staff were Helpful; Staff were Respectful?

4.3 Service Assessment

Most respondents visiting the Passport Office or SCC found their task easy to accomplish (87% easy or very easy) however, a small minority found it difficult (5% difficult/very difficult).  Most respondents could accomplish the task they set out to (86%), another 9 per cent were partially able to do what they needed to; leaving only 5 per cent of clients unable to accomplish the task they set out to accomplish.  Furthermore, more than 9 in 10 (91%) were given the information they needed to understand their next step in the process but a small minority (6%) left not understanding what they needed to do next.

Table 4.3.a: Level of ease

Q10: Thinking about what you were trying to do at the office today, how easy or difficult was it to do?

Table 4.3.b: Task accomplished

Q11: Were you able to do what you needed to do in the [Service Canada Centre / Passport Office]?

Table 4.3.c: Information for next step obtained

Q04: How much do you agree or disagree with the following statements: Your visit today gave you the information you needed to understand what to do next?

4.4. Overall Satisfaction

With such high levels of success in achieving what visitors set out to do, it is not surprising that 9 out of 10 respondents (91%) were satisfied with their visit to the Passport Office or SCC.

Table 4.4.a: Overall satisfaction

Q07: Overall, how satisfied were you with your experience at the [Service Canada Centre / Passport Office] today?

4.5. Language

A large majority of clients that visited a Passport Office or SCC reported that they were served in their official language of choice. Overall, 93% of clients that visited one of the eighteen selected in-person locations that offer bilingual services (4 Passport Offices, 14 SCCs) agreed/strongly agreed that services were provided in the official language of their choice. Results suggest, however, that clients requesting to be served in French were less likely to report having received service in their language of choice (89%) compared to those that wanted service in English (94%).

Table 4.5.a: Service in language of choice in bilingual offices

Q05:  I was provided with service in the official language of choice, that is English or French.

4.6. Respondent Profile

Fifty-nine per cent of respondents of the in-person survey fell into at least one of the target categories. 

More specifically:

• 26% were aged 18-30
• 26% were older than 60 years
• 5% live in rural or remote areas
• 2% are Indigenous
• 2% have problems with mobility
• 2% have difficulty with their eyesight
• 1% have hearing difficulties. 

More than half (63%) of respondents have post-secondary education.

Table 4.6.a: Demographic statements

Q15: Which of the following, if any, apply to you?

Table 4.6.b: Education

Q14: What is the highest level of formal education you have completed

5. Detailed Findings – IVR Interviews

The results of the analyses found no significant differences between the findings for the different services accessed except for ease of service, with 1 800 O‑Canada being perceived as easier to use than the EI or Pension specialized call centres.  However, respondents accessing 1 800 O‑Canada were less likely to accomplish the task they set out to do than those accessing the two other services.  Due to no further significant differences between the services, IVR findings will be discussed at a total level only and not broken out by services accessed. 

5.1. Program Accessed & Main Reason for Visit

Of those who accessed 1 800 O-Canada, the majority (72%) did so as a first step to handling their request. One in four respondents have taken other steps before accessing 1 800 O Canada.

Table 5.1.a: 1 800 O Canada First step

Q8. Thinking about the reason you contacted 1 800 O-Canada, was 1 800 O-Canada your first step to handle your request today? 

The main reason for calling 1 800 O‑Canada was to obtain general information about a program or service (42%).  One-in-ten respondents who called this number did so to learn how to apply for a program, followed closely by a follow-up regarding an application status (6%). 

Calls to the EI line were mainly to obtain an application status (26%) or to get assistance with a two-week report (25%).  Getting information about a payment (14%) or gathering general benefit information (10%) were also reasons cited for calling the EI line. 

The main reasons cited for phoning the pension line were more fragmented, and included general information about benefits (19%), application status (18%) and address or direct deposit change (15%).  Obtaining payment information (9%) and an explanation of correspondence (8%) were also cited. 

Table 5.1.b: Main reason for visit

Q04: What was the main reason for your call today?

5.2. Staff

In almost all instances, phone staff were seen as helpful (91% agree/strongly agree at a total level).

Table 5.2.a: Helpful Staff

Q01: How much do you agree or disagree that the staff were helpful?

Significantly more urban than rural respondents found the staff helpful (94% vs. 87% agree/strongly agree).  This was also the case for those who classified themselves as comfortable in English or French in comparison to those who were uncomfortable in either official language (94% vs. 82% agree/strongly agree).  Compared to non-indigenous respondents, Indigenous respondents were more inclined to find the staff relatively less helpful (93% vs. 85% agree/strongly agree).  

Respondents in Ontario were more likely to rate the staff as helpful in comparison with other regions (96% vs. 86-90% agree/strongly agree). Similarly, college or university educated respondents were more likely to rate the staff as helpful than were respondents who had only completed high school or less (95% vs. 88% agree/strongly agree).

Staff were considered respectful (94% agree/strongly agree at a total level) in almost all instances.

Table 5.2.b:  Respectful Staff

Q01a: How much do you agree or disagree that the staff were respectful?

Those uncomfortable with the official languages were less likely than those who are comfortable with the official languages to have felt the staff were respectful (88 vs. 95% agree/strongly agree).  Higher educated respondents also were more inclined to find the staff respectful (96% vs. 92% agree/strongly agree).

5.3. Service Assessment

Most respondents found their task easy to accomplish (77% easy/very easy). However, one-in-ten found it difficult (10% difficult/very difficult at a total level).  Respondents who were comfortable with the official languages found it easier (79% easy/very easy) than those who are uncomfortable (69% easy/very easy).  Respondents accessing 1 800 O‑Canada found their task easier to accomplish than those who accessed a specialized call centre (86% vs. 75-80% easy/very easy).

Most respondents accessing services via telephone could accomplish the required task (87%), while another 8 percent were able to partially accomplish their task; leaving only 3 percent of clients unable to accomplish the task they set out to accomplish.  Respondents accessing 1 800 O‑Canada service were less likely to accomplish the task they set out to do than those accessing the specialized call centres (66% vs. 86-88%). Some of those who accessed 1 800 O‑Canada accomplished their task either partially (20%) or not at all (8%), indicating that accomplishing the task is not necessarily tied to the ease of the task. 

Furthermore, 9-in-10 (91% strongly agree/agree) were given the information they needed to understand their next step in the process but a small minority (5% strongly agree/agree) did not understand what they needed to do next.  Respondents in urban areas were more likely to respond that they were given the information they needed in comparison with those who live in rural areas (93% vs 89% strongly agree/agree).  This is also true for those living in Ontario compared to other regions (96% vs 86-92% strongly agree/agree).

Table 5.3.a: Level of ease

Q05: Thinking about what you were trying to do on the phone today, how easy or difficult was it to do?

Table 5.3.b: Task accomplished

Q06: Were you able to do what you needed to do on your call?

Table 5.3.c: Information for next step obtained

Q01b: How much do you agree or disagree with the following statements: Your call today gave you the information you needed to understand what to do next?

5.4. Overall Satisfaction

With such high levels of success in achieving what callers set out to do, it is not surprising that 9-out-of-10 respondents were satisfied (89% very satisfied or satisfied) with their use of these services.  As might be expected given the results of the service assessment, this is more readily evident for those in urban areas (92% vs. 84% very satisfied or satisfied) and those who are comfortable with the official languages (90% vs. 82% very satisfied or satisfied). 

Table 5.4.a: Overall satisfaction

Q07: Overall, how satisfied were you with your experience on the phone today?

5.5. Respondent Profile

Eighty-six per cent of respondents of the IVR survey fell into at least one of the target categories. 

More specifically:

• 47% were older than 60 years
• 28% lived in a rural or remote area
• 20% were aged 18-30
• 19% had difficulty moving around
• 14% had hearing difficulties
• 13% were uncomfortable with English or French
• 10% were Indigenous
• 10% had difficulty seeing

Half (50%) of the respondents had a post-secondary education.

Table 5.5.a: Demographic statements

Q10: Which of the following, if any, apply to you?

Table 5.5.b: Education

Q9: What is the highest level of formal education you have completed

6. Appendix A: In-Person Survey Instrument

6.1. English Survey

Q003 - INTRO_DISP: INTRO DISPLAY Single coded

Not back

Hi, I’m [Name] with Kantar TNS, we are conducting a survey on behalf of the Government of Canada on your experience today with [Service Canada Centre/Passport Office]. Would you be interested in participating in our survey? It should take no more than 5 minutes of your time. Your participation in this survey is voluntary and your responses are anonymous.”

[IF NECESSARY] This survey is registered with the Marketing Research and Intelligence Association and can be verified at surveyverification.ca (MRIA# 20180226-146X).

Normal

• 1. Yes
• 2. No
• GO TO SCREEN OUT

Scripter notes: Insert [Service Canada Centre / Passport Office] based on location.

Q004 - Q1: Level of agreement Matrix

Not back | Number of rows: 3 | Number of columns: 7

How much do you agree or disagree with the following statements?

Rows: Random | Columns: Normal

Rendered as Dynamic Grid

• Staff were helpful
  • Strongly disagree
  • Disagree
  • Neither agree nor disagree
  • Agree
  • Strongly agree
  • Don’t know
  • Not Applicable
• Staff were respectful
  • Strongly disagree
  • Disagree
  • Neither agree nor disagree
  • Agree
  • Strongly agree
  • Don’t know
  • Not Applicable
• Your visit today gave you the information you needed to understand what to do next
  • Strongly disagree
  • Disagree
  • Neither agree nor disagree
  • Agree
  • Strongly agree
  • Don’t know
  • Not Applicable

Q005 - Q2: SERVICE LANGUAGE Single coded

Not back

I was provided with service in the official language of my choice, that is English or French.

Normal

1. Strongly disagree
2. Disagree
3. Neither agree nor disagree
4. Agree
5. Strongly agree
6. Don’t know

Q006 - Q3: CHOICE LANGUAGE Single coded

Not back

What was your official language of choice?

Normal

1. English
2. French
3. No preference
4. Don’t know

Q007 - Q4: PROGRAM DISCUSSED Single coded

Not back

When you came to the office today, which program or service were you here to access or discuss?

1. Social Insurance Number
2. Employment Insurance or employment related services
3. Apprenticeship Programs or Service
4. Canada Pension Plan - Disability
5. Canada Pension Plan
6. Old Age Security or Guaranteed Income Supplement
7. Services for Veterans
8. My Service Canada Account
9. Passport
10. Other

Scripter notes: PROGRAMMING INSTRUCTION:  AUTO CODE PASSPORT FOR 5,11,21,28 @Q001_LOC  and skip question

Q008 - MAIN_REASON: MAIN REASON Single coded

Not back

What was the main reason for your visit today?

1. Social Insurance Number [HEADER DO NOT ADD]
2. Apply for a Social Insurance Number
3. Get general information about the Social Insurance Number Program
4. Provide additional information related to my application
5. Other
6. Employment Insurance or employment related services [HEADER DO NOT ADD]
7. Apply for Employment Insurance
8. Change my name/address or direct deposit information
9. Get general information
10. Get the status of my application
11. Get general information about Record of Employment on the Web (ROE Web)
12. Get general information about training/going to school
13. Get general information about Job Bank
14. Provide additional information related to my application
15. Other
16. Apprenticeship Programs or Service [HEADER DO NOT ADD]
17. Apply for a grant or a loan
18. Change my name/address or direct deposit information
19. Get general information about a grant or a loan
20. Get the status of my application
21. Provide additional information related to my application
22. Other
23. Canada Pension Plan Disability [HEADER DO NOT ADD]
24. Apply for Canada Pension Plan - Disability
25. Change my name/address or direct deposit information
26. Get general information about Canada Pension Plan - Disability
27. Get the status of my application
28. Get a certified true copy of a document or get a statutory declaration
29. Provide additional information related to my application
30. Other
31. Canada Pension Plan [HEADER DO NOT ADD]
32. Apply for a pension
33. Change my name/address or direct deposit information
34. Get general information about a pension
35. Get the status of my application
36. Request credit splitting or pension sharing
37. Apply for a death benefit
38. Get a certified true copy of a document or get a statutory declaration
39. Provide additional information related to my application
40. Other
41. Old Age Security or Guaranteed Income Supplement [HEADER DO NOT ADD]
42. Apply for Old Age Security or Guaranteed Income Supplement
43. Change my name/address or direct deposit information
44. Get general information about a pension
45. Get the status of my application
46. Get a certified true copy of a document or get a statutory declaration
47. Provide additional information related to my application
48. Other
49. Services for Veterans [HEADER DO NOT ADD]
50. Apply for a program or service
51. Get general information about programs or services for Veterans
52. Provide additional information related to my application
53. Other
54. OTHER [HEADER DO NOT ADD]
55. Apply for a program or service
56. Change my name/address or direct deposit information
57. Follow-up on my application
58. General information about a program or service
59. Get the status for a program or service
60. Provide additional information related to my application
61. Other
62. My Service Canada Account[HEADER DO NOT ADD]
63. Get my “My Service Canada Account” personal access code
64. Other
65. Passport[HEADER DO NOT ADD]
66. Apply for a Passport
67. Renew a Passport
68. Submit documentation for a previously submitted passport application (i.e. new pictures, other information)
69. Get general information about Passports
70. Other

Scripter notes: PROGRAMMING INSTRUCTION: ONLY SHOW THE ANSWER LIST THAT GOES WITH THE SELECTION AT Q4 – WHICH PROGRAM OR SERVICE

Q009 - DISPL2: Display 2 Text

Not back

Now, thinking about what you were trying to do at the office today.

Q010 - Q5: Level of Ease Single coded

Not back

How easy or difficult was it to do?

Normal

1. Very difficult
2. Difficult
3. Neither easy nor difficult
4. Easy
5. Very easy
6. Don’t know

Q011 - Q6: TASK ACCOMPLISHED Single coded

Not back

Were you able to do what you needed to do in the [Service Canada Centre / Passport Office]?

Normal

1. Yes
2. No
3. Partially
4. Don’t know

Scripter notes: PROGRAMMING INSTRUCTION: INSERT PASSPORT OFFICE FOR 5,11,21,28 @Q001_LOC ALL OTHERS REACH SERVICE CANADA CENTRE

Q012 - Q7: OVERALL SATISFACTION Single coded

Not back

Overall, how satisfied were you with your experience at the [Service Canada Centre / Passport Office] today?

Normal

1. Very dissatisfied
2. Dissatisfied
3. Neither satisfied nor dissatisfied
4. Satisfied
5. Very satisfied
6. Don’t know

Scripter notes: Insert [Service Canada Centre / Passport Office] based on location

Q013 - DEMO_DSPL: DEMO DISPLAY Text

Not back

The last few questions are for statistical purposes only. All responses will remain confidential.

Q014 - Q9: EDUCATION Single coded

Not back

What is the highest level of formal education you have completed?

Normal

1. Less than High school
2. High School diploma or equivalent
3. CEGEP or College
4. University or above
5. Prefer not to say

Q015 - Q10: DEMO STATEMENTS Multi coded

Not back | Min = 1

Which of the following, if any, apply to you.

Select all that apply

Normal

1. You are aged 18-30 (programming note: Selecting this age category precludes selection of the other age category)
2. You are aged 60+ (programming note: Selecting this age category precludes selection of the other age category)
3. You are Indigenous
4. You have difficulty hearing
5. You have difficulty seeing
6. You have difficulty moving around
7. You live in a rural/remote area
8. You are not comfortable in English or French
9. None of the above *Position fixed *Exclusive
10. Prefer not to answer *Position fixed *Exclusive

Q016 - END_DISP: END DISPLAY Text

Not back

Thank-you for your participation. Your feedback is important to the Government of Canada and will be used to help improve service.

6.2. French Survey

Q003 - INTRO_DISP: INTRO DISPLAY Single coded

Not back

Bonjour! Je suis [Name] de Kantar TNS. Nous effectuons un sondage au nom du gouvernement du Canada sur l’expérience que vous avez vécue aujourd’hui au [Centre Service Canada / Bureau des passeports]. Seriez-vous intéressé(e) à participer à notre sondage? Cela ne devrait pas prendre plus de 5 minutes de votre temps. Votre participation à ce sondage est volontaire et vos réponses sont anonymes.

[AU BESOIN] Ce sondage est enregistré auprès de l’Association de la recherche et de l’intelligence marketing et peut être vérifié auprès de verificationsondage.ca (ARIM No 20180226-146X).

Normal

• 1. Oui
• 2. Non
• GO TO SCREEN OUT

Scripter notes: Insérez [Centre Service Canada / Bureau des passeports] selon l’endroit.

Q004 - Q1: Level of agreement Matrix

Not back | Number of rows: 3 | Number of columns: 7

Dans quelle mesure êtes-vous d’accord ou en désaccord avec chacun des énoncés suivants :

Rows: Random | Columns: Normal

Rendered as Dynamic Grid

• Le personnel a été utile
  • 1 Tout à fait en désaccord
  • 2 En désaccord
  • 3 Ni d’accord ni en désaccord
  • 4 D’accord
  • 5 Tout à fait en accord
  • Je ne sais pas
  • Ne s’applique pas
• Le personnel a été respectueux
  • 1 Tout à fait en désaccord
  • 2 En désaccord
  • 3 Ni d’accord ni en désaccord
  • 4 D’accord
  • 5 Tout à fait en accord
  • Je ne sais pas
  • Ne s’applique pas
• Votre visite aujourd’hui vous a permis d’obtenir l’information dont vous aviez besoin pour comprendre ce que vous deviez faire ensuite
  • 1 Tout à fait en désaccord
  • 2 En désaccord
  • 3 Ni d’accord ni en désaccord
  • 4 D’accord
  • 5 Tout à fait en accord
  • Je ne sais pas
  • Ne s’applique pas

Q005 - Q2: SERVICE LANGUAGE Single coded

Not back

On m’a offert le service dans la langue officielle de mon choix, c’est-à-dire en français ou en anglais.

Normal

1. Tout à fait en désaccord
2. En désaccord
3. Ni d’accord, ni en désaccord
4. D’accord
5. Tout à fait en accord
6. Je ne sais pas

Q006 - Q3: CHOICE LANGUAGE Single coded

Not back

Quelle était la langue officielle de votre choix?

Normal

1. Anglais
2. Français
3. Pas de préférence
4. Je ne sais pas

Q007 - Q4: PROGRAM DISCUSSED Single coded

Not back

Lorsque vous êtes venu(e) au centre/bureau aujourd’hui, quel est le programme ou service auquel vous vouliez accéder ou dont vous vouliez discuter?

Random

1. Numéro d’assurance sociale
2. Assurance-emploi ou services liés à l’emploi
3. Programmes d’apprentissage ou services connexes
4. Régime de pensions du Canada - Invalidité
5. Régime de pensions du Canada
6. Sécurité de la vieillesse ou supplément de revenu garanti
7. Services aux anciens combattants
8. Mon dossier Service Canada
9. Passeport
10. Autre

Scripter notes: PROGRAMMING INSTRUCTION: AUTO CODE PASSPORT FOR 5,11,21,28 @Q001_LOC and skip question

Q008 - MAIN_REASON: MAIN REASON Single coded

Not back

Quelle était la raison principale de votre visite aujourd’hui?

Random

1. Numéro d’assurance sociale [EN-TÊTE – NE PAS AJOUTER]
2. Présenter une demande de numéro d’assurance sociale
3. Obtenir des renseignements généraux concernant le programme du numéro d’assurance sociale
4. Fournir des informations additionnelles relatives à ma demande
5. Autre
6. Assurance-emploi ou services liés à l’emploi [EN-TÊTE – NE PAS AJOUTER]
7. Présenter une demande de prestations d’assurance-emploi
8. Modifier mon nom/adresse ou mes renseignements concernant le dépôt direct
9. Obtenir des renseignements généraux
10. Connaître l’état de ma demande
11. Obtenir des renseignements généraux concernant le Relevé d’emploi sur le Web (RE Web)
12. Obtenir des renseignements généraux sur la formation/le retour aux études
13. Obtenir des renseignements généraux sur le Guichet emplois
14. Fournir des informations additionnelles relatives à ma demande
15. Autre
16. Programmes d’apprentissage ou services connexes [EN-TÊTE – NE PAS AJOUTER]
17. Présenter une demande de bourse ou de prêt
18. Modifier mon nom/adresse ou mes renseignements concernant le dépôt direct
19. Obtenir des renseignements généraux concernant une bourse ou un prêt
20. Connaître l’état de ma demande
21. Fournir des informations additionnelles relatives à ma demande
22. Autre
23. Régime de pensions du Canada - Invalidité [EN-TÊTE – NE PAS AJOUTER]
24. Présenter une demande de prestations d’invalidité du Régime de pensions du Canada
25. Modifier mon nom/adresse ou mes renseignements concernant le dépôt direct
26. Obtenir des renseignements généraux concernant le Programme de prestations d’invalidité du Régime de pensions du Canada
27. Connaître l’état de ma demande
28. Obtenir une copie certifiée conforme d’un document ou obtenir une déclaration statutaire
29. Fournir des informations additionnelles relatives à ma demande
30. Autre
31. Régime de pensions du Canada [EN-TÊTE – NE PAS AJOUTER]
32. Présenter une demande de pension
33. Modifier mon nom/adresse ou mes renseignements concernant le dépôt direct
34. Obtenir des renseignements généraux au sujet d’une pension
35. Connaître l’état de ma demande
36. Présenter une demande de partage des crédits ou de partage des pensions
37. Présenter une demande de prestations de décès
38. Obtenir une copie certifiée conforme d’un document ou obtenir une déclaration statutaire
39. Fournir des informations additionnelles relatives à ma demande
40. Autre
41. Sécurité de la vieillesse ou Supplément de revenu garanti [EN-TÊTE – NE PAS AJOUTER]
42. Présenter une demande de pension de la Sécurité de la vieillesse ou du Supplément de revenu garanti
43. Modifier mon nom/adresse ou mes renseignements concernant le dépôt direct
44. Obtenir des renseignements généraux au sujet d’une pension
45. Connaître l’état de ma demande
46. Obtenir une copie certifiée conforme d’un document ou obtenir une déclaration statutaire
47. Fournir des informations additionnelles relatives à ma demande
48. Autre
49. Services aux anciens combattants [EN-TÊTE – NE PAS AJOUTER]
50. M’inscrire à un programme ou présenter une demande de services
51. Obtenir des renseignements généraux concernant les programmes ou les services offerts aux anciens combattants
52. Fournir des informations additionnelles relatives à ma demande
53. Autre
54. AUTRE [EN-TÊTE – NE PAS AJOUTER]
55. M’inscrire à un programme ou présenter une demande de services
56. Modifier mon nom/adresse ou mes renseignements concernant le dépôt direct
57. Faire un suivi de ma demande
58. Obtenir des renseignements généraux concernant un programme ou un service
59. Connaître l’état d’un programme ou d’un service
60. Fournir des informations additionnelles relatives à une demande
61. Autre
62. Mon dossier Service Canada [EN-TÊTE – NE PAS AJOUTER]
63. Obtenir mon code d’accès personnel pour m’inscrire à « Mon dossier Service Canada » 
64. Autre
65. Passeport [EN-TÊTE – NE PAS AJOUTER]
66. Présenter une demande de passeport
67. Renouveler un passeport
68. Soumettre des documents pour une demande de passeport préalablement soumise (c.-à-d. nouvelles photos, autres informations)
69. Obtenir des renseignements généraux concernant les passeports
70. Autre

Scripter notes: PROGRAMMING INSTRUCTION: ONLY SHOW THE ANSWER LIST THAT GOES WITH THE SELECTION AT Q4 – QUEL PROGRAMME OU SERVICE

Q009 - DISPL2: Display 2 Text

Not back

Maintenant, en pensant à ce que vous essayiez de faire au bureau aujourd’hui.

Q010 - Q5: Level of Ease Single coded

Not back

Dans quelle mesure avez-vous trouvé l’exercice facile ou difficile?

Normal

1. Très difficile
2. Difficile
3. Ni facile, ni difficile
4. Facile
5. Très facile
6. Je ne sais pas

Q011 - Q6: TASK ACCOMPLISHED Single coded

Not back

Avez-vous été en mesure de faire ce que vous deviez faire lors de votre visite?

Normal

1. Oui
2. Non
3. En partie
4. Je ne sais pas

Scripter notes: PROGRAMMING INSTRUCTION: INSERT PASSPORT OFFICE FOR 5,11,21,28 @Q001_LOC ALL AUTRES REACH SERVICE CANADA CENTRE

Q012 - Q7: OVERALL SATISFACTION Single coded

Not back

Dans l’ensemble, dans quelle mesure avez-vous été satisfait(e) de votre expérience lors de votre visite aujourd’hui?

Normal

1. Tout à fait insatisfait(e)
2. Insatisfait(e)
3. Ni satisfait(e), ni insatisfait(e)
4. Satisfait(e)
5. Tout à fait satisfait(e)
6. Je ne sais pas

Scripter notes: Insert [Centre Service Canada /Bureau des passeports] based on location

Q013 - DEMO_DSPL: DEMO DISPLAY Text

Not back

Les dernières questions serviront uniquement à des fins de statistiques. Toutes vos réponses demeureront confidentielles

Q014 - Q9: EDUCATION Single coded

Not back

Quel est le plus haut niveau de scolarité que vous avez atteint?

Normal

1. Niveau inférieur au diplôme d'études secondaires
2. Diplôme d’études secondaires ou l’équivalent
3. Cégep ou Collège communautaire
4. Université ou plus
5. Je préfère ne pas répondre

Q015 - Q10: DEMO STATEMENTS Multi coded

Not back | Min = 1

Lequel des énoncés suivants s’applique à vous, le cas échéant.

Sélectionnez tout ce qui s’applique

Normal

1. Vous avez de 18 à 30 ans (programming note: Selecting this age category precludes selection of the Autre age category)
2. Vous avez 60 ans ou plus (programming note: Selecting this age category precludes selection of the Autre age category)
3. Vous êtes Autochtone
4. Vous avez de la difficulté à entendre
5. Vous avez de la difficulté à voir
6. Vous avez de la difficulté à vous déplacer
7. Vous demeurez dans un milieu rural ou une région éloignée
8. Vous n’êtes pas à l’aise avec le français ou l’anglais
9. Aucune de ces réponses *Position fixed *Exclusive
10. Je préfère ne pas répondre *Position fixed *Exclusive

Q016 - END_DISP: END DISPLAY Text

Not back

Nous vous remercions de votre participation. Vos commentaires sont importants pour le gouvernement du Canada et serviront à améliorer les services.

7. Appendix B: IVR Survey Instrument

7.1. English Survey

2018/04/03 UPDATE

Programming Instruction

HIDDEN VARIABLE:

RECORD INCOMING CALL CENTRE AND LANGUAGE

CODES FOR INCOMING CALL CENTRE

PEN CPP-d PENCPPD_EN / PENCPPD_FR

CPP – OTHER PENCPPOTHER_EN / PENCPPOTHER_FR

OAS/GIS PENOAS_EN / PENOAS_FR

EI EI_EN/ EI_FR

1800 OCANADA OCAN_EN / OCAN_FR

CODES FOR LANGUAGE

• ENGLISH
• FRENCH

GENERAL ERROR FOR EVERY QUESTION (if incorrect number pressed)  “You have pressed an invalid number.”

Display – 1 800 OCANADA

INTRO OCAN

Kantar TNS is conducting research on behalf of the Government of Canada on your experience today with 1 800 O CANADA.

Participation is voluntary and your responses are anonymous. This research is registered with the national survey registration system under the number: (20180226-146X) and can be verified at surveyverification.ca. It will take no more than 5 minutes to complete.

Display – PENSION

INTRO PEN

Kantar TNS is conducting research on behalf of the Government of Canada on your experience today with the Pensions Call Centre.

Participation is voluntary and your responses are anonymous. This research is registered with the national survey registration system under the number: (20180226-146X) and can be verified at surveyverification.ca. It will take no more than 5 minutes to complete.

Display – EI

INTRO EI

Kantar TNS is conducting research on behalf of the Government of Canada on your experience today with the Employment Insurance Call Centre.

Participation is voluntary and your responses are anonymous. This research is registered with the national survey registration system under the number: (20180226-146X) and can be verified at surveyverification.ca.  It will take no more than 5 minutes to complete.

Service Assessment (Not Spoken)
Randomize Q1,Q1A, Q1B

Q1 , How much do you agree or disagree that staff were helpful?  Press a number from 1 to 5 closest to your level of agreement, where 1 is strongly disagree and 5 is strongly agree, Press 9 for don’t know.

• 1 Strongly disagree
• 2
• 3
• 4
• 5 Strongly agree
• 9 Don’t know

Q1a How much do you agree or disagree that staff were respectful? Press a number from 1 to 5 closest to your level of agreement. where 1 is strongly disagree and 5 is strongly agree, Press 9 for don’t know.

• 1 Strongly disagree
• 2
• 3
• 4
• 5 Strongly agree
• 9 Don’t know

Q1b How much do you agree or disagree that your call today gave you the information you needed to understand what to do next? Press a number from 1 to 5 closest to your level of agreement. where 1 is strongly disagree and 5 is strongly agree, Press 9 for don’t know, Press 8 for not applicable.

• 1 Strongly disagree
• 2
• 3
• 4
• 5 Strongly agree
• 8 Not applicable
• 9 Don’t know

Overall Experience (Not Spoken)

PROGRAMMING INSTRUCTION:

ASK Q4 FOR 1 800 O CANADA - OCAN_EN / OCAN_FR

ASK Q4A FOR EI - EI_EN/ EI_FR

ASK Q4B FOR PENSION (Disability, CPP, OAS) - PENCPPD_EN / PENCPPD_FR OR

PENCPPOTHER_EN / PENCPPOTHER_FR OR PENOAS_EN / PENOAS_FR

Q4 – When you called today, which of the following was the main reason for your call?  Press

1. for obtain general information about a program or service
2. for learn how to apply for a program
3. for follow-up on the status of an application
4. for follow-up on a decision that was made related to a program
5. for Other

Q4A – When you called today, which of the following was the main reason for your call?  Press

1. for get the status of my application
2. for get information about a payment
3. for get general information about benefits
4. for get assistance with an EI report
5. for Other

Q4B – When you called today, which of the following was the main reason for your call?  Press

1. for get the status of my application
2. for get information about a payment
3. for get general information about benefits
4. for change my address or direct deposit information
5. for get an explanation of correspondence
6. for Other

Display

Q5 Now, thinking about what you were trying to do on the phone today.

How easy or difficult was it to do?  Please use a scale of 1 to 5 where 1 is very difficult and 5 is very easy. Press a number from 1 to 5. Press 9 for don’t know

• Very difficult
• 2
• 3
• 4
• 5 Very easy
• 9 Don’t know

Q6 Were you able to do what you needed to do on your call?  Press 1 for yes, 2 for no, 3 for partially and 9 for don’t know.

• 1 Yes
• 2 No
• 3 Partially
• 9 Don’t know

Q7 – Overall, how satisfied were you with your experience on the phone today?  Press a number from 1 to 5 closest to your level of satisfaction, where 1 is very dissatisfied and 5 is very satisfied.   Please press 9 for don’t know.

• 1 Very dissatisfied
• 2
• 3
• 4
• 5 Very satisfied
• 9 Don’t know

PROGRAMMING INSTRUCTION:

ASK Q8 for 1-800-O-Canada Only - OCAN_EN / OCAN_FR

Q8.  Thinking about the reason you contacted 1-800 O CANADA, was 1 800 O CANADA your first step to handle your request today?  That is, did you call 1 800 O CANADA first?  Press 1 for yes, 2 for no, or 9 for don’t know.

• 1 Yes
• 2 No
• 9 Don’t know

About You (Not Spoken)

Q9 The last few questions are for statistical purposes only.  All responses will remain confidential.

Please enter the number that represents the highest level of formal education you have completed? Press 1 for less than high school, press 2 for high school diploma or equivalent, press 3 for CEGEP or college, press 4 for University or above and press 9 for prefer not to say

• 1 Less than High school
• 2 High School diploma or equivalent
• 3 CEGEP or College
• 4 University or above
• 9 Prefer not to say

Q10. For each of the following items please indicate if they apply to you.  Press 1 for yes, 2 for no and 9 for prefer not to say

Q10A You are aged 18 to 30 (DO NOT READ - programming note: Selecting this age category precludes selection of the other age category)

Q10B You are aged 60 or over (DO NOT READ - programming note: Selecting this age category precludes selection of the other age category) – [IF Q10A=1, DO NOT ASK Q10B]

Q10C You are Indigenous

Q10D You have difficulty hearing

Q10F You have difficulty seeing

Q10F You have difficulty moving around

Q10G You live in a rural or remote area

Q10H You are uncomfortable in English or French

• 1 Yes
• 2 No
• 9 Prefer not to say

Q11. Which Region of Canada do you live in? Press…

• 1 for Atlantic
• 2 for Quebec
• 3 for Ontario
• 4 for West
• 5 for Territories
• 9 for Prefer not to say

PROGRAMMING INSTRUCTION:

ASK Q12A FOR WEST (Q11=4)

ASK Q12B FOR ATLANTIC (Q11=1)

ASK Q12C FOR TERRITORIES (Q11=5)

ALL ELSE GO TO THANK YOU

Q12A Which province do you live in? Press…

West (DO NOT READ):

• 1 for British Columbia
• 2 for Alberta
• 3 for Saskatchewan
• 4 for Manitoba
• 9 for Prefer not to say
• [SKIP TO THANK YOU]

Q12B Which province do you live in? Press…

• Atlantic(DO NOT READ):
• 1 for Nova Scotia
• 2 for New Brunswick
• 3 for Newfoundland or Labrador
• 4 for Prince Edward Island
• 9 for Prefer not to say
• [SKIP TO THANK YOU]

Q12C Which territory do you live in? Press…

• Territories(DO NOT READ)
• 1 for Yukon
• 2 for Northwest territories
• 3 for Nunavut
• 9 for Prefer not to say

THANK YOU. Thank-you for your participation. Your feedback is important to the Government of Canada and will be used to help improve service. Goodbye.

7.2. French Survey

2018/04/03 UPDATE

PROGRAMMING INSTRUCTION

HIDDEN VARIABLE:

RECORD INCOMING CALL CENTRE AND LANGUAGE

CODES FOR INCOMING CALL CENTRE

PEN CPP-d PENCPPD_EN / PENCPPD_FR

CPP – OTHER PENCPPOTHER_EN / PENCPPOTHER_FR

OAS/GIS PENOAS_EN / PENOAS_FR

EI EI_EN/ EI_FR

1800 OCANADA OCAN_EN / OCAN_FR

CODES FOR LANGUAGE

• ENGLISH
• FRENCH

GENERAL ERROR FOR EVERY QUESTION (if incorrect number pressed)  “Vous avez appuyé sur un numéro non valide.”

Display – 1 800 O-CANADA

INTRO OCAN

Kantar TNS effectue actuellement un sondage au nom du gouvernement du Canada au sujet de votre expérience aujourd’hui auprès de 1 800 O-CANADA afin d’évaluer la qualité des services.

Votre participation est volontaire et vos réponses sont anonymes. Cette recherche est enregistrée auprès du système national d’enregistrement des sondages sous le numéro : (20180226-146X) et peut être vérifiée auprès de verificationsondage.ca. Le sondage ne devrait pas prendre plus de 5 minutes à remplir.

Display – Pension

INTRO PEN

Kantar TNS effectue actuellement un sondage au nom du gouvernement du Canada au sujet de votre expérience aujourd’hui auprès du Centre d’appels des pensions afin d’évaluer la qualité des services.

Votre participation est volontaire et vos réponses sont anonymes. Cette recherche est enregistrée auprès du système national d’enregistrement des sondages sous le numéro : (20180226-146X) et peut être vérifiée auprès de verificationsondage.ca. Le sondage ne devrait pas prendre plus de 5 minutes à remplir.

Display – Assurance-emploi

INTRO EI

Kantar TNS effectue actuellement un sondage au nom du gouvernement du Canada au sujet de votre expérience aujourd’hui auprès du Centre d’appels de l’assurance-emploi afin d’évaluer la qualité des services.

Votre participation est volontaire et vos réponses sont anonymes.  Cette recherche est enregistrée auprès du système national d’enregistrement des sondages sous le numéro : (20180226-146X) et peut être vérifiée auprès de verificationsondage.ca. Le sondage ne devrait pas prendre plus de 5 minutes à remplir.

Évaluation du service (not spoken)
Randomize Q1,Q1A, Q1B

Q1 Dans quelle mesure êtes-vous d’accord ou en désaccord que le personnel a été utile?  Appuyez sur un chiffre de 1 à 5 pour indiquer votre niveau d’accord, où 1 signifie Tout à fait en désaccord et 5 signifie Tout à fait en accord. Veuillez appuyer sur le 9 pour « Je ne sais pas ».

• 1 Tout à fait en désaccord
• 2
• 3
• 4
• 5 Tout à fait en accord
• 9 Je ne sais pas

Q1a Dans quelle mesure êtes-vous d’accord ou en désaccord que le personnel a été respectueux? Appuyez sur un chiffre de 1 à 5 pour indiquer votre niveau d’accord, où 1 signifie Tout à fait en désaccord et 5 signifie Tout à fait en accord. Veuillez appuyer sur le 9 pour « Je ne sais pas ». 

• 1 Tout à fait en désaccord
• 2
• 3
• 4
• 5 Tout à fait en accord
• 9 Je ne sais pas

Q1b Dans quelle mesure êtes-vous d’accord ou en désaccord que votre appel aujourd’hui vous a permis d’obtenir l’information dont vous aviez besoin pour comprendre ce que vous deviez faire ensuite? Appuyez sur un chiffre de 1 à 5 pour indiquer votre niveau d’accord, où 1 signifie Tout à fait en désaccord et 5 signifie Tout à fait en accord. Veuillez appuyer sur le 9 pour « Je ne sais pas ». Appuyez sur le 8 pour « Ne s’applique pas »

• 1 Tout à fait en désaccord
• 2
• 3
• 4
• 5 Tout à fait en accord
• 8- Ne s’applique pas
• 9 Je ne sais pas

Expérience globale (not spoken)

PROGRAMMING INSTRUCTION:

ASK Q4 FOR 1 800 O CANADA - OCAN_EN / OCAN_FR

ASK Q4A FOR EI - EI_EN/ EI_FR

ASK Q4B FOR PENSION (Disability, CPP, OAS) - PENCPPD_EN / PENCPPD_FR OR

PENCPPOTHER_EN / PENCPPOTHER_FR OR PENOAS_EN / PENOAS_FR

Q4 – Lorsque vous avez appelé aujourd’hui, quelle était la raison principale de votre appel?  Appuyez sur le…

1. pour obtenir des renseignements généraux concernant un programme ou un service
2. pour apprendre comment présenter une demande pour un programme
3. pour faire un suivi quant à l’état d’une demande
4. pour faire un suivi suite à une décision rendue dans le cadre d’un programme
5. pour une autre raison

Q4A – Lorsque vous avez appelé aujourd’hui, quelle était la raison principale de votre appel? Appuyez sur le…

1. pour connaître l’état de votre demande
2. pour obtenir des renseignements généraux concernant un paiement
3. pour obtenir des renseignements généraux concernant les prestations
4. pour obtenir de l’aide au sujet d’une déclaration d’admissibilité à l’assurance-emploi
5. pour une autre raison

Q4B – Lorsque vous avez appelé aujourd’hui, quelle était la raison principale de votre appel?  Appuyez sur le…

1. pour connaître l’état de votre demande
2. pour obtenir des renseignements généraux concernant un paiement
3. pour obtenir des renseignements généraux concernant les prestations
4. pour changer votre adresse ou des informations au sujet du dépôt direct
5. pour obtenir une explication au sujet de la correspondance
6. pour une autre raison

Display

Q5  - Maintenant, veuillez penser à ce que vous essayiez de faire au téléphone aujourd’hui.

Dans quelle mesure avez-vous trouvé l’exercice facile ou difficile?  Veuillez utiliser une échelle de 1 à 5 où 1 signifie Très difficile et 5 signifie Très facile. Appuyez sur un chiffre de 1 à 5. Appuyez sur le 9 pour « Je ne sais pas ».

• 1 Très difficile
• 2
• 3
• 4
• 5 Très facile
• 9 Je ne sais pas

Q6 - Avez-vous réussi à faire ce que vous deviez faire lors de cet appel?  Appuyez sur 1 pour Oui, 2 pour Non, 3 pour En partie et sur 9 pour « Je ne sais pas ».

• 1 Oui
• 2 Non
• 3 En partie
• 9 Je ne sais pas

Q7 –Dans l’ensemble, dans quelle mesure avez-vous été satisfait(e) de votre expérience au téléphone aujourd’hui? Appuyez sur un chiffre de 1 à 5 pour indiquer votre niveau de satisfaction, où 1 signifie Tout à fait insatisfait(e) et 5 signifie Tout à fait satisfait(e).. Appuyez sur le 9 pour « Je ne sais pas ».

• 1 Tout à fait insatisfait(e)
• 2
• 3
• 4
• 5 Tout à fait satisfait(e)
• 9 Je ne sais pas

Programming instruction:

ASK Q8 for 1-800-O-Canada Only - OCAN_EN / OCAN_FR

Q8 - En pensant à la raison pour laquelle vous avez appelé le 1 800 O-CANADA, est-ce que cet appel constituait la première étape de votre démarche aujourd’hui? En d’autres mots, avez-vous d’abord appelé le 1 800 O-CANADA?  Appuyez sur 1 pour Oui, sur 2 pour Non et sur 9 pour « Je ne sais pas ».

• 1 Oui
• 2 Non
• 9 Je ne sais pas

À propos de vous (not spoken)

Q9 - Les dernières questions serviront uniquement à des fins de statistiques. Toutes vos réponses demeureront confidentielles. 

Veuillez inscrire le chiffre qui représente le plus haut niveau de scolarité que vous avez atteint. Appuyez sur 1 pour Niveau inférieur au diplôme d’études secondaires, sur 2 pour un Diplôme d’études secondaires ou l’équivalent, sur 3 pour le Cégep ou le collège communautaire, sur 4 pour l’Université ou plus et sur le 9 pour « Je préfère ne pas répondre ». 

• 1 Niveau inférieur au diplôme d’études secondaires
• 2 Diplôme d’études secondaires ou l’équivalent
• 3 CÉGEP ou Collège communautaire
• 4 Université ou plus
• 9 Je préfère ne pas répondre

Q10 - Pour chacun des énoncés suivants, veuillez indiquer s’il s’applique à vous. Appuyez sur 1 pour Oui, sur 2 pour Non et sur 9 pour « Je préfère ne pas répondre ».

Q10A Vous avez de 18 à 30 ans (DO NOT READ) programming note: Selecting this age category precludes selection of the other age category)

Q10B Vous avez 60 ans ou plus (DO NOT READ) programming note: Selecting this age category precludes selection of the other age category) – [IF Q10A=1, DO NOT ASK Q10B]

Q10C Vous êtes un(e) Autochtone

Q10D Vous avez de la difficulté à entendre

Q10E Vous avez de la difficulté à voir

Q10F Vous avez de la difficulté à vous déplacer

Q10G Vous demeurez dans un milieu rural ou une région éloignée

Q10H Vous n’êtes pas à l’aise avec le français ou l’anglais

• 1 Oui
• 2 Non
• 9 Je préfère ne pas répondre

Q11 - Dans quelle région du Canada demeurez-vous? Appuyez sur le…

• 1 pour l’Atlantique
• 2 pour le Québec
• 3 pour l’Ontario
• 4 pour l’Ouest
• 5 pour les Territoires
• 9 pour Je préfère ne pas répondre

Programming instruction:

ASK Q12A FOR WEST (Q11=4)

ASK Q12B FOR ATLANTIC (Q11=1)

ASK Q12C FOR TERRITORIES (Q11=5)

ALL ELSE GO TO THANK YOU

Q12A Dans quelle province demeurez-vous? Appuyez sur le…

Ouest :

• 1 pour la Colombie-Britannique
• 2 pour l’Alberta
• 3 pour la Saskatchewan
• 4 pour le Manitoba
• 9 pour Je préfère ne pas répondre
• [SKIP TO THANK YOU]

Q12B Dans quelle province demeurez-vous? Appuyez sur le…

Atlantique :

• 1 pour la Nouvelle-Écosse
• 2 pour le Nouveau-Brunswick
• 3 pour Terre-Neuve-et-Labrador
• 4 pour l’Île du Prince-Édouard
• 9 pour Je préfère ne pas répondre
• [SKIP TO THANK YOU]

Q12C Dans quelle territoire demeurez-vous? Appuyez sur le…

Territoires :

• 1 pour le Yukon
• 2 pour les Territoires du Nord-Ouest
• 3 pour le Nunavut
• 9 pour Je préfère ne pas répondre

MERCI. Nous vous remercions de votre participation. Vos commentaires sont importants pour le gouvernement du Canada et serviront à améliorer les services. Au revoir.

Footnote

¹ Please note that some people may not have been approached as the interviewer may have been busy with an interview when another potential respondent was leaving the building.