Summer 2018
Supplier Name: Léger
Contract Number: EP961-191339/001/CY
Contract Value: $39,550.00 (Harmonized sales tax (HST) included)
Award Date: 2018-08-23
Delivery Date: 2018-10-12
Related publications (registration number: POR 038-18)
For more information on this report, please contact the Department of Public Works and Governmnet Services Canada at:
porcoordcomm-ropcoordcomm.pwgsc@tpsgc-pwgsc.gc.ca
Ce rapport est aussi disponible en français.
This public opinion research report presents the results of focus groups conducted by Léger on behalf of the Department of Public Works and Government Services Canada. The research was conducted with public servants in September 2018.
Ce rapport est aussi disponible en français sous le titre : Évaluation qualitative de la mise à jour des concepts infopaie et suivre monCAS 2018.
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from the Department of Public Works and Government Services Canada. For more information on this report, please contact the Department of Public Works and Government Services Canada at: porcoordcomm-ropcoordcomm.pwgsc@tpsgc-pwgsc.gc.ca.
International Standard Book Number (ISBN):
978-0-660-28334-0
Related publications (registration number):
POR 038-18
Catalogue Number:
P4-84/2018F-PDF (Final Report, French)
ISBN 978-0-660-28335-7
©Her Majesty the Queen in Right of Canada, as represented by the Minister of Public Services and Procurement, 2018.
Léger is pleased to present Public Services & Procurement Canada (PSPC) with this report on findings from a series of focus groups dedicated to assessing self-service online tools dedicated to pay issues.
This report was prepared by Léger who was commissioned by PWGSC (contract number EP961-191339/001/CY awarded August 23, 2018).
PSPC commissioned Léger to conduct a qualitative assessment study of two online tools. PSPC’s teams are currently developing two online tools that will allow Government of Canada employees to obtain information on their pay file and file tracking. These two applications, "Pay Insight" and “Track MyCASE”, are still in the design stage. The intent of these online tools is to display information in using a structure and logic that will be easily understood by users, while providing employees with enhanced visibility into their pay file and possibly even contributing to the prevention and resolution of problems, resulting in an increased level of control and satisfaction.
The objective of the study was to assess the likely communications effectiveness of both tools among the intended target audience: public servants. Key elements of the study are to evaluate clarity, user-friendliness, ease of understanding, user expectations about both online solutions (prototype stage) and to identify elements and/or features that generate the most positive or negative reactions.
Copies of the moderator’s guide used in the focus group sessions can be found in Appendix.
A total of five (5) focus groups were held: three (3) in person and two (2) online. The in-person focus groups were held in two (2) different locations: Ottawa and Montréal. Two (2) separate groups were conducted in Ottawa in English and one (1) group in Montréal in French. In addition to the in-person groups, Léger conducted a series of two (2) online via the ITracks video chat platform with participants from different regions of Canada.
Groups consisted of Canadian public servants from different departments with a good mix of genders and experience working for the federal government. The reader is directed to Appendix for a more detailed methodology and for a copy of the recruitment screener.
All participants were recruited using the Government Electronic Directory Services list of employees. Public servants were separated in two (2) different groups: 1) public servant working in Departments and Agencies that are served by Pay Centre; and 2) public servant working in Departments and Agencies that are not served by Pay Centre. Public servants working in Departments and Agencies that are not served by Pay Centre did not have to evaluate the “Track MyCASE” online tool concept as part of this study.
Léger adhered to all Marketing Research and Intelligence Association (MRIA) and Government of Canada standards for conducting qualitative research. To thank them for their time and to show our appreciation for their participation, participants received a financial incentive of $150 each. The following table shows the distribution of participants in this study for each group:
| Group number | Location | Date and time | Number of participants | Group Composition | Language |
|---|---|---|---|---|---|
| 1 | Ottawa | 11 September 17:30 EST | 9 | In-person with public servant (Pay Centre Departments and Agencies) |
EN |
| 2 | Ottawa | 11 September 19:30 EST | 9 | In-person with public servant (Pay Centre Departments and Agencies) |
EN |
| 3 | Montreal | 17 September 17:30 EST | 9 | In-person with public servant (Non-Pay Centre Departments and Agencies) |
FR |
| 4 | Online | 19 September 17:30 EST | 6 | Online with public servant (Pay Centre Departments and Agencies) |
FR |
| 5 | Online | 19 September 19:30 EST | 7 | Online with public servant (Non-Pay Centre Departments and Agencies) |
FR |
It should be noted that the presentation of the screenshots of the Pay Insight and Track my Case application is happening within a broader context of fairly low confidence, sometimes even outspoken cynicism regarding pay and compensation issues at the Government of Canada. At many times, comments expressed about the actual content of the App tested tended to be about the expected outcome of whatever action the civil servant may undertake as opposed to an evaluation of the value of the content presented in the App. While the overall perspective of participants is rather positive, many of the negative comments heard tended to be about broader considerations vis-à-vis pay issues rather than the prototype applications tested.
Almost all participants visit Compensation Web Applications (CWA) just about every pay cycle. They mainly use CWA to track their pay check for accuracy or changes over time. Although they use CWA for many other Human resources (HR) and pay related information, the ‘pay stub’ remains the key driver of most visits. While they remain fairly positive about the user-friendliness of CWA, one main critique remains the “Track MyCASE” application, mainly for its perceived lack of transparency and the pace of change in the cases participants are tracking.
When presented with the initial screenshots of "Pay Insight", focus group participants tended to have similar reactions:
One interesting finding in sessions is that initial reactions to the screenshots tended to be positive, comments on each page when drilling down were more negative and that overall reactions to the App after the demo was shown tended to be positive as well. This may be due to the fact that the demo was clearer on the intended purpose of the App and its value added compared to what the actual screens of the App delivered. As well, when taking a step back after the demo, participants tended to be better at separating comments about the App itself from pay and HR issues in general.
The design of ”Track MyCASE” tended to generate positive comments about the added layers of information the App would provide. The added precision and the change is some language used compared to what they know now, were often identified as positive (i.e. In the Status section “In Review” and “Awaiting Approval”). However, information provided by the hover bubbles is seen as redundant with the status. Several comments were again tied to the language used to describe cases and their progress. Again many of the additions suggested were tied to their perceptions of the current process in resolving cases more than about the App itself (total time to resolve a case, wanting a service standards or “resolved by” date, etc.). In this regards, participants would expect the information to be more detailed in the pop-up ticket.
This report was compiled by Léger, based on the research conducted specifically for this project. The analysis presented represents what Léger believes were the most salient points during the focus group sessions.
Findings from this qualitative research (i.e. focus groups) should be considered directional only and results should not be projected as representative of all federal employees and who match the profile of the group participants. It is intended to provide deeper insights into the underlying reasons for opinions or lack thereof.
This report includes a detailed analysis of the focus group findings in section two (2). Details of the research methodology, the recruitment guide and moderator’s guide are included in the appendices.
The findings of this research will help officials of PSPC and the Government of Canada in making a decision on the self-service online tool concepts to be designed and implemented.
I hereby certify as Senior Officer of Léger that the deliverables fully comply with the Government of Canada’s political neutrality requirements outlined in the Directive on the Management of Communications. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leaders.
Christian Bourque
Executive Vice-President and Associate
Léger
Christian Bourque
Executive Vice-President and Associate
Léger
cbourque@leger360.com
514-982-2464
For more information, contact: porcoordcomm-ropcoordcomm.pwgsc@tpsgc-pwgsc.gc.ca.
Most focus group participants consider themselves fairly highly skilled when it comes to using online applications and searching for information online. The majority of them do not experience any major problems in this regard, considering themselves "quite good" when using online tools. However, some participants pointed out that the use of applications becomes more and more complicated over time.
When asked specifically about the federal government's online applications, employees have doubts about their efficiency, simplicity, user-friendliness and contemporaneity. Many employees find it difficult to access the information they want in federal applications. Even to access their own personal information, there are many steps and procedures. They also believe they are working with outdated applications. They aren’t easy to use, require lots of training and learning and the least experienced employees require the assistance of colleagues to perform certain tasks on online applications. The authentication process is also deemed too fastidious.
Employees had no problem discussing Compensation Web Applications (CWA) as they all, without exception, use them. CWA are used primarily by employees to access their pay stubs and to validate their pay information. Some, mostly those who have had problems with their pay, are using CWA bi-weekly to follow up on their file. Several mentioned that having to go through two (2) or even three (3) different applications to recover all their pay information is not user-friendly and makes the use of CWA difficult. Moreover, the information they obtain in the CWA is not easily decipherable. A few employees explained to us that the legends on the back of the pay stub have disappeared. As a result, it is now difficult for employees to understand abbreviations and acronyms and they are often unable to obtain an explanation from the pay department.
The “Track MyCASE” application, which is currently included in the CWA, received fairly negative reviews by many federal employees who participated in the groups. Although a minority said they found it useful and easy to consult, most of them consider it difficult to understand the information found in the application. They find it rather difficult to look up and track information about their case. This is especially true when a person has several cases to follow in the application. A major concern is that it is not all cases that are uploaded in the application. For some participants, the “Track MyCASE” application is unnecessary because they feel that the information is not updated regularly and that all cases are indicated as "suspended" or "in progress". Therefore, there is nothing to justify a regular visit.
During the focus group sessions, the Pay Insight application was introduced to participants by using screenshots of the application (from the home screen, then drilling down on key menu options). The screenshots presented the fictitious profile of an employee named Louis-Philippe. Since the application is still under development, no interaction was possible with a beta version or a demo of the online application. Therefore, the group moderator presented the Pay Insight concept page by page to gather comments from participants on each of the pages. The objective was to obtain their general impression about the concept, the pages of the application, their level of understanding, their expectations and possible obstacles while using the application.
At the end of the process, when participants had seen all the pages of the application and explored its content (from the screenshots), we asked them what their general opinion was of the application. The vast majority of participants had a rather good opinion about the Pay Insight concept. Most participants gave a score of 7 or 8 out of 10 for their overall assessment of the tool.
Participants’ comments tended to point to the Pay Insight as being a significant improvement of this online tool over the tools they currently have access to in the course of their work. One participant noted that this specific application brings the federal government's suite of online tools into the 21st century. Several participants have also mentioned the user-friendliness of the application, its clarity, its more modern visual aspect and its intuitive navigation. Fewer spontaneously mentioned that the prototype would allow them to access more information then they currently have. While several comments pointed to the addition of information, their overall comment on the value added of the Application focused more on navigation and look and feel more than depth of information.
Expectations run very high with regard to the application. Its modern look, its intuitive navigation, and the centralization of the employee's information in one unique application set very high expectations. However, many participants would like to see "Pay Insight" in action to be convinced of its effectiveness. For them, Pay Insight must offer real effective performance to employees, and expectations about the interactive nature or capacity to interact directly with the information found in the App were also very high. To participants, the App must answer their need for information. Many of them are reluctant to be too enthusiastic about the application. Having had many bad experiences in the past with federal government applications, they want to see the final product and hope that this application will not only be another disappointment. Although the overall assessment of Pay Insight is rather positive, the majority of participants considered that there is still room for improvement with the application. Indeed, many participants mentioned that the application lacks a lot of important information. A quick access on the main page to pay stubs, R1 and T4 documents was requested by several employees. We must remember that their main use of the CWA is to get their pay stubs and other pay documents. Also, many mentioned the need for more information to be made more understandable. Codes and abbreviations should be simplified or explained using a legend or glossary. Some suggested that if you hover your mouse over a code should open a window that includes a definition. Some participants raised the idea that there was redundant information between certain pages of the application (e.g. information from the home page, the profile and the employment page). The information should, in their opinion, be better organized to avoid repetition and facilitate navigation in the application.
Some perceptions will need to be addressed to ensure that the purpose of the application is well understood before it is deployed to the federal employees. First, only a few participants spontaneously mentioned at the outset that the idea of the application was to centralize all the pay information in a single application. This should be emphasized in the communications surrounding the application since the "one-stop shop" aspect is definitely attractive for many participants. Also, very few participants noted that this was new information that they did not have access to previously. Those who saw that appreciated the effort of transparency made by the government. Second, the application could be perceived by some employees as an attempt to reverse the burden of pay problems on them. It would be up to employees to ensure that their information is correct in the application. Pay problems would then be their responsibility. Third, it is therefore imperative that the launch of the tool be done with as few clashes as possible, to convince employees of the tool's effectiveness and necessity. The application should not be seen as a simple crisis management tool for Phoenix-related problems.
The first page presented to the participants is the Home page of the application. The comments on this page are generally very positive: the information is presented clearly and simplified. The visual appearance is also highly appreciated by federal employees: the information blocks and their colour provide a good impression. Several participants compared this page to banking or other online applications.
While the appearance and user-friendliness of this section are appreciated, there is room for improvement according to participants. Several pieces of information presented in the coloured boxes could be synthesized or combined under the same heading. Some information elements are also redundant when compared with other pages (e.g. the profile and the employment pages).
Several participants wondered whether the information boxes at the top of the page would be clickable and editable. Some others suggested that these coloured boxes should serve as a link to access the other pages/sections of the application: profile, job, benefits, etc.
The information at the bottom of the Home page about "cases" and "recent activities" was not very well understood. Information about "closed files" and "active pay files" did not seem to be understood by the vast majority of participants: "What does this mean?" "The percentage of what?" "Does this percentage only concern my own cases?" "What do the numbers mean?" Participants also expected the blue "check" buttons in these sections to be clickable and provide access to a description of the cases.
The "recent activities" section left federal employees who participated in the focus groups very confused. They wondered what the codes referred to (e.g., PCCM0426262431). This is information that was not understood by any of the participants. There is also a lack of information to fully understand what is being referred to (e.g. your Job Correction). Participants expected to be able to click on the recent activity section and get more information since, in its current form, the information presented does not allow for a good understanding.
The buttons with the "Help" pictograms on the right-hand side of the page were not quickly noticed by many participants. Some would like them to be larger and more visible among the other information on the page. In addition, the pictograms used are disappointing for many participants. They expected to have access to a discussion space with real people responsible for pay or HR either through "live chat" or direct email. These pictograms left participants with a sense of disappointment when they were introduced to the Help section. For them, in their current form, the pictograms are misleading.
According to participants, some key elements missing from the Home page were: Information on the last update date of the page information, a direct link to pay stubs and statements, a button to request a pay action and a glossary to understand codes and terms.
This page is appreciated by participants because it summarizes in one place a lot of important employee information. The participants seemed to appreciate the possibility to edit some of the information on the page. However, the pictogram used for "editing" is not known to everyone. A few participants were not familiar with the pictogram or with the distinction between the black and grey icon. Some participants suggested that the pictogram be presented in the text in the header of the page where there is mention of the "editing" icon.
In the header of the text, it is mentioned to follow the department's procedures or to contact an HR contact person. Several participants mentioned that they would prefer to have a clickable email address directly inserted in the profile page that would allow the appropriate contact person to be reached. In their opinion, the HR contact's information and the mean to contact that person must be included in the application.
Another comment about this page is that it contains a lot of information that has been seen on the Home page. Participants suggested that there should be a way to avoid duplication between these two (2) pages. The Profile page should be the place where all the information about the employee is found.
According to participants, some key elements missing from the Home page were: My phone number, work addresses; my email address, my bank details for direct payment and my contact details in case of emergency.
The Employment page was also appreciated by participants because it provides key information for employees. They particularly appreciated having access to a career history within the federal government. However, there is room for improvement. Some of them believe that it could be important for the blue buttons at the top of the page (proof of employment, completing an application form, submitting a pay action request form) to appear directly on the application's Home page. For these buttons, the participants had some difficulty differentiating between the following two buttons: completing an application form and completing a pay action request form. The difference between these two buttons should be more noticeable.
Some participants requested that the information presented in the application should not include abbreviations and only complete words. The "deduction" section should be more detailed to show all the deductions that are applied.
It has also been mentioned that this page could quickly become very long depending on the career path in government of an employee. It was suggested that it should be possible to filter or reduce the information on this page by year.
According to participants, some key elements missing from the Home page were: Access to pay stubs, T4 and R1 slips.
Many participants found it very interesting to have access to the information contained in the Benefits page. However, they have high expectations regarding the information they would like to see on the page and the level of interaction possible with the information and the insurance companies. Many asked to be able to click on the many blocks (e.g. medical, dental, etc.) in order to see the policy details. As such, they would like to see information about the amount of insurance coverage, who the beneficiaries are, buyback details, etc.
They would also like to have contact information from insurance companies. For instance, they would like a link to the company's website, an email address to contact a representative and a phone number.
Some participants pointed out the need to have a clearer vocabulary on this page. For example, on the pension plan enrolment section, some blocks of information have a "low" or "high" profile. Many participants were unaware of what this distinction was referring to.
According to participants, some key elements missing from the Home page were: Information on surrender deductible options, amount of insurance policies and amount of coverage, beneficiaries, links (web, email and phone) to the insurance company.
The Support pages are the ones that raised the most negative comments and questions. As mentioned previously, this section creates, for some people, expectations that the application is not able to meet. For instance, the pictograms suggest to participants that they will have the opportunity to interact directly and in real time with pay or HR staff directly from the application. The "Talk to us" pictogram is reminiscent of a real-time "chat". The megaphone in the "Take Action" tab is also misinterpreted by the participants. The choice of the pictograms that will be used in the real application should not be open to interpretation or create false expectations among federal employees.
The "Help me" tab in the Help section also creates its share of disappointments. In fact, it is a series of links to documentation. For the participants, it shouldn’t be called "Help" since it’s just links to documentation. For them, "Help" means to have someone from the HR or the pay center help them.
Some suggested, instead of links to documentation, to create a frequently asked questions (FAQ) section.
Regarding the "Take Action" tab, participants did not know exactly what to expect when they would click on a specific task. It was difficult for them to visualize what would happen. Some mentioned the idea to have a wizard help than could take you step by step during the process of the "Take Action".
The other information on the Support pages, such as the "balance guide" and the "job confirmation" is not immediately understood by respondents. They were not sure what would happen if they clicked on these pictograms and did not seem to understand what the titles of these pictograms mean.
Participants were pleased with the idea of an application that would allow them to follow their cases in a clear and simple way. This is particularly true in the context where the CWA cases tracking tool does not generate positive feedback from federal employees. The visual presentation of the ”Track MyCASE” application was highly appreciated by participants. It is clear, simple and easy to understand. The Case Home page is well received and does not appear to pose a critical problem of understanding among the participants. The name of the statutes is clear and easy to understand. Most participants were happy to have this information presented in this form. From the outset, they expected to be able to click on the different cases and get more information about each one.
However, some aspects of the application could be improved. The colours used to distinguish the different statuses are not considered intuitive by participants. Red is seen as a colour that informs of a problem or emergency; whereas in this case it is the colour that informs that the case is completed and closed. There is a mismatch between the message conveyed by the colour and the status name.
Regarding the information that is provided by moving the mouse over a status (hover information); there is redundancy of information that is not necessary for participants. For example, the status "pending approval" provides almost identical information in the mouse over pop-up: "your file has been processed and is simply waiting for final approval". This does not provide any additional useful information for the employee. They would like to see information about the case itself, such as information on the time remaining before final approval is given.
In fact, participants would like to obtain as much information as possible about their case: the estimated time to respond to the request, the date the case was received, the priority level of the case, the officer assigned to the case and the last time the case was updated in the application. The information contained in the pop-up of a case, could be enhanced with this information. Once a case is completed, the sheet should include information about the nature of the solution that has been provided.
Participants would also like to have an alert system on their "Pay Insight" Home page or by email that would inform them when there is movement in regards to the follow-up of their case. This alert could be on their Home page or delivered by email. In this way, employees would not have to constantly consult the page and look for any changes in their situation. They would access the application when there would be changes.
When it comes to the appraisal of the prototypes shown (PayInsight and\or TrackMyCase), there were no clear differences between French and English language groups. French-speaking employees did not express a higher or lower level of appreciation than the English-speaking employees about the two applications and the demo video. Their expectations regarding the functions of the applications are also very similar. Comments regarding the language used in current applications and the prototypes were also similar.
The more important difference was between the employees working for a department or an agency paid by the Miramichi Pay Centre and those that are not served by the Pay Centre. In general, employees serviced by the Pay Centre expressed more cynicism and frustration during the group sessions. The main object of their criticism were centered on the Phenix system and the services they have been receiving from the Miramichi Pay Centre when dealing with pay issues or cases. However, their opinion of the applications was no less positive than other employees. But, being more negative about Phenix and pay service issues, they may have expressed a higher level of expectations for these applications than other employees and voiced greater concerns over the capacity of the “system” to deliver on the promise of resolving current cases.
Overall, employees felt positive about the two applications presented to them in the focus groups: "Pay Insight" and ”Track MyCASE”. This is a significant and welcomed improvement for employees in terms of the tools and resources they currently have available to obtain pay information. That being said, the prototypes presented showed that expectations are very high in this area. Before a possible launch, applications should be improved on different levels: clearer language, no abbreviations or codes, the addition of a glossary or captions, reorganization of certain information elements, etc. Based on the comments received, these two applications could well be an interesting addition to the toolkit for federal employees.
A total of three (3) focus groups were held in two (2) different locations: Ottawa and Montréal. Two (2) separate groups were conducted in Ottawa in English and one (1) group in Montréal in French. In the two (2) locations, groups were held in specialized facilities with observation rooms. Léger ensured that all focus groups rooms include a television for showing the concepts. Léger also arranged a video-link for remote observers.
Groups consisted of Canadian public servants from different departments with a good mix of genders and experience working for the federal government. The reader is directed to Appendix for a copy of the recruitment screener.
Ten (10) participants were recruited for each group in anticipation that in the event of last minute cancellations there would be eight (8) individuals attending in each group. Each group lasted approximately 2 hours.
Léger conducted a series of two (2) online via the ITracks video chat platform with participants from different regions of Canada. The groups lasted approximately 120 minutes, and consisted of between six (6) and eight (8) participants (out of ten (10) people recruited for each group). One (1) session was conducted in English and one session was conducted in French. All recruits with an Internet connection, as well as a webcam, were eligible to participate in the group.
All participants were recruited using the Government Electronic Directory Services list of employees. Public servants were separated in two (2) different groups: 1) public servant working in Departments and Agencies that are served by Pay Centre; and 2) public servant working in Departments and Agencies that are not served by Pay Centre.
Public servants working in Departments and Agencies that are not served by Pay Centre did not have to evaluate the "Track MyCASE" online tool concept as part of this study.
Léger adhered to all MRIA and Government of Canada standards for conducting qualitative research. To thank them for their time and to show our appreciation for their participation, participants received a financial incentive of $150 each.
The following table shows the distribution of participants in this study for each group:
| Group number | Location | Date and time | Number of participants | Group Composition | Language |
|---|---|---|---|---|---|
| 1 | Ottawa | 11 September 17:30 EST | 9 | In-person with public servant (Pay Centre Departments and Agencies) |
EN |
| 2 | Ottawa | 11 September 19:30 EST | 9 | In-person with public servant (Pay Centre Departments and Agencies) |
EN |
| 3 | Montreal | 17 September 17:30 EST | 9 | In-person with public servant (Non-Pay Centre Departments and Agencies) |
FR |
| 4 | Online | 19 September 17:30 EST | 6 | Online with public servant (Pay Centre Departments and Agencies) |
FR |
| 5 | Online | 19 September 19:30 EST | 7 | Online with public servant (Non-Pay Centre Departments and Agencies) |
FR |
The following table details the profile of respondents by department or agency.
| Agency/Department | Number of participants |
|---|---|
| Agriculture and Agri-Food Canada | 4 |
| Patented Medicine Prices Review Board | 1 |
| Public Health Agency of Canada | 2 |
| Public Safety Canada | 3 |
| Canadian Dairy Commission | 2 |
| Employment and Social Development Canada | 1 |
| Canadian Food Inspection Agency | 1 |
| Environment and Climate Change Canada | 2 |
| Natural Resources Canada | 4 |
| Health Canada | 1 |
| Canada School of Public Service | 1 |
| Court Administration Service | 1 |
| Justice Canada | 1 |
| Canada Border Services Agency | 1 |
| Statistics Canada | 3 |
| Canada Revenue Agency | 2 |
| Transport Canada | 5 |
| Parks Canada | 1 |
| Atlantic Canada Opportunities Agency | 3 |
| Library and Archives Canada | 1 |
| Total | 40 |
| Group | Location | Group profile | Language | Participants | Dates | Time (Eastern time) |
Type | Location |
|---|---|---|---|---|---|---|---|---|
| GR01 | Ottawa | Pay Centre Departments and Agencies | EN | 8-10 | Sept 11, 2018 | 5:30 PM | In-person | Stratcom 100 Sparks Street Suite 802/ 8th Floor Ottawa, ON K1P 5B7 |
| GR02 | Ottawa | Pay Centre Departments and Agencies | EN | 8-10 | Sept 11, 2018 | 7:30 PM | In-person | Stratcom 100 Sparks Street Suite 802/ 8th Floor Ottawa, ON K1P 5B7 |
| GR03 | Montreal | Non-Pay Centre Departments and Agencies | FR | 8-10 | Sept 17, 2018 | 5:30 PM | In-person | Léger 700-507 Place d’Armes Montréal (QC) H2Y 2W8 |
| GR04 | Across Canada | Pay Centre Departments and Agencies | FR | 8 | Sept 19, 2018 | 5:30 PM | Online | ITracks |
| GR05 | Across Canada | Non-Pay Centre Departments and Agencies | EN | 8 | Sept 19, 2018 | 7:30 PM | Online | ITracks |
Nom du participant(e) :
Numéro de téléphone à la maison :
Numéro de cellulaire :
Adresse courriel :
Date du recrutement :
Recruteur :
Groupe no :
Confirmation - courriel envoyé le :
Hello/Bonjour (pause), the Government of Canada is conducting a research project. Would you prefer that I continue in English or French? Préférez-vous continuer en français ou en anglais?
Note: If at this point the respondent prefers to respond in French then the interviewer must be able to either proceed with the interview in French or read the following statement: "Je vous remercie. Quelqu'un vous rappellera bientôt pour mener le questionnaire avec vous en français."
Hello, I'm 
from Léger, a marketing research company. We are organizing a research project on behalf of the Government of Canada. We are currently recruiting for a series of small group discussions to obtain the opinion and perception of federal government employees about self-service online tools that will allow them to obtain information regarding their pay file.
We are conducting these group discussions in the next couple of weeks and I would like to know if you are interested in taking part. Those who attend will receive [$150 not taxable income] in appreciation for their time. May I ask you a few quick questions at this time to see if you qualify to participate in the group discussion? Your participation is voluntary. All information collected, used and/or disclosed will be used for research purposes only and the research is entirely confidential. The names of participants will not be provided to the government or any third party.
For online groups only: The financial incentive [$150 not taxable income] will be mailed to you following the group session in which you will participate.
May I continue?
Note to recruiter: When terminating a call with someone, say: Thank you for your cooperation. We already have enough participants who have a similar profile to yours, so we are unable to invite you to participate.
To begin:
In which province or territory do you live?
| Province | Province / Territory abbreviation | Groups |
|---|---|---|
| British Columbia (-3 hours) | BC | Groups 4, 5 |
| Alberta (-2 hours) | AB | Groups 4, 5 |
| Saskatchewan (-1 hour) | SK | Groups 4, 5 |
| Manitoba (-1 hour) | MB | Groups 4, 5 |
| Ontario (+0 hour) | ON | Groups 1, 2, 4, 5 |
| Quebec (+0 hour) | QC | Groups 1, 2, 3, 4, 5 |
| New Brunswick (+1 hour) | NB | Groups 4, 5 |
| Nova Scotia (+1 hour) | NS | Groups 4, 5 |
| Prince Edward Island (+1 hour) | PE | Groups 4, 5 |
| Newfoundland (+1.5 hour) | NF | Groups 4, 5 |
| Northwest Territories (-2 hours) | NT | Groups 4, 5 |
| Yukon (-3 hours) | YK | Groups 4, 5 |
| Nunavut (+0 hour) | NU | Groups 4, 5 |
Is your address located in the National Capital Region (Ottawa/Gatineau)?
Is your address located in the Greater Montreal Area?
The group discussions we are organizing are going to be held over the Internet. They will be "online focus groups." Participants will need to have a computer (no tablet, no cell), a high-speed internet connection and a WebCam in order to participate in these groups. Would you be able to participate under these conditions?
1. Are you a… Do not ask the question – (proceed by observation)
2. Which of the following age groups applies to you? Are you…?
3. Sometimes participants are asked to watch videos, read a document and/or write out their answers to a questionnaire during the discussion. Is there any reason why you could not participate?
Note: Thank and terminate if respondent offers any reson such as sight or hearing problem, a written or verbal language problem, a concern with not being able to communicate effectively or if you have a concern.
4. In which government department or agency do you work?
If Public Services and Procurement Canada (Code 27), exclude
If Treasury Board of Canada Secretariat (Code 29), exclude
5. What is your job title?
Thank and terminate if:
Accept other job titles
96) (Please specify):
6. How long have you worked for the federal government?
7. Do you use the Compensation Web Applications to access your pay stubs or other pay information?
8. Have you ever contacted the pay centre or your compensation advisor for a pay issue either by telephone, through a pay action request, through a feedback form?
9. Are you currently working in “Human Resources”?
10. Are you currently working in “Information Technology (IT)”?
11. Are you currently working in “Compensation/Pay”?
12. Are you currently working directly in “a Deputy Minister’s Office”?
13. Are you currently working directly in “a Minister’s Office”?
14. Are you currently working directly in “Communication”?
15. Are you currently working directly in “Finance”?
Note for recruiters: Read 9 to 15 completely
If Yes to one of the question 9 to 15, Thank and terminate
Note for recruiters: Do not read
We want groups with a good mix of participants who have had problems with their pay and others who have not. Make sure to have a good mix [See question 8].
1. Have you ever attended a focus group discussion for which you received a sum of money?
If yes
2. And when was the last time you attended a discussion group?
3. How many times did you attend a discussion group or an in-depth interview in the last five years?
| Group | Location | Group profile | Language | Participants | Dates | Time (Eastern time) |
Type | Location |
|---|---|---|---|---|---|---|---|---|
| GR01 | Ottawa | Pay Centre Departments and Agencies | EN | 8-10 | Sept 11, 2018 | 5:30 PM | In-person | Stratcom 100 Sparks Street Suite 802/ 8th Floor Ottawa, ON K1P 5B7 |
| GR02 | Ottawa | Pay Centre Departments and Agencies | EN | 8-10 | Sept 11, 2018 | 7:30 PM | In-person | Stratcom 100 Sparks Street Suite 802/ 8th Floor Ottawa, ON K1P 5B7 |
| GR03 | Montreal | Non-Pay Centre Departments and Agencies | FR | 8-10 | Sept 17, 2018 | 5:30 PM | In-person | Léger 700-507 Place d’Armes Montréal (QC) H2Y 2W8 |
| GR04 | Across Canada | Pay Centre Departments and Agencies | FR | 8 | Sept 19, 2018 | 5:30 PM | Online | ITracks |
| GR05 | Across Canada | Non-Pay Centre Departments and Agencies | EN | 8 | Sept 19, 2018 | 7:30 PM | Online | ITracks |
Group specifications:
Someone from our office will be calling you back to confirm these arrangements. Could I please have your phone number where we can reach you during the evening and during the day?
Name:
Evening phone: Work phone:
Thank you very much!
Recruited by:
Confirmed by:
As we are only inviting a small number of people to take part, your participation is very important to us. If for some reason you are unable to participate, please call so that we can get someone to replace you. You can reach us at at our office. Please ask for .
Now I have a few questions that relate to privacy, your personal information and the research process. We will need your consent on a few issues that enable us to conduct our research. As I run through these questions, please feel free to ask me any questions you would like clarified.
P1) First, we will be providing the hosting facility and session moderator with a list of respondents’ names and profiles (screener responses) so that they can sign you into the group. This information will not be shared with the Government of Canada department organizing this research. Do we have your permission to do this? I assure you it will be kept strictly confidential.
We need to provide the facility hosting the session and the moderator with the names and background of the people attending the focus group because only the individuals invited are allowed in the session and the facility and moderator must have this information for verification purposes. Please be assured that this information will be kept strictly confidential. Go to P1A
P1a) Now that I’ve explained this, do I have your permission to provide your name and profile to the facility (Group 1-2-3) / to the online platform (Group 4-5)?
P2) An audio and/or video tape of the group session will be produced for research purposes. The tapes will be used by the team of researchers at Léger to assist in preparing a report on the research findings.
Do you agree to be audio and/or video taped for research purposes only?
It is necessary for the research process for us to audio/video tape the session as the researcher needs this material to complete the report.
P2a) Now that I’ve explained this, do I have your permission for audio/video taping?
Do you have a pen handy so that I can give you the address where the group will be held? It will be held at:
To ensure that the focus groups run smoothly, we remind you:
If necessary and only if necessary (e.g. for research validation), please refer recruits to the Public Opinion Coordinator who conducts the research:
Mike Walker
Coordonnateur de recherche en opinion publique | Public Opinion Research Coordinator
Secteur des communications | Communications Sector
Travaux publics et Services gouvernementaux Canada | Public Works and Government Services Canada
Téléphone | Telephone: 873.469.3466
Duration: 10 minutes
Explanation
Describe how a discussion group functions:
Duration: 15 minutes
First, let’s discuss how you feel about your digital skills.
1.1 First, how would you rate your digital skills? Would you say they are poor, okay, good or excellent? By digital skills, I mean your ability to successfully and effectively use technology at your disposal.
1.2 Do you think your digital skills are sufficient to effectively use the technologies and interactive tools offered by your employer?
1.3 Would you say it is easy for you to find online information you need in your job?
Now, let’s discuss the Compensation Web Applications.
1.4 Most of you said that you are using the Compensation Web Applications
Duration: 85 minutes
I will now show you some screenshots. These are screenshots of an online application that is currently in the design phase. This is only a demo. But we would like to hear your opinion on this.
2.1 Homepage
The moderator introduces the Homepage to the television screen and distributes printed sheets of the Homepage screen to the participants. Participants are invited to circle or cross out elements that are not clear to them or that they do not like.
Review the information on the home page by major sections:
2.2 For each section of the Homepage?
2.3 Profile, Employment, Benefits pages
The moderator introduces the other pages to the participants by distributing printed sheets of the screenshot to the participants. Participants are invited to circle or cross out elements that are not clear to them or that they do not like.
2.4 Support pages
The moderator introduces the support pages to the participants by distributing printed sheets of the screenshot to the participants. Participants are invited to circle or cross out elements that are not clear to them or that they do not like.
I will now show you a short video that further explains the concept of Pay Insight. Please note that this short video is not made by a professional.
The facilitator introduces the Demo and lets participants watch the video. The Demo is presented on the TV screen.
Remember that this application is still being designed. It is not final. It can change again before being finalized.
2.5 First of all, what is your overall reaction to the Pay Insight concept? What makes you say that?
Write down your first impressions and then we’ll discuss.
Please note that the look of the application may change since it is only a demo. It is not final.
Note: Section E—Track MyCase—Only with Pay Centre Departments and Agencies Focus Groups
2.6 Track my Cases page
I will now show you some more screenshots. These screenshots present the Track MyCASE application. This application will be integrated into PayInsight.
The moderator introduces the Cases pages to the participants by distributing printed sheets of the screenshot to the participants. Participants are invited to circle or cross out elements that are not clear to them or that they do not like.
Repeat this section for all the screenshots (8) presented for the Track MyCASE concept.
Duration: 10 minutes
3.1 For the last section of our discussion, I will ask you to think about your employment situation and pay issues (or problems with your pay account) and the steps you are taking to have them resolved (or even explained).
Duration: 5 minutes
4.1 Finally, if you had a chance to speak directly with the creator of this online tool, what would you tell him or her? What should be his number one priority to ensure the online tool adequately meets the needs of employees who have pay issues and need information about their pay account?
4.2 Do you have any other remarks/suggestions or comments about this online tool?
Thank you very much for your valuable participation!
End of group