Client Perspective Research 2018

Topics menu

Executive Summary

The Canada School of Public Service (CSPS) was established in 2004 to provide a broad range of learning opportunities and establish a culture of learning within the public service. To deliver on its mandate, CSPS launched GCcampus, an online learning portal designed to provide a government-wide, integrated approach to sharing learning resources, reducing costs of training for clients and effectively deploying technology to advance federal public servants' continuing education. GCcampus represents a primary point of interaction with CSPS for many of its clients within the public service. In order to ensure that it is effectively marketing its learning opportunities to this population, and, moreover, that these opportunities are responsive to the needs of the public service, CSPS identified the need to conduct research on its brand among the public service.

An online survey (Phase 1) was conducted with 4,632 federal public servants, drawn from a random selection of employees who are indeterminately employed within the federal public service. The Phase 1 results were based on a random selection drawn from a database of 247,540 public servants, representing 94% of the total population of 262,696 public servants. The final sample obtains a +/-1.5 percentage point margin of error, calculated at a 95% confidence interval (i.e., had the entire population of public servants been interviewed, individual survey results among the entire population would be within 1.5 percentage points of the results, 19 times out of 20). The survey was conducted between late March and mid-April, using a questionnaire requiring 24 minutes on average to complete. The response rate for the survey was 13%. A similar survey (Phase 2) was also conducted with 348 individuals considered to be members of internal audiences, including CSPS employees and those working outside CSPS to liaise with departments. This includes Departmental Training Coordinators or Points of Contact (POC) that liaise with CSPS on training programs that benefit public servants in their departments. This attempted census, capturing 36% of the population, was collected online in April, using a 24-minute survey questionnaire.

Given the high degree of coverage of the population of public servants used for the random sample selection, as well as the significant sample size of public servants, we believe that the results of the Phase 1 survey may be extrapolated to the entire population of federal public servants with confidence. The Phase 2 results are based on an attempted census with a response rate reflecting roughly one in three members of the population. We are not aware of any significant response bias, other than a slight over representation of CSPS employees, for which the results were weighted in the analysis. Therefore, we also believe that the results of Phase 2 may be extrapolated to the population of internal audiences with confidence.

Phase 3 consists of 20 focus groups conducted with Phase 1 survey respondents. The purpose of the groups was to provide additional context to the survey results, adding the dimension of personal experience to add context to the quantitative survey findings. The groups were divided between the regions (with 11 groups conducted among participants outside of the National Capital Region (NCR)) and Ottawa (with 9 groups conducted among participants working in the NCR).

Key Findings

Awareness (Public Servants)

Survey results highlight broad awareness of CSPS among federal public servants. In fact, nine in ten (89%) indicated that they had heard something about the organization. Familiarity with the CSPS GCcampus portal is somewhat lower, although three quarters (75%) have heard of it, and about half of public servants understand that GCcampus is the CSPS online platform. As expected, familiarity with CSPS and GCcampus is considerably higher among internal audiences, although not as high for the portal as it is for the School.

Results further suggest that public servants have known about these services for many years and are routinely kept up-to-date. Among those who have heard of CSPS, the majority first heard of the organization more than two years ago. Initial exposure to GCcampus is significantly more recent, with just three in ten having first heard of the service during this time frame. Clear majorities said they have heard, read, or seen something about CSPS or GCcampus within the past year.

During the focus groups, participants also indicated a high level of awareness of both CSPS and GCcampus, but many also said they were not clear on what either CSPS or GCcampus offer them. "Unknown" was among the most common descriptions of both CSPS and GCcampus when Phase 1 survey respondents were asked to describe each with a single word or phrase. Focus group participants often echoed this in remarks that encouraged the School to communicate with them and with their management more clearly and completely about what it is that they have to offer public servants.

Along with the general lack of familiarity with the School, focus group participants were also unclear on the relationship between the "Canada School of Public Service" and "GCcampus." During the Phase 1 survey, respondents were split between those who identify GCcampus as the online portal to CSPS products, and those who said they did not know what the relationship between the two is. During the focus group discussions, it became clear that while many intuited the relationship between the School and GCcampus, it was more a matter of guesswork than knowledge gained as a result of a clearly and consistently communicated brand.

This lack of clarity seems to have also contributed to a misperception about what GCcampus actually is. While many focus group participants understand it to be a portal to CSPS products, many described it as the "online arm" or as consisting only of online offerings, rather than understanding GCcampus as the online portal for all dealings with the School, both online and in-person. By comparison, CSPS is seen as the broader and more encompassing brand, whereas GCcampus is seen as merely an online spin-off.

In terms of a brand that best communicates what the School offers public servants, most focus group participants said that the "Canada School of Public Service" is the clearer, and better-known option. That said, GCcampus remains a viable option as a sub-brand, provided it is presented under a common look and feel, according to some.

Reach (Public Servants and Internal Audiences)

Results demonstrate wide-ranging use of the products and services offered, given that eight in ten public servants who have heard of CSPS and GCcampus indicated that they have used them (more than seven in ten of all public servants). For most public servants responding in the Phase 1 survey, services used were courses, workshops or training programs, attended online or in-person, although about one in three read or used information or watched videos available on GCcampus. Two in three of those public servants aware of the website have created a GCcampus account, and four in ten have accessed products and services on the site. Among those who have accessed products and services, half accessed them within the past three months, although one in seven said it was more than two years ago. For eight in ten public service users of the services or products, the primary motivation was that it was required of them. However, more than one in three also said that they found something that was of interest to them, and another 18% had something recommended to them by a colleague.

Three in four public servants attending a course, workshop or training did so online. However, this method is somewhat less popular for events, where just under two in three attended online. Email consistently ranks as the most effective method for reaching public servants with information about CSPS and GCcampus, although colleagues and supervisors were also commonly mentioned. Supervisor referrals appear far more effective in eliciting attendance, as a clear plurality of public servants (43%) cited their supervisor as the medium through which they most recently accessed a CSPS product or service.

During the focus groups, participants often said that the School seems to be moving more of its courses online. For most, this is not seen as a positive direction. The online learning environment was described by many as limited in value and lacking in the person-to-person interactions between students and instructors that aid in learning through shared experiences. Participation in events online – described by participants as "webinars" – are seen as fraught with technical challenges. Even when functioning properly, these are seen by several as a more limited form of participation than in-person. On the other hand, in terms of communicating about CSPS offerings, email was also identified as the most effective means of providing this information, according to many participants. Many did, however, emphasize the need to target email communications to the particular needs of users.

Internal audiences responding in the Phase 2 survey are much more likely to have engaged with CSPS, accessing and using a greater number of products and services, including more than eight in ten who have taken a course, workshop or other training online, and almost as many who have attended in-person. Seven in ten have also used GCcampus material and/or watched a video, and nearly as many have also participated in events, conferences or armchair discussions, either in-person or online.

Feedback on CSPS Products and Services (Public Servants and Internal Audiences)

Public servants responding to the Phase 1 survey who have attended a recent event, conference, or armchair discussion are reasonably satisfied with CSPS/GCcampus events. Most (four in five) rated their experience with the most recent event, conference, or armchair discussion as positive. Three in four feel it was well organized, and nearly as many indicated that it was easy to register for and attend a CSPS/GCcampus event. Two in three expressed significant interest in attending future events, and/or would recommend other events to friends or colleagues. Less positive is the fact that only half of public servants who attended an event, conference, or armchair discussion feel they learned something that they were later able to apply to their work, although one in five said "maybe" or were unsure.

Internal audiences in Phase 2 are considerably more positive. As with public servants, organization is the top-rated aspect of these events, while relevance is rated less positively. In Phase 2, ratings were provided separately for events held in-person and online, with in-person events garnering more positive results, with the exception of relevance for which results are the same.

Similar to attitudes towards events, the majority of public servants in Phase 1 who attended are satisfied with the most recent course, workshop or other training opportunity. Three in four public servants who have attended CSPS/GCcampus training feel that it was well organized. Nearly as many found it easy to register for and attend. Somewhat less positively, roughly two in three believe CSPS/GCcampus training to be very relevant, they would be very interested in attending future courses or workshops, the training provided a lot of useful information, or they would recommend other courses or workshops to friends or colleagues. Less positive input was provided about the instructor of the course or workshop, with only half rating the performance positively, although more than one third were not sure or do not recall. As with events, only just over half of attendees believe they learned something from the training that they were able to later apply to their work.

Internal audiences in Phase 2 are considerably more positive than other public servants with regard to the evaluation of training, and as with events, in-person training garners higher ratings than training delivered online. In-person training rated positively by between eight and nine in ten internal audience members, with welcoming nature and organization at the top of the list, followed by interest in future training, and willingness to recommend the training to others. In the case of in-person training, relevance is not too far down the list. Ease of registration in online training is rated the most positively (78%), while organization and relevance fall somewhere just below. Interest in future online courses and willingness to make a recommendation to others, however, fall considerably below these (67% and 69%, respectively). Lower still, the welcoming nature of an online course is only rated positively by 56%.

Relative to events and training, satisfaction with material is slightly lower. Seven in ten public servants in Phase 1 rated the information or videos found through CSPS or GCcampus positively. Just under seven in ten found the information and/or videos easy to access or well organized. Slightly fewer found that material relevant or that they would be interested in accessing more material in the future. Again reflecting a less positive result, roughly six in ten believe the information or videos provided useful information or that they would recommend the material to friends or colleagues. As with the courses and discussions, just over half of public servants in the survey feel that they were able to later apply the information in print or video to their work, again suggesting only a moderate impact.

Results are marginally higher among internal audiences responding in Phase 2, with three in four agreeing that it was easy to access, well organized, relevant and sparked interest in future material. Seven in ten also said they would recommend the material to others.

Phase 2 respondents also provided their second-hand observations about public servants' perception of the performance of CSPS. Results point to the welcoming environment as the aspect of the School held in highest regard, followed by the relevance and usefulness of the products and services. Ease of access and organization are judged by internal audiences to be the areas in which public servants have the least positive view of the School. These results, however, run counter, to some extent, to the actual feedback provided by public servants in Phase 1.

Most public servants and internal audiences (Phases 1 and 2) who accessed services or products feel the experience with accessing and using products and services offered by CSPS or GCcampus is easy. Only one in three, however, said they have not had any challenges with accessing CSPS or GCcampus services. Challenges experienced related predominantly to slow connection speeds, difficulty obtaining log-on information, or the website freezing or crashing. More than eight in ten did not have any difficulty accessing products or services from CSPS or GCcampus in the official language of their choice.

Phase 3 focus group participants expressed a range of views about the products and services offered by the School, from those who described their experience as extremely valuable, to those who found very little use in courses and events they had participated in from the School. Participants were far more likely to describe their experience with courses and events than they were of materials. A common criticism was that CSPS course offerings lack a central focus and cohesive structure that enables clients to clearly understand the CSPS brand and role it plays in the training arena among public servants (i.e., the space it occupies in terms of training for public servants). Perhaps more importantly, this lack of organizational structure seems to make perusing and finding relevant courses a more daunting task, according to some. Further, the CSPS overall service offering is also seen by some as lacking in clear value in terms of assisting public servants with critical elements of either their current job, or their career progression. These two perceptions are brought together under the overall view among many that CSPS courses are too diffusely targeted to be valuable to people in meeting specific challenges in their day-to-day jobs, or in achieving specifically focused career objectives.

Client Service with CSPS (Public Servants and Internal Audiences)

One in four Phase 1 public servants who are aware of CSPS services have contacted CSPS or GCcampus with questions or issues related to a product or service offered, nearly half within the past year. Feedback about the service provided in that contact is reasonably positive. Three in four of those who contacted CSPS or GCcampus for support feel that the person they dealt with was courteous. On the other hand, just under two in three felt that the client support person they dealt with understood their needs. Only half feel that the person they dealt with provided good information or advice, was able to satisfactorily answer the question or problem, was generally knowledgeable, or that the question or problem was solved in a reasonable amount of time.

As might be expected, the incidence of contact among internal audiences (Phase 2) is considerably higher at six in ten. Feedback on the contact is similar, although results for the courteousness of the individual contacted are higher.

Interest in a Mobile App (Public Servants and Internal Audiences)

Interest in a mobile app is fairly low, with only about one quarter of public servants indicating any likelihood of downloading a CSPS or GCcampus app. Internal audiences are more receptive than other public servants with over half indicating some likelihood of using it. Many less interested in an app indicated either significant barriers with being able to use an app (e.g., limitations of BlackBerrys, no access to a smart phone for work purposes), or a preference for using the website, and/or reserving apps for other, more commonly used purposes.

If an app were available, half would be interested in some of the tested features, including course materials, information about events, courses or other products, or job aids. Responses among internal audiences, however, are considerably less positive. In terms of suggestions provided by public servants and internal audiences for app content or features, the most frequently put forward include tailored suggestions for training and/or events, and notifications about upcoming events, ability to maintain a calendar, and receive last-minute notifications. Another key feature put forward is the ability to build and maintain a personal profile or learning history of courses taken and certificates completed, as well as learning objectives. Finally, features related to language tools and the ability to keep important or frequently needed information (e.g., list of key terms, flashcards) were also put forward.

Interest in Future Products and Services (Public Servants and Internal Audiences)

There is moderately high interest among public servants in hearing more about learning opportunities offered by CSPS or GCcampus, with seven in ten somewhat or very interested, although only one in four indicated a strong interest. When asked about suggestions for products or services that would be of particular interest, central themes related to courses in language training and/or retirement planning. These were followed by suggestions for courses and events to address soft skills (e.g., time management, communications, team dynamics, etiquette) and performance development tools (e.g., software, project management).

Among managers of other public servants, four in five managers agree that continuous learning opportunities result in staff who are more productive and happier with their careers. One third, however, feel that it is difficult to allow staff the time needed for continuous learning while still meeting work objectives. Most see the value in sending staff for continuous learning since they will get the needed training on the job.

Among internal audiences, time constraints are seen as a primary barrier to pursuing further learning, according to eight in ten, and four in ten believe that public servants would prefer to seek learning opportunities outside of the federal government. Another one third feel that many public servants – particularly older ones – simply do not see continuous learning as necessary. One in four also believe that supervisors within their organization are not supportive of continuous learning.

Feedback on CSPS Communications (Internal Audiences)

While internal audiences in Phase 2 indicated broad satisfaction with the quality of the information they receive from the School, concerns were flagged in other areas. For example, only six in ten feel they receive enough information from CSPS, and just over half feel that the information is timely.

In terms of the services provided by CSPS, half had used the Departmental Annual Learning Report or the Director's newsletter. Just over four in ten had made use of self-service reports in I-LMS or the POC Forum, and about one in three have used the Operations Dashboard, the RTC plenary sessions, or RTC training.

Among users of these products and services, results are varied. Nine in ten find the RTC's training useful, and eight in ten rated the director's newsletter and RTC plenary sessions as helpful. Only two in three, however, said the same about the self-service reports in I-LMS and the POC Forum. Six in ten found the Departmental Annual Learning Report and the RTC working groups to be of use, and only half feel the operations dashboards are helpful.

Internal Audience Views about their Perceived Value (Public Servants and Internal Audiences)

Although three in four CSPS employees feel comfortable contributing ideas, only six in ten feel that management is listening. Other employees supporting the School are even less apt to feel comfortable in contributing ideas or that their contribution will be listened to by the School. Highlighting similarly modest results, roughly six in ten CSPS employees and others supporting the School believe that their work is valued by the School, management, and public servants. The same proportion believes that the School is heading in the right direction.

Political Neutrality Certification

This certification is to be submitted with the final report submitted to the Project Authority.

I hereby certify as Senior Officer of EKOS Research Associates Inc. that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research.

Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leaders.

Signed by: Will Daley (Vice-President)