Grants and Contributions Applicants Client Experience Research (Year 2)

Employment and Social Development Canada [ESDC]

June 1, 2022

POR# 060-21
SUPPLIER: Ipsos Limited Partnership
CONTRACT AWARD DATE: 2021-12-08
CONTRACT #: G9292-229941/001/CY
Contract value: $140,330.26 (tax included)

Ce rapport est aussi disponible en français.

For more information on this report, please contact nc-por-rop-gd@hrsdc-rhdcc.gc.ca

Grants and Contributions Applicants Client Experience Research (Year 2)

It is available upon request in multiple formats (large print, MP3, braille, e-text, DAISY), by contacting 1-800 O-Canada (1-800-622-6232).
By teletypewriter (TTY), call 1-800-926-9105.

© His Majesty the King in Right of Canada, as represented by the Minister of Families, Children and Social Development, 2022
https://publications.gc.ca/site/fra/services/modeleDroitsAuteur.html
For information regarding reproduction rights: droitdauteur.copyright@HRSDC-RHDCC.gc.ca.

PDF
Cat. No. : Em20-148/2022E-PDF
ISBN: 978-0-660-44219-8

Recherche sur l’expérience client des subventions et contributions (Année 2)

Ce document offert sur demande en médias substituts (gros caractères, MP3, braille, fichiers de texte, DAISY) auprès du 1-800 O-Canada (1-800-622-6232).
Si vous utilisez un téléscripteur (ATS), composez le 1-800-926-9105.

© Sa Majesté le Roi du Chef du Canada, représenté par le ministre de la Famille, des Enfants et du Développement social, 2022
https://publications.gc.ca/site/fra/services/modeleDroitsAuteur.html
Pour des renseignements sur les droits de reproduction: droitdauteur.copyright@HRSDC-RHDCC.gc.ca.

PDF
Nº de cat. : Em20-148/2022F-PDF
ISBN : 978-0-660-44220-4

Political Neutrality Statement

I hereby certify as Senior Officer of Ipsos that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Policy on Communications and Federal Identity and the Directive on the Management of Communications. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.

Mike Colledge
President
Ipsos Public Affairs

Additional information

Supplier Name: Ipsos Limited Partnership
PSPC Contract Number: G9292-229941/001/CY
Contract Award Date: 2021-12-08

Executive Summary

Grants & Contributions CX Survey – Results At a Glance

• 1,942 SURVEYS CONDUCTED
• METHODOLOGY: ONLINE SURVEY
• FIELDWORK: February 16 to March 15, 2022

Overall Service Experience

Click for larger view

Figure 1: Overall Service Experience

This horizontal bar chart shows responses to three questions about the overall service experience and presents results for Year 1 and Year 2. Respondents were asked to provide ratings on a 5-pt scale and results were grouped by those who provided a rating of 4 or 5. All 1942 respondents in Year 2 answered as follows:

• Overall Satisfaction: Year 2 77%. Year 1 70%.
• Ease: Year 2 79%. Year 1 74%.
• Effectiveness: Year 2 78%. Year 1 70%.

Satisfaction with Service Channels

Click for larger view

Figure 2: Satisfaction with Service Channels

This horizontal bar chart shows responses to a question about satisfaction with the overall quality of service provided by the service channels used during the applicant process and presents results for Year 1 and Year 2. Respondents were asked to provide ratings on a 5-pt scale and results were grouped by those who provided a rating of 4 or 5. Sample sizes vary by service channel, only those who used each channel during their experience were asked about it.

• Email support from Program Officer (627 respondents answered this question in Year 2): Year 2 79%. Year 1 80%.
• GCOS Web Portal (623 respondents answered this question in Year 2): Year 2 76%. Year 1 67%.
• Online (1365 respondents answered this question in Year 2): Year 2 71%. Year 1 66%.
• Email support from SC (1580 respondents answered this question in Year 2): Year 2 70%. Year 1 65%.
• In-Person (29 respondents answered this question in Year 2): Year 2 62%. Year 1 66%.
• Mail (139 respondents answered this question in Year 2): Year 2 58%. Year 1 63%.
• Phone support from SC (427 respondents answered this question in Year 2): Year 2 61%. Year 1 61%.
• 1 800 O Canada (72 respondents answered this question in Year 2): Year 2 48%.Year 1 49%.

Satisfaction with Client Experience by Program

Click for larger view

Figure 3: Satisfaction with Client Experience by Program

This vertical bar chart shows responses to a question about satisfaction with overall service experience by program and presents results for Year 1 and Year 2. Respondents were asked to provide ratings on a 5-pt scale and results were grouped by those who provided a rating of 4 or 5. Sample sizes vary by program.

• EAF (207 respondents answered this question in Year 2): Year 2 78%. Year 1 77%.
• NHSP (384 respondents answered this question in Year 2): Year 2 83%. Year 1 73%.
• CSJ (865 respondents answered this question in Year 2): Year 2 79%. Year 1 69%.
• YESS+ (152 respondents answered this question in Year 2): Year 2 62%. Year 1 60%.
• UT&IP (32 respondents answered this question in Year 2): Year 2 78%. Year 1 73%.
• EL&CCI (65 respondents answered this question in Year 2): Year 2 19%. Year 1 60%.
• SDPP (153 respondents answered this question in Year 2): Year 2 72%. Year 1 53%.
• FCRP (20 respondents answered this question in Year 2): Year 2 65%.
• IELCC (8 respondents answered this question in Year 2): Year 2 75%.
• IWILI (13 respondents answered this question in Year 2): Year 2 62%.
• SWP (4 respondents answered this question in Year 2): Year 2 75%.
• SDG (39 respondents answered this question in Year 2): Year 2 (39%).

Funding approval

Click for larger view

Satisfaction by Approval Status

• Year 2: Approved 82%. Denied 47%.
• Year 1: Approved 74%. Denied 41%.

Figure 4: Funding approval

This horizontal bar chart shows responses to a question about whether the applicant received funding approval and presents results for Year 1 and Year 2. A total of 1820 respondents in Year 2 answered as follows:

• Year 2: Approved 93%. Denied 7%.
• Year 1: Approved 90%. Denied 10%.

Strengths

Click for larger view

Figure description: Strengths

• Service in choice of official language 93%
• Completing steps online made the process easier 88%
• Ease of determining if organization is eligible for program funding 84%
• Confident personal information protected 83%

Areas for Improvement

Click for larger view

Figure description: Areas for Improvement

• Determine the amount of time each phase is anticipated to take 58%
• Client journey took reasonable time 66%
• Needed to explain situation only once 67%
• Ease of completing the budget document 67%

* referred to as [program] web portal in Year 1

Significantly higher / lower than total

Significantly higher/lower than Year 1

Key Findings - Quantitative Findings

Overall, applicants to Grants and Contributions programs were more satisfied with the process in Year 2 compared to Year 1. 

• More than three-quarters (77%) of applicants were satisfied with their overall experience, an increase of seven points from Year 1 (70%), while fewer were dissatisfied (7%, -5 points). The vast majority felt it was easy to apply (79%) and move smoothly through all steps (78%), with improvement on both measures compared to Year 1.
• Satisfaction was highest for NHSP (83%), CSJ (79%), EAF (78%) and UT&IP (78%), followed by IELCC and SWP (both 75%), SDPP (72%), FCRP (65%), YESS (62%) and IWLIL (62%), while ratings were lowest for SDG (39%) and EL&CCI (19%). 

Satisfaction was driven by improvements in the overall ease, effectiveness and timeliness of the process among CSJ and NHSP applicants. First-time applicants, those applying to higher complexity programs and organizations more reliant on volunteers continued to experience more difficulties navigating the application process.  

• The overall increase in satisfaction is due primarily to higher ratings among applicants to NHSP and CSJ, who represent the vast majority of Grants and Contributions applicants and experienced the most positive improvement in the various aspects of the service experience. Those who applied to EL&CCI were less satisfied than in Year 1.
• Overall, applicants provided higher ratings for several aspects of ease and effectiveness and fewer reported experiencing problems or issues during the application process. The most notable improvements in the service experience have been in the timeliness of service, clarity of the application process, issue resolution and ease of getting assistance. 

The aspect of service which had the greatest impact on satisfaction is the helpfulness of Service Canada and 1-800 O-Canada phone representatives, followed by the amount of time it took from start to finish, the ease of getting help and clarity of process. These also represent the aspects of service where ratings were relatively lower than other areas.   

• The helpfulness of phone representatives (Service Canada and 1-800 O-Canada) has taken on increased importance and become the primary driver of satisfaction in Year 2, while the ease of getting help when needed has also become more important. 
• Working to further reduce the amount of time the application process takes, improving access to assistance when needed and the ability of Service Canada phone representatives to assist applicants represent the greatest opportunities to improve the service experience given their strong impact on satisfaction and relatively lower ratings. 
• Awareness of service standards also had an impact on satisfaction and fewer than half of applicants were aware of each standard in Year 2. Those who were aware of service standards had a more positive experience, with fewer issues and were more satisfied with most service channels, the timeliness of service, clarity of the process (including issue resolution) and ease of getting assistance. Working to more clearly communicate service standards to applicants should help to improve impressions of the application process. 

Those who were applying to the program for the first time, in particular for higher complexity programs, had greater difficulty with the process which required more contact with Service Canada and time on the part of the applicant. The resulting impact was lower overall satisfaction with their experience applying.   

• First-time applicants were typically smaller, younger organizations, which relied more heavily on volunteers to complete the application and required more contact with Service Canada and effort to complete the application. They experienced more challenges particularly in regards to the clarity of the steps and timelines for the application process, ease of getting assistance and overall timeliness of the client journey. 
• Notably, most applicants to CSJ reported they apply on the same basis, while applicants to most other programs and specifically those of higher complexity had less experience applying for the particular program. 
• As noted in Year 1, satisfaction declined by the number of times the client contacted Service Canada. While applicants reported having fewer contacts than in Year 1, the highest proportion continue to have had more than 10 or more contacts during their experience which is more prominent among applicants to higher complexity programs. 

Applicants who were not approved for funding continued to have considerably lower satisfaction with their experience than those who were funded. Most were not provided an explanation for the decision and few who were expressed satisfaction with the rationale.  Providing organizations which were unsuccessful a better understanding of the reasons for not receiving approval should help to improve their satisfaction with the process.   

• While fewer applicants were denied funding than in Year 1 (7% vs. 10%), only half (47%) of those were satisfied with the application process compared to more than eight in ten (82%) of those approved. 
• Most of those who did not receive funding approval were not provided an explanation why and of those who were few were satisfied with the explanation provided. Applicants to programs other than EAF and CSJ were less likely to have been satisfied with the explanation provided. 
• Notably, applicants to CSJ were more likely to have received funding approval compared to Year 1 and those who applied to EL&CCI were considerably less likely. Given the impact of funding approval on satisfaction with the application process, the shifts observed in the proportion who received funding is likely a contributing factor to the increase in satisfaction seen for CSJ and decrease for EL&CCI compared to Year 1. 

Support provided through email from a program officer remained the highest rated service channel, while telephone channels continued to receive lower ratings. Satisfaction with online channels, including the GCOS web portal and Government of Canada website, has increased and a greater proportion of applicants felt completing steps online improved the ease of the process.   

• Eight in ten were satisfied with the email support from a program officer (79%), followed by the GCOS web portal (76%), Government of Canada website (71%) and email support form a Service Canada office (70%). 
• Fewer were satisfied with in-person service at a Service Canada office (62%), telephone support from a Service Canada office (61%) and mail (58%), while satisfaction remained lowest for the 1-800 O-Canada (48%). 
• The highest rated aspects of service remained the provision of service in choice of official language, that completing steps online made the process easier, confidence in security of their personal information and ease of determining eligibility. Improvement has been made in the ease of completing steps online and determining eligibly. However, compared to Year 1, fewer were confident their information was protected or were provided service in their choice of official language.  
• Aspects of service with lower ratings included the ease of determining how long each phase of the process was anticipated to take, the overall amount of time it took, having to explain their situation only once and ease of completing the budget document. Applicants were more satisfied with the timeliness of service and only explaining themselves once than in Year 1, but they remain areas for further improvement. 

Applicants continued to rely most heavily on the Government of Canada website in the stages leading up to submitting their application and the vast majority found it easy to find what they were looking for. There has been improvements in the clarity of information online and ease of determining the steps to apply, however ratings were lower for determining how long each phase of the process is anticipated to take.   

• When learning about the program, applicants were most likely to have received email communication directly from the GoC, ESDC, or the program they applied to (57%) or to visit the Government of Canada website for the program (48%), followed by the general Government of Canada website (25%), or talked to peers/community network (23%). A greater proportion received an email from the GoC, ESDC, or the program than in Year 1 (driven by an increase among CSJ and NHSP applicants), while fewer talked to peers/community network or their MP.
• CSJ and NHSP applicants were more likely to have received an email from the GoC, ESDC, or program directly during the awareness stage, while those applying to all other programs relied more heavily on the Government of Canada website and experienced more difficulty finding the information they were looking for when doing so (EL&CCI, SDPP and SDG applicants in particular). 
• Among those who used the GoC website, ratings were highest for the ease of determining if their organization was eligible for funding, when the application period takes place, and finding general information about the program. Determining the amount of time each phase of the application process is anticipated to take was considered the most difficult information to find.

The vast majority submitted their application online and found it easy to do so. Applicants found it easier to complete most aspects of the application and that it took a reasonable amount of time to complete compared to Year 1, however impressions of the process remain much weaker among those who applied to higher complexity program.   

• Most submitted their application using the online fillable form (51%), followed by the GCOS web portal (35%), while fewer downloaded the application and submitted by email (10%) or mail (3%). YESS and CSJ applicants were more likely to have submitted using the GCOS web portal, while applicants to all programs except for CSJ were more likely to have downloaded the application and submitted them by email. 
• Improvements observed in the ease of completing the application were due to primarily to higher ratings among CSJ applicants and to a lesser extent NHSP. 
• Applicants to all other programs continued to experience more difficulty and were less likely to feel it took a reasonable amount of time to complete. Completing the budget document and narrative questions were the most challenging aspects of the application submission. 

Among those approved for funding, most found the tasks associated with funding agreement close-out easy to complete, although applicants to higher complexity programs continued to experience more challenges.   

• Consistent with Year 1, a strong majority felt each aspect of the funding agreement close-out was easy to complete and ratings were very consistent between tasks with the exception of resolving any outstanding issues with funding which received lower ratings. 
• Those who received funding through CSJ were more likely to find it easy to complete most aspects of the funding agreement close-out, while applicants of all other programs (except for NHSP) were generally less likely and ratings have declined for recipients of EAF and YESS funding across most components.

Almost all applicants supported diverse communities with their application.   

• Virtually all applicants (97%) reported that the funding they applied for would support diverse (GBA+) communities. Nearly three-quarters (73%) of applicant organizations said the funding would support those who identify as youth, followed by women (63%), those belonging to a minority racial or ethnic background (62%), low socio-economic status (53%) and Black Canadians (52%).
• Overall satisfaction is consistent among applicants who assist GBA+ communities and those who do not. In addition, overall satisfaction among those who assist GBA+ communities has increased compared to Year 1. 

Most organizations operate in Ontario, Quebec, or British Columbia and organizations operating in Quebec reported the highest satisfaction.   

• Almost two in five (38%) of applicant organizations operate in Ontario, followed by one quarter in Quebec (25%), and one in ten (13%) in British Columbia. Newfoundland and Labrador (3%), Prince Edward Island (2%), Yukon, Northwest Territories, and Nunavut (1%) were regions where the fewest number of applicant organizations operate. 
• Applicant organizations operating in Quebec reported the highest level of satisfaction with their experience (83%) and were more likely to be satisfied compared to all clients. Eight in ten (81%) organizations in Atlantic Canada were satisfied, followed by three-quarters (76%) of those in Ontario, while closer to seven in ten (72%) applicant organizations in the West or Territories which is lower compared to all client.  

Findings from the qualitative research aligned with quantitative findings.  

• Those with less external support in the Grants and Contributions process, and those with less experience in applying for grants generally, were less satisfied with the application journey and process.  

Those who identified as belonging to an equity-seeking group experienced barriers to, or concerns about, applying.  

• Some felt that ESDC itself could/should play a role in filling these gaps, particularly since many of these organizations serve equity-seeking groups.
• It is challenging for ‘bare bones’ organizations that have some general / overarching needs for funding, such as equipment upgrade, staffing, programming support.
• These applicants felt that there was little opportunity to ‘tell their story’ in terms of the full context of themselves/the communities they serve.

The organizations who are more confident when applying are better resourced and funded, have levels of collaboration, or have experienced grant writers on staff as advisors.  

• They described the application process satisfactory and easier to undertake, and they feel more confident in the type of content and information they provide within their application. 
• Enablers to applying for federal funding included experience, diversity, support and collaboration.
• Barriers included lack of resources, the need for partnerships, application requirements, lack of flexibility, perceptions of favouritism or bias, lack of soft metrics, lack of diversity, and vendor challenges.

Many participants expressed concern with a lack of user-friendliness when applying to a program through the GCOS web portal, or the fillable PDF.

• They struggled with various aspects of the portal including a general lack of user-friendliness, being unable to save progress or losing responses, confusing language/legalese, and a lack of online collaboration within their teams.
• However, there were those who considered themselves technically savvy, or had a great deal of experience applying for grants and using other similar portals, who did not share the same concerns and considered applying online easy and straightforward. 

Feedback on Service Dimensions were mixed – very few felt that their application experiences aligned with all three that were tested in the research.  

• In general, participants are seeking a flexible relationship that recognizes the importance of the work that their organizations do in their communities – and treats them as valued partners who are filling gaps and providing services that otherwise are not available to those who need them the most. 
• There was much positive feedback on Ease and Effectiveness, along with some concerns, but Emotions was the Service Dimension where many described negative emotions and experiences, frustrations and pain points.

Concerns and considerations about diversity, equity and inclusion were brought forward throughout the discussions.  

• Participants felt that there needs to be more inclusive practices in place, such as ensuring that public-facing Service Canada staff are culturally diverse, providing additional support to organizations serving equity-seeking groups and communities, and providing flexibility for those who are executing projects on-the-ground to take their communities’ unique considerations into account. 

Many wished for transparency around evaluation criteria for successful applications.  

• This included knowing the overall envelope, and what a reasonable amount their organization should apply for.
• This information would determine whether or not applying for the fund is worth the considerable time and effort it takes to apply. That said, most intended to apply for funding in future, as it is an important source of funding, especially for those organizations who lack core funding or who are not charities who can receive donations. 

An ideal experience is one that is timely where timelines are met, communication is proactive, and a two-way discussion about the both funded and unfunded applications is facilitated.  

• Further, a clear understanding of decision criteria and what content is expected from applicants, were also important to participants/applicants. 
• Making it easier for applicants to access educational information related to the grant or contribution, was consistently raised as contributing to an ideal experience.

Reasons for higher dissatisfaction with Early Learning and Childhood Innovation (ELCC) were uncovered through the qualitative research.  

• A clear picture emerged about the experience they had, most of whom characterized it as being very poor. However, this was not in relation to the decision itself, but because of significant delays for the decision, the manner and timing in which it was communicated, and the poor emotional Service Dimension experienced.
• For Sustainable Development Goals (SDG), there was a desire for greater recognition of social enterprise within it and other programs. 

Objectives and Methology

Background: Gs&Cs Client Experience Research

The Program Operations Branch (POB) within Employment and Social Development Canada (ESDC) handles the operation and coordination of the Grants and Contributions (Gs&Cs) programs across the Department. To comply with the treasury Board Policy on Service and Digital 3.1 on digital transformation, 3.2 on decision-making, service delivery, use of technology and data and client-centric service and 4.2 on client centric service design and delivery, ESDC requires the gathering of data on the client experience to assist in effectively managing service delivery. 

To meet these requirements, POB utilized the Client Experience (CX) Measurement Framework to guide the research on the Gs&Cs business line of client service delivery experience. the data collected through the implementation of the Client Experience (CX) Measurement Framework provides key information to:

• Better understand the needs and expectations of organizations;
• Identify obstacles and challenges from the perspective of the organization;
• Identify strengths and opportunities to improve CX, including opportunities to implement changes and test new approaches related to program design and delivery; 
• Assess the extent to which clients’ expectations are being met; 
• Identify and prioritize resources and opportunities tied to CX improvements; 
• Assess the impact of improvements made to the CX over time; and 
• Explore how ESDC’s leadership at all levels can play an important role in creating a positive CX.

This is the second year of POB’s Client Experience Research Program (FY 2021/22). Year Two builds on the first year of research by continuing to use a systematic approach to measuring CX in Gs&Cs service delivery and allowing the department to track process on CX indicators over time. 

The detailed methodology and research instruments for all aspects of the quantitative and qualitative research are available under a separate cover.

Note: Program intakes in grants and contributions vary widely, meaning that some year-to-year or program comparisons should be done with caution.

Research Objectives – Quantitative Research

The Client Experience Research Project is carried out in two phases, a quantitative phase and a qualitative phase. The primary objectives of Year 2 are to focus on monitoring selected programs that were previously studied in Year 1 and to capture new CX insights from programs that have not previously been studied.

The research objectives for the quantitative research were to:

• Measure service satisfaction, ease, and effectiveness of the end-to-end client experience; 
• Measure CX with service channels; 
• Build a baseline of client experience across the spectrum of Gs&Cs programs by introducing new programs while also starting to assess change and consistency by including some of the same programs as Year One; 
• Provide diagnostic insights regarding the opportunities for improvement; and 
• Assess how potential changes in service delivery might affect the CX.

Methodology – Quantitative Research

An online survey was conducted with 1,942 Service Canada applicants across 12 programs. the survey was fielded from February 16, 2022 to March 15, 2022, and took on average approximately 16 minutes to complete. the survey sample size has a margin of error of +/-2.2%.

Applicants were defined as organizations that applied for grants and contributions funding (including both funded and unfunded) within the last two intake years (2019/20 or 2020/21). A random sampling of organizations that applied to CSJ, EAF or NHSP were included, while all organizations for remaining programs were invited to complete the survey.

ESDC distributed the survey links to participating organizations. Fieldwork launch was executed using two approaches in order to better understand the impact on response rates. Half of the sampling of organizations that applied to CSJ, EAF and NHSP were sent an information email in advance of receiving the survey invitation email containing the survey link, while the other half (and those who applied to all other programs) were only sent the survey invitation email.

The exact intake periods referred to in this study are as follows:

• Canada Summer Jobs (CSJ):January to February 2020; December 2020 to February 2021
• Early Learning and Child Care – Innovation (ELCCI): October 2020 to January 2021
• Enabling Accessibility Fund (EAF): June 2020 to November 2020
• Foreign Credential Recognition Program (FCRP): March to April 2019; February 2020 to June 2020
• Indigenous Early Learning and Childcare (IELCC): February 2021 to April 2021
• Innovative Work Integrated Learning Initiative (IWILI): September 2020 to November 2020
• New Horizons for Seniors Program (NHSP): September 2020 to Oct. 2020
• Social Development Partnerships Program (SDPP): June 2020 to July 2020; March 2021 to April 2021; December 2020 to January 2021
• Student Work Placement (SWP): November 2020 to December 2020
• Sustainable Development Goals (SDG): Grants: May 2019 to November 2019; Contributions: June 2019 to Sept. 2019
• Union training and Innovation Program (UTIP): July to August 2020
• Youth Employment and Skills Strategy (YESS): June 2019 to July 2019; March 2021 to April 2021

Four (4) of the programs included in the survey have different streams that applicants can apply for.
The relevant streams referred to in this study and the exact intake periods are as follows:

• Enabling Accessibility Fund (EAF):
• Small Projects (June 2020 – July 2020)
• Youth Innovation (June 2020 – Nov. 2020)
• New Horizons for Seniors Program (NHSP):
• Small grant (up to $5000) (Sept. 2020 – Oct. 2020)
• Community-based projects (up to $25,000) (Sept. 2020 – Oct. 2020)
• Social Development Partnerships Program (SDPP):
• Supporting Black Canadian Communities (June 2020 – July 2020)
• Supporting Black Canadian Communities – West Intermediaries (Mar. 2021 – April 2021)
• Disability – Community Inclusion Initiative (Dec. 2020 – Jan. 2021)
• Union training and Innovation Program (UTIP):
• Investments in training Equipment (July 2020 – Aug. 2020)
• Innovation and Apprenticeship (July 2020 – Aug. 2020)

Of the 8,704 organizations that were invited to participate, a total of 1,942 organizations completed the survey. the response rate for the survey was 22% which is consistent with industry standards for a survey of this nature.

Note: “n=” represents the number of respondents to a question, it is known in statistical language as the size of the sample. Sample sizes below n=30 are considered small and below n=10 considered very small. Results of small and very small sample sizes should be interpreted with caution and findings viewed as directional in nature.

The quantitative survey also served as a recruitment tool for the qualitative research, by asking if organizations would be interested in voluntarily participating in focus groups or in-depth interviews at a later date.

* Response rate exceeding 100% may be due to applicants applying to more than one program and/or sampling procedures. Only those organizations with email contact information on file were invited to participate, which does not represent the total volume of applicants. 

Calibration of the Data – Quantitative Approach

Weighting adjustments were made to bring the sample into proportion with the universe by program volume based on 2019 and 2020 figures (depending on the most recent intake for the particular program).

The final data was weighted by the number of respondents in each program in proportion to the total number of applicants as detailed below. the universe proportions used to develop the targets were based on figures provided by ESDC.

Note Regarding Program Complexity

For the purpose of this study, program complexity has been defined as low, moderate, and high complexity as outlined in the following table. these service standard clusters are informed by departmental reporting in the Performance Measurement and Management Framework.

Note: Canada Summer Jobs does not fit into these distinct clusters and has been analyzed as a separate group.

Note on Reporting Conventions – Quantitative Data

Throughout the report, subgroup results have been compared to average of all applicants (i.e., total) and statistically significant differences at the 95% confidence level noted using green and red boxes.

Where subgroup results are statistically higher than the total a green box has been used and where results are statistically lower than the total a red box has been used.

Additionally, where results in Year 2 are statistically higher than Year 1, a green arrow has been used and where results in Year 2 are statistically lower than Year 1, a red arrow has been used.

Significantly higher / lower than total

Significantly higher/lower than Year 1

For the purposes of legibility, values of less than 3% have not been labelled in charts throughout the report.

Research Objectives – Qualitative Research

Building on the quantitative results, the qualitative research explored the following through focus group discussions and individual interviews with Gs&Cs applicants who have applied for funding within the past two intake years (FY 2019-20 and 2020-21): 

• Client needs and expectations: Validate and/or deepen insights regarding quantitative findings, explore the aspects that make it easy for clients as well as the obstacles/barriers clients face when going through the client experience, the impact of potential changes, and aspects that could transform the experience into a simpler and more responsive process.
• Service dimensions: Assess which service dimensions hold greater or lesser value for clients with respect to accessing service, given the complexity of the services and clients’ capacity to effectively use online services. this would allow us to validate themes to be covered in the department’s survey.
• Organizational characteristics: Investigate and understand organizational characteristics, qualities, and experiences to identify barriers and challenges faced by organizations. this may include organizational characteristics for those serving diverse populations, those organizations that were (un)successful in obtaining funding, and/or organizations that did (not) re-apply for funding.
• New or unique quantitative findings: Explore the nuances and features of the quantitative findings by probing clients or a subset of the client population to explain and contextualize their recent experiences with the Gs&Cs application process
• Based on findings from the quantitative research where certain programs had lower levels of client experience satisfaction, it was determined that applicants from Early Learning and Child Care Innovation (ELCC) and Sustainable Development Goals (SDG) would be targeted for the in-depth interviews.
• the focus groups shifted to target programs other than ELCC and SDG
• Additional questions based on quantitative findings included:
• Understanding perceptions on the length and complexity of the application
• Technical issues with the application process

Research Objectives – Qualitative Research cont’d

The findings of the qualitative research will be used to:

• Explore the client’s interactions with the department and the challenges clients may face;
• Build a deeper awareness and understanding of the clients’ experiences to inform program design and service delivery improvements;
• Identify opportunities for service-related changes, enhancements, and/or policy improvements; and
• Support, challenge, and/or enrich the findings from the survey.

Methodology – Qualitative Research

Respondents from the survey were asked a question whether they would be interested in taking part in follow-up qualitative research. After conducting an analysis of the sample to ensure a mix of programs, regions, and to ensure inclusion of participants in both official languages, potential participants were contacted randomly and asked if they would like to be taken through the screening questionnaire to confirm their eligibility for an in-depth interview or focus group. the breakdown of participation is as follows:

The chart below provides a detailed description of the fieldwork.

Methodology – Qualitative Research Data Collection and Analysis

Data Collection 

With participants’ consent, all qualitative research sessions are both audio and video taped. Verbatim transcripts from each and every focus group and interview is created; however, names or personal identifying details are not captured and/or removed or redacted by the moderator in order to ensure participants’ privacy. 

Moderators also capture high level findings on each topic of their own observations – what the overall reaction was, any nuances, and any non-verbal cues on body language or tone. Because our transcripts are anonymous, they are able to comment on any variations by group or audience, if they have not been placed in separate groups – for example, moderators can provide a sense of different opinions by older vs. younger participants, or males vs. females, depending on the topic.

Data Analysis

We identify some basic elements to qualitative analysis: 

• Universal agreement where participants all agree, or there is agreement across different groups of stakeholders
• Consensus perspectives that reflect the view of most participants; areas of wide agreement without much counter point (Many, most, several)
• Conflicting or polarized perspectives where views are much more divided, or if there is a spectrum or variety of views (Some vs others)
• Minority perspectives, often expressed by one or two participants as a counterpoint to a consensus viewpoint, or if they have an individual take or example/story (a few, a couple, mention)
• Verbatim commentary, providing examples of what participants actually said during a discussion (direct unattributed quotes)
• External context, for this project it is the results of quantitative research that provided a foundation for the qualitative research conducted and the discussion questions posed.

Detailed Findings

Program Level Highlights

Enabling Accessibility Fund (EAF)

Program level-highlights

Click for larger view

Figure 10: Satisfaction with Service Channels

This vertical bar chart shows responses to a question about how satisfied or dissatisfied the applicant was with the service they received from the service channels they used throughout the application process and presents results for Year 1 and Year 2. Respondents were asked to provide ratings on a 5-pt scale and results were grouped by those who provided a rating of 4 or 5. Sample sizes vary by service channel, only those who used each channel during their experience were asked about it.

• Email Support from Program Officer: Year 2 80%. Year 1 83%.
• GCOS Web Portal: Year 2 78%. Year 1 77%.
• GoC Website: Year 2 72%. Year 1 76%.
• 1-800-OCanada: Year 2 67%.
• Phone Support from SC: Year 2 65%. Year 1 65%.
• Email Support from SC: Year 2 64%. Year 1 68%
• Mail: Year 2 47%. Year 1 100%.

Base: EAF applicants – Year 2 (n=207); Year 1 (n=56) Analysis was also conducted by program stream and no statistically significant differences were observed in survey responses due in part to small samples sizes among the Youth Innovation stream. Small Projects (n=192); Youth Innovation (n=15*) Note: Figures for ‘[PROGRAM] web portal’ is reported in Year 1 and compared with ‘GCOS web portal’ in Year 2.

Significantly higher / lower than total

Significantly higher/lower than Year 1

New Horizons for Seniors Program (NHSP)

Program level-highlights

Click for larger view

Figure 11: Satisfaction with Service Channels

This vertical bar chart shows responses to a question about how satisfied or dissatisfied the applicant was with the service they received from the service channels they used throughout the application process and presents results for Year 1 and Year 2. Respondents were asked to provide ratings on a 5-pt scale and results were grouped by those who provided a rating of 4 or 5. Sample sizes vary by service channel, only those who used each channel during their experience were asked about it.

• Email Support from Program Officer: Year 2 80%. Year 1 81%.
• Email Support from SC: Year 2 76%. Year 1 72%.
• SC Office: Year 2 75%. Year 1 71%.
• GoC Website: Year 2 74%. Year 1 70%
• GCOS Web Portal: Year 2 73%. Year 1 67%.
• Phone Support from SC: Year 2 68%. Year 1 68%.
• Mail: Year 2 56%. Year 1 56%.
• 1-800-OCanada: Year 2 50%. Year 1 68%.

Base: NHSP applicants – Year 2 (n=384); Year 1 (n=431)
Analysis was also conducted by program stream and no statistically significant differences were observed in survey responses.
Small grant (n=52); Community-based projects (n=332)
Note: Figures for ‘[PROGRAM] web portal’ is reported in Year 1 and compared with ‘GCOS web portal’ in Year 2.

Significantly higher / lower than total

Significantly higher/lower than Year 1

Canada Summer Jobs (CSJ)

Program level-highlights

Click for larger view

Figure 12: Satisfaction with Service Channels

This vertical bar chart shows responses to a question about how satisfied or dissatisfied the applicant was with the service they received from the service channels they used throughout the application process and presents results for Year 1 and Year 2. Respondents were asked to provide ratings on a 5-pt scale and results were grouped by those who provided a rating of 4 or 5. Sample sizes vary by service channel, only those who used each channel during their experience were asked about it.

• Email Support from Program Officer: Year 2 80%. Year 1 80%.
• GCOS Web Portal: Year 2 77%. Year 1 66%.
• GoC Website: Year 2 71%. Year 1 65%.
• Email Support from SC: Year 2 71%. Year 1 64%.
• Mail: Year 2 61%. Year 1 65%.
• SC Office: Year 2 60%. Year 1 67%.
• Phone Support from SC: Year 2 59%. Year 1 69%.
• 1-800-OCanada: Year 2 49%. Year 1 48%.

Base: CSJ applicants – Year 2 (n=865); Year 1 (n=942)
Note: Figures for ‘[PROGRAM] web portal’ is reported in Year 1 and compared with ‘GCOS web portal’ in Year 2.

Significantly higher / lower than total

Significantly higher/lower than Year 1

Youth Employment and Skills Strategy (YESS)+

Program level-highlights

Click for larger view

Figure 13: Satisfaction with Service Channels

This vertical bar chart shows responses to a question about how satisfied or dissatisfied the applicant was with the service they received from the service channels they used throughout the application process and presents results for Year 1 and Year 2. Respondents were asked to provide ratings on a 5-pt scale and results were grouped by those who provided a rating of 4 or 5. Sample sizes vary by service channel, only those who used each channel during their experience were asked about it.

• Email Support from Program Officer: Year 2 83%. Year 1 78%.
• GCOS Web Portal: Year 2 69%. Year 1 75%.
• GoC Website: Year 2 60%. Year 1 56%.
• Phone Support from SC: Year 2 59%. Year 1 50%.
• Email Support from SC: Year 2 54%. Year 1 60%.

Base: YESS applicants – Year 2 (n=152); Year 1 (n=25*) *small sample size
Note: Figures for ‘[PROGRAM] web portal’ is reported in Year 1 and compared with ‘GCOS web portal’ in Year 2.

Significantly higher / lower than total

Significantly higher/lower than Year 1

Union Training and Innovation Program (UT&IP)

Program level-highlights

Click for larger view

Figure 14: Satisfaction with Service Channels

This vertical bar chart shows responses to a question about how satisfied or dissatisfied the applicant was with the service they received from the service channels they used throughout the application process and presents results for Year 1 and Year 2. Respondents were asked to provide ratings on a 5-pt scale and results were grouped by those who provided a rating of 4 or 5. Sample sizes vary by service channel, only those who used each channel during their experience were asked about it.

• Email Support from Program Officer: Year 2 89%. Year 1 100%.
• GoC Website: Year 2 74%. Year 1 60%.
• Email Support from SC: Year 2 58%. Year 1 64%.
• GCOS Web Portal: Year 2 40%. Year 1 70%.

Base: UT&IP applicants – Year 2 (n=32); Year 1 (n=26*)
Analysis was also conducted by program stream and no statistically significant differences were observed in survey responses due in part to small samples sizes among applicants to either stream. Investments in Training Equipment (n=24*); Innovation and Apprenticeship (n=8**)
Note: Figures for ‘[PROGRAM] web portal’ is reported in Year 1 and compared with ‘GCOS web portal’ in Year 2.

*small sample size

Significantly higher / lower than total

Significantly higher/lower than Year 1

Early Lerning and Child Care Innovation (EL&CCI)

Program level-highlights

Click for larger view

Figure 15: Satisfaction with Service Channels

This vertical bar chart shows responses to a question about how satisfied or dissatisfied the applicant was with the service they received from the service channels they used throughout the application process and presents results for Year 1 and Year 2. Respondents were asked to provide ratings on a 5-pt scale and results were grouped by those who provided a rating of 4 or 5. Sample sizes vary by service channel, only those who used each channel during their experience were asked about it.

• Mail: Year 2 50%.
• GoC Website: Year 2 45%. Year 1 50%.
• GCOS Web Portal: Year 2 44%. Year 1 100%.
• Phone Support from SC: Year 2 33%.
• Email Support from Program Officer: Year 2 30%.
• Email Support from SC: Year 2 24%. Year 1 20%.

Base: EL&CCI applicants – Year 2 (n=65); Year 1 (n=5**)
Note: Figures for ‘[PROGRAM] web portal’ is reported in Year 1 and compared with ‘GCOS web portal’ in Year 2 .

** very small sample size

Significantly higher / lower than total

Significantly higher/lower than Year 1

Social Development Partnerships (SDPP)

Program level-highlights

Click for larger view

Figure 16: Satisfaction with Service Channels

This vertical bar chart shows responses to a question about how satisfied or dissatisfied the applicant was with the service they received from the service channels they used throughout the application process and presents results for Year 1 and Year 2. Respondents were asked to provide ratings on a 5-pt scale and results were grouped by those who provided a rating of 4 or 5. Sample sizes vary by service channel, only those who used each channel during their experience were asked about it.

• Email Support from Program Officer: Year 2 83%. Year 1 75%.
• Email Support from SC: Year 2 73%. Year 1 38%.
• GoC Website: Year 2 70%. Year 1 33%.
• GCOS Web Portal: Year 2 69%. Year 1 80%.
• Phone Support from SC: Year 2 61%. Year 1 100%.
• Mail: Year 2 50%.

Base: SDPP applicants – Year 2 (n=153); Year 1 (n=17*)
Analysis was also conducted by program stream and no statistically significant differences were observed in survey responses due in part to small samples sizes among applicants to two of the three streams. Supporting Black Canadian Communities (n=128); Supporting Black Canadian Communities – West Intermediaries (n=19*); Disability – Community Inclusion Initiative (n=6**)
Note: Figures for ‘[PROGRAM] web portal’ is reported in Year 1 and compared with ‘GCOS web portal’ in Year 2.

*small sample size

Significantly higher / lower than total

Significantly higher/lower than Year 1

Program Levels Highlights- New Programs Added in Year 2

Please note that the following programs were added in Year 2 and as such, there is no Year 1 data for comparison.

• Foreign credential recognition program (fcrp)
• Indigenous early learning and childcare (ielcc)
• Innovative work integrated learning initiative (iwili)
• Student work placement (swp)
• Sustainable development goals (sdg)

Foreign Credential Recognition Program (FCRP)

Program level-highlights

Click for larger view

Figure 17: Satisfaction with Service Channels

This vertical bar chart shows responses to a question about how satisfied or dissatisfied the applicant was with the service they received from the service channels they used throughout the application process. Respondents were asked to provide ratings on a 5-pt scale and results were grouped by those who provided a rating of 4 or 5. Sample sizes vary by service channel, only those who used each channel during their experience were asked about it.

• Phone Support from SC: 100%
• Email Support from Program Officer: 91%
• GoC Website: 71%
• GCOS Web Portal: 67%
• Email Support from SC: 100%

Base: FCRP applicants (n=20*)

*small sample size

Significantly higher / lower than total

Indigenous Early Learning and Childcare (IELCC)

Program level-highlights

Click for larger view

Figure 18: Satisfaction with Service Channels

This vertical bar chart shows responses to a question about how satisfied or dissatisfied the applicant was with the service they received from the service channels they used throughout the application process. Respondents were asked to provide ratings on a 5-pt scale and results were grouped by those who provided a rating of 4 or 5. Sample sizes vary by service channel, only those who used each channel during their experience were asked about it.

• Email Support from Program Officer: 75%
• Email Support from SC: 71%
• GoC Website: 60%

Base: IELCC applicants (n=8*)

** very small sample size

Significantly higher / lower than total

Innovative Work Integrated Learning Initiative (IWILI)

Program level-highlights

Click for larger view

Figure 19: Satisfaction with Service Channels

This vertical bar chart shows responses to a question about how satisfied or dissatisfied the applicant was with the service they received from the service channels they used throughout the application process. Respondents were asked to provide ratings on a 5-pt scale and results were grouped by those who provided a rating of 4 or 5. Sample sizes vary by service channel, only those who used each channel during their experience were asked about it.

• Phone Support from SC: 100%
• GCOS Web Portal: 67%
• Email Support from SC: 64%
• Email Support from Program Officer: 60%
• GoC Website: 50%

Base: IWILCI applicants (n=13*)

* small sample size

Significantly higher / lower than total

Student Work Placement (SWP)

Program level-highlights

Click for larger view

Figure 20: Satisfaction with Service Channels

This vertical bar chart shows responses to a question about how satisfied or dissatisfied the applicant was with the service they received from the service channels they used throughout the application process. Respondents were asked to provide ratings on a 5-pt scale and results were grouped by those who provided a rating of 4 or 5. Sample sizes vary by service channel, only those who used each channel during their experience were asked about it.

• Email Support from SC: 67%
• Email Support from Program Officer: 33%

Base: SWP applicants (n=4**)

* very small sample size

Sustainable Development Goals (SDG)

Program level-highlights

Click for larger view

Figure 21: Satisfaction with Service Channels

This vertical bar chart shows responses to a question about how satisfied or dissatisfied the applicant was with the service they received from the service channels they used throughout the application process. Respondents were asked to provide ratings on a 5-pt scale and results were grouped by those who provided a rating of 4 or 5. Sample sizes vary by service channel, only those who used each channel during their experience were asked about it.

• GCOS Web Portal: 56%
• GoC Website: 50%
• Email Support from Program Officer: 50%
• Phone Support from SC: 46%
• Email Support from SC: 43%

Base: SDG applicants (n=39*)

* small sample size

Significantly higher / lower than total

Service Channel Assessments

Satisfaction with Service Channels

• Satisfaction with service has increased compared to Year 1 among those using the GCOS web portal (+9 pts), Government of Canada website (+5 pts) and those who received email support from a Service Canada office (+5 pts). At eight in ten (79%), applicants remained most satisfied with email support from a program officer. Three-quarters (76%) were satisfied with the GCOS web portal and seven in ten for the Government of Canada website (71%) and email support form a Service Canada office (70%). Six in ten were satisfied with in-person service at a Service Canada office (62%), telephone support from a Service Canada office (61%) and mail (58%). Satisfaction remained lowest for the 1 800 O-Canada phone line (48%).

How satisfied or dissatisfied were you with the overall quality of service you received from each of the following?

Click for larger view

Figure 22: Satisfaction with Service Channels

This horizontal bar chart shows responses to a question about how satisfied or dissatisfied the applicant was with the overall quality of service provided by the service channels used during the applicant process and presents results for Year 1 and Year 2. Respondents were asked to provide ratings on a 5-pt scale. Sample sizes vary by service channel, only those who used each channel during their experience were asked about it.

• Email support from a Program Officer: Year 2 49% very satisfied, 30% somewhat satisfied, 13% neutral, 3% somewhat dissatisfied, 3% very dissatisfied, 2% Don’t know. 79% very/somewhat satisfied. A total of 627 respondents answered this question. Year 1 80% very/somewhat satisfied.
• GCOS Web portal: Year 2 42% very satisfied, 34% somewhat satisfied, 17% neutral, 4% somewhat dissatisfied, 2% very dissatisfied, 2% Don’t know. 76% very/somewhat satisfied. A total of 623 respondents answered this question. Year 1 67% very/somewhat satisfied.
• Government of Canada website: Year 2 31% very satisfied, 40% somewhat satisfied, 20% neutral, 4% somewhat dissatisfied, 2% very dissatisfied, 4% Don’t know. 71% very/somewhat satisfied. A total of 1365 respondents answered this question. Year 1 66% very/somewhat satisfied.
• Email support from a Service Canada office: Year 2 40% very satisfied, 30% somewhat satisfied, 14% neutral, 4% somewhat dissatisfied, 3% very dissatisfied, 10% Don’t know. 70% very/somewhat satisfied. A total of 1580 respondents answered this question. Year 1 65% very/somewhat satisfied.
• Service Canada office: Year 2 41% very satisfied, 21% somewhat satisfied, 12% neutral, 9% somewhat dissatisfied, 18% very dissatisfied, 0% Don’t know. 62% very/somewhat satisfied. A total of 29 respondents answered this question. Year 1 66% very/somewhat satisfied.
• Telephone support from a Service Canada office: Year 2 36% very satisfied, 24% somewhat satisfied, 19% neutral, 3% somewhat dissatisfied, 6% very dissatisfied, 12% Don’t know. 61% very/somewhat satisfied. A total of 286 respondents answered this question. Year 1 66% very/somewhat satisfied.
• Mail: Year 2 30% very satisfied, 28% somewhat satisfied, 14% neutral, 4% somewhat dissatisfied, 1% very dissatisfied, 22% Don’t know 58% very/somewhat satisfied. A total of 139 respondents answered this question. Year 1 63% very/somewhat satisfied.
• 1800 O-Canada phone line: Year 2 20% very satisfied, 29% somewhat satisfied, 20% neutral, 8% somewhat dissatisfied, 10% very dissatisfied, 14% Don’t know. 48% very/somewhat satisfied. A total of 72 respondents answered this question. Year 1 49% very/somewhat satisfied.

Q26. On a scale from 1 to 5, where 1 means very dissatisfied and 5 means very satisfied, how satisfied or dissatisfied were you with the overall quality of service you received from each of the following?
Base: Used channel at aware, apply or follow-up stage. Note: Figures for ‘[PROGRAM] web portal’ is reported in Year 1 and compared with ‘GCOS web portal’ in Year 2.

Note: values less than 3% not labelled

* small sample size

** very small sample size

Significantly higher/lower than Year 1

Satisfaction with Service Channels by Program

• Applicants to EL&CCI and SDG were less satisfied with nearly all service channels used during their experience, while applicants to YESS were less satisfied with the GoC website and email support from a Service Canada office. NHSP applicants were more likely to be satisfied with email support from a Service Canada office.
• Compared to Year 1, applicants to CSJ and SDPP were more likely to be satisfied with the GoC website and email support form a Service Canada office. CSJ applicants were also more likely to be satisfied with the GCOS web portal. Satisfaction with service channels for all other programs remained statistically consistent.

How satisfied or dissatisfied were you with the overall quality of service you received from each of the following?

Q26. On a scale from 1 to 5, where 1 means very dissatisfied and 5 means very satisfied, how satisfied or dissatisfied were you with the overall quality of service you received from each of the following?
Base: Used channel at aware, apply or follow-up stage

* small sample size

** very small sample size

Significantly higher/lower than Year 1

Number of Contacts with Service Channels

• Consistent with Year 1, there were significant differences in the number of contacts applicants had with Service Canada depending on the channel they used.  
• Applicants who went to a Service Canada office, called 1-800 O-Canada and to a lesser extent communicated with the Government of Canada by mail or called a Service Canada office directly were more likely to have been in contact with Service Canada only once during their experience. Those who used the GCOS web portal, went online to the Government of Canada website or emailed a program officer directly were much more likely to have used the channel 5 times or more.
• Compared to Year 1, applicants report using the GCOS web portal more times and having fewer contacts emailing a Service Canada office or calling 1-800 O-Canada

Thinking about your overall experience, how many times did you use each of the following?

Click for larger view

Figure 23: Number of Contacts with Service Channels

This horizontal bar chart shows responses to a question about how many times the applicant used each service channels during their experience and presents results for Year 1 and Year 2. Sample sizes vary by service channel, only those who used each channel during their experience were asked about it.

• Go to a Service Canada office: Year 2 40% one time, 3% two times, 8% three times, 3% four times, 0% five or more times, 47% Don’t know. Year 1 42% one time, 6% two times, 7% three times, 3% four times, 1% five or more times, 41% Don’t know.
• Communicate by mail with the Government of Canada: Year 2 31% one time, 7% two times, 6% three times, 2% four times, 8% five or more times, 47% Don’t know. Year 1 42% one time, 11% two times, 6% three times, 2% four times, 4% five or more times, 35% Don’t know.
• Call a Service Canada office directly: Year 2 31% one time, 12% two times, 10% three times, 3% four times, 10% five or more times, 34% Don’t know. Year 1 25% one time, 18% two times, 10% three times, 4% four times, 20% five or more times, 24% Don’t know.
• Call 1800 O-Canada: Year 2 40% one time, 16% two times, 6% three times, 3% four times, 5% five or more times, 31% Don’t know. Year 1 23% one time, 18% two times, 9% three times, 5% four times, 14% five or more times, 31% Don’t know.
• Email a Service Canada Office: Year 2 22% one time, 12% two times, 8% three times, 5% four times, 16% five or more times, 37% Don’t know. Year 1 19% one time, 13% two times, 11% three times, 6% four times, 22% five or more times, 30% Don’t know.
• Go online to the Government of Canada website: Year 2 14% one time, 15% two times, 9% three times, 5% four times, 27% five or more times, 29% Don’t know. Year 1 18% one time, 15% two times, 9% three times, 6% four times, 28% five or more times, 25% Don’t know.
• Go online to the program they applied for web portal: Year 2 13% one time, 12% two times, 13% three times, 7% four times, 44% five or more times, 16% Don’t know. Year 1 13% one time, 12% two times, 10% three times, 7% four times, 37% five or more times, 21% Don’t know.
• Email a Program Officer directly: Year 2 13% one time, 16% two times, 11% three times, 8% four times, 31% five or more times, 20% Don’t know. Year 1 9% one time, 22% two times, 16% three times, 9% four times, 37% five or more times, 7% Don’t know.

* small sample size

** very small sample size

Significantly higher/lower than Year 1

Q25. Thinking about your overall experience, how many times did you [IF MULTIPLE SOURCES ‘use each of the following’ IF ONLY ONE SOURCE ‘use the following’]? Please provide one response per item. If you are uncertain, please provide your best guess.
Base: Used channel at aware, apply or follow-up stage n=

Number of Contacts by Program

• Across all service channels, a greater proportion of applicants' report having been in contact with Service Canada 1 to 3 times and fewer 10+ times. Three in ten (28%, +13 pts) applicants were in contact with Service Canada 10 or more times, one in ten (13%, -2 pts) were in contact 7 to 9 times and two in ten (21%, -2 pts) 4 to 6 times or 1 to 3 times (19%, +7 pts).
• As observed in Year 1, satisfaction with the service experience declines by the number of times the client contacted Service Canada and continued to be lower among those who had 10 or more contacts through any channel during the client journey.
• Those applying to all programs but EAF, NHSP and CSJ and in particular YESS (55%, -17 pts), SDG (44%), and EL&CCI (38%, -26 pts) were more likely to have been in contact with Service Canada 10 or more times. Applicants to EAF were more likely to have been in contact 1 to 3 times.
• Compared to Year 1, applicants to CSJ were more likely to have been in contact with Service Canada 1 to 3 times (and fewer 10+ times), while applicants to EAF and all programs but EAF, NHSP and CSJ were less likely to report being in contact 10 or more times.

Thinking about your overall experience, how many times did you use each of the following?

Q25. Thinking about your overall experience, how many times did you [IF MULTIPLE SOURCES ‘use each of the following’ IF ONLY ONE SOURCE ‘use the following’]? Please provide one response per item. If you are uncertain, please provide your best guess.

* small sample size

** very small sample size

Significantly higher/lower than Year 1

Barriers and Issue Resolution

Encountered a Problem – % Yes

• Just over one in five (22%) applicants encountered a problem or issue during the application process, a decrease of 13 points from Year 1.
• Those applying to EL&CCI, SDG, SDPP and NHSP were more likely to say they have encountered a problem.
• Compared to Year 1, CSJ applicants experienced the same decline as observed for the total of all programs.

Thinking about your overall experience getting information and applying for [PROGRAM], did you experience any problems or issues during this process? – % Yes

Click for larger view

Figure 24: Encountered a Problem

This vertical bar chart shows responses to a question about whether the applicant experienced any problems or issues during the application process and presents results for Year 1 and Year 2

• Total: Year 2 22%. A total of 1942 respondents answered this question. Year 1 35%.
• EAF: Year 2 21% A total of 207 respondents answered this question. Year 1 23%.
• NHSP: Year 2 27%. A total of 384 respondents answered this question. Year 1 32%.
• CSJ: Year 2 20%. A total of 865 respondents answered this question. Year 1 35%.
• YESS: Year 2 25%. A total of 152 respondents answered this question. Year 1 20%.
• UT&IP: Year 2 9%. A total of 32 respondents answered this question. Year 1 19%.
• EL&CCI: Year 2 52%. A total of 65 respondents answered this question. Year 1 20%.
• SDPP: Year 2 33% A total of 153 respondents answered this question. Year 1 47%.
• FCRP: Year 2 25%. A total of 20 respondents answered this question.
• IELCC: Year 2 25%. A total of 8 respondents answered this question.
• IWLCI: Year 2 46%. A total of 13 respondents answered this question.
• SWP: Year 2 75%. A total of 8 respondents answered this question.
• SDG: Year 2 46%. A total of 39 respondents answered this question.
• ALL BUT EAF, NHSP, CSJ: Year 2 35%. A total of 486 respondents answered this question. Year 1 31%.

Q27. Thinking about your overall experience getting information and applying for [INSERT PROGRAM], did you experience any problems or issues during this process?
Base: All respondents n=1942

* small sample size

** very small sample size

Significantly higher / lower than total

Significantly higher/lower than Year 1

Explanation of Problem or Issue

• The most common problems or issues were that it took too long to receive a funding decision (34%), technical difficulties (27%), that it took too long to receive an update (23%), and that the application form was too long (21%).
• Compared to Year 1, fewer applicants reported experiencing most problems or issues, while a greater proportion said the process was complicated/time-consuming.

How would you describe the problem or issue you experienced?

Click for larger view

Figure 25: Explanation of Problem or Issue

This horizontal bar chart shows coded responses to an open-ended question about how the applicant would describe the problem or issue they experienced and presents results for Year 1 and Year 2. A total of 482 respondents from Year 2 who experienced problem or issue answered as follows:

• Took too long to receive a funding decision: Year 2 34%. Year 1 43%.
• Technical Difficulties. Year 2 27%.
• Took too long to receive an update on my application: Year 2 23%. Year 1 37%.
• Application form was too long: Year 2 21%.
• Online application portal was confusing: Year 2 30%.
• Website information was confusing: Year 2 19%. Year 1 15%.
• Application form was too complicated: Year 2 19%. Year 1 25%.
• Online account creation was confusing: Year 2 18%.
• I received different answers from different Program Officers: Year 2 16% Year 1 22%.
• Application requirements were difficult to understand: Year 2 13%. Year 1 16%.
• Government of Canada website information was confusing: Year 2 11%.
• Telephone lines were busy: Year 2 16%. Year 1 11%.
• Information on the program was difficult to understand: Year 2 11%. Year 1 16%.
• Staff were not knowledgeable / could not answer my questions: Year 2 10%Year 1 13%.
• Process was complicated/time-consuming: Year 2 5%. Year 1 1%.
• Poor communication/ lack of follow up/ long to receive response: Year 2 5%. Year 1 7%.

Note: Only responses of 3% or more for Year 2 are shown.
Q28. How would you describe the problem or issue you experienced?
Base: Experienced problem or issue (n=482)

Significantly higher/lower than Year 1

Explanation of Problem or Issue by Program

• Among those who experienced a problem or issue, EAF applicants were more likely to cite technical difficulties. Those applying to all programs but EAF, NHSP and CSJ were more likely to say that it took too long to receive their funding decision and an update on their application, application requirements were too difficult to understand and program information was difficult to understand. Those applying to EL&CCI were also more likely to say that staff were not knowledgeable and could not answer their questions, while NHSP were also more likely to say that the application requirements were too difficult to understand.

How would you describe the problem or issue you experienced?

Note: Only responses of 3% or more for Year 2 Totals are shown.
Q28. How would you describe the problem or issue you experienced?
Base: Experienced problem or issue

* small sample size

** very small sample size

Significantly higher / lower than total

Significantly higher/lower than Year 1

Ease of Issue Resolution

• Among those who experienced a problem or issue, three in ten (29%) felt it was easily resolved which is consistent with Year 1.
• Those having experienced a problem or issue applying for EL&CCI were less likely to say their problem was easily resolved.

Please rate the following statement: The problem or issue was easily resolved.

Click for larger view

Figure 26: Ease of Issue Resolution

This vertical bar chart shows responses to a question about the extent to which the applicant agrees or disagrees that the problem or issue was easily resolved and presents results for Year 1 and Year 2. Those who experienced problem or issue answered as follows:

• Total: Year 2: 10% strongly agree, 18% somewhat agree, 31% neutral, 18% somewhat disagree, 19% strongly disagree. A total of 482 respondents answered this question. Year 1: 10% strongly agree, 16% somewhat agree, 30% neutral, 25% somewhat disagree, 17% strongly disagree.
• EAF: Year 2 35% strongly/somewhat agree. A total of 43 respondents answered this question. Year 1 39% strongly/somewhat agree.
• NHSP: Year 2 29% strongly/somewhat agree. A total of 104 respondents answered this question. Year 1 25% strongly/somewhat agree.
• CSJ: Year 2 30% strongly/somewhat agree. A total of 175 respondents answered this question. Year 1 26% strongly/somewhat agree.
• YEES: Year 2 21% strongly/somewhat agree. A total of 38 respondents answered this question. Year 1 0% strongly/somewhat agree.
• • UT&IP: Year 2 0% strongly/somewhat agree. A total of 3 respondents answered this question. Year 1 0% strongly/somewhat agree
• EL & CCI: Year 2 3% strongly/somewhat agree. A total of 34 respondent answered this question. Year 1 100% strongly/somewhat agree.
• SDPP: Year 2 24% strongly/somewhat agree. A total of 51 respondents answered this question. Year 1 13% strongly/somewhat agree.
• FCRP: Year 2 40% strongly/somewhat agree. A total of 5 respondents answered this question.
• IELCC: Year 2 50% strongly/somewhat agree. A total of 2 respondents answered this question.
• IWLCI: Year 2 33% strongly/somewhat agree. A total of 6 respondents answered this question.
• • SWP: Year 2 17% strongly/somewhat agree. A total of 3 respondents answered this question.
• SDG: Year 2 17% strongly/somewhat agree. A total of 18 respondents answered this question.
• ALL BUT EAF, NHSP, CSJ: Year 2 17% strongly/somewhat agree. A total of 160 respondents answered this question. Year 1 16% strongly/somewhat agree.

Q29. On a scale from 1 to 5, where 1 is strongly disagree and 5 is strongly agree, please rate the following statement: The problem or issue was easily resolved.
Base: Experienced problem or issue (n=482)

* small sample size

** very small sample size

Significantly higher / lower than total

Significantly higher/lower than Year 1

Drivers of Satisfaction

Drivers of satisfaction

The primary driver of satisfaction in the service experience is the helpfulness of Service Canada and 1-800 O-Canada phone representatives, followed by the amount of time it took from start to finish was reasonable, the ease of getting help when needed and clarity of the process. 

• Other prominent drivers include the ease of finding information about the program, finding out what information is needed when applying and confidence in issue resolution.

When comparing the drivers of satisfaction between Year 1 and Year 2, the helpfulness of Service Canada phone representatives has increased in importance and become the primary driver of satisfaction, while timeliness of service has decreased in importance but remains the second most prominent driver overall.

• Other aspects of service which have increased in importance in driving satisfaction include the ease of getting help when needed and the ease of finding out what information needs to be provided when applying and putting together the information needed to apply. 
• Aspects of service that have decreased in importance include needing to explain your situation only once, overall ease of applying and ease of completing the project timelines

Overall, the greatest opportunities to improve the service experience are improving the helpfulness of Service Canada phone representatives and the timeliness of service. 

• In order to summarize what potential changes could result in an increase in overall satisfaction, the service attributes that most strongly drive satisfaction for Service Canada clients are determined and compared to Service Canada’s performance against these attributes.
• The resulting analysis found that common areas for potential improvement include improving the helpfulness of Service Canada phone reps and the timeliness of service. The most prominent secondary areas for improvement include the ease of getting assistance, clarity of process and confidence in issue resolution. 
• The provision of service in either Official Language, the ease gained from completing steps online, ease of determining eligibility and ease of finding information about the program are relative strengths for the organization and areas that should be maintained. 
• In Year 2, the primary driver of satisfaction in the service experience is the helpfulness of Service Canada phone representatives, followed by the amount of time it took from start to finish was reasonable, the ease of getting help when needed and clarity of process.
• The strength of the drivers’ analysis is strong and has an R2 of 0.63 (consistent with Year 1, 0.63).

Click for larger view

Figure 27: Drivers of Satisfaction

This horizontal bar chart shows results of a regression analysis that was conducted to identify the primary service attributes driving overall satisfaction with the service experience. Results were reported by impact score per service attribute as follows:

• Service Canada phone representatives were helpful 0.49
• The amount of time it took was reasonable 0.26
• It was easy for me to get help when I need it 0.21
• Throughout the process it was clear what would happen next and when it would happen 0.17
• Find general information about [program]: 0.14
• Find out what information you need to provide when applying for [program] 0.13
• I was confident that any issues or problems would have been easily resolved 0.12
• Being able to complete steps online made the process easier for me 0.11
• Putting together the information you needed to apply for [program] 0.10
• After you submitted your application to [program], did your organization receive approval for funding? 0.10
• Determine if your organization is eligible for [program] funding 0.10
• Completing the narrative questions (i.e. funding objectives, description of project, scope of project, etc.) 0.09
• I was provided with service in my choice of English or French 0.08
• Determine the steps to apply for funding 0.07
• Understand the information about [program] 0.06
• The application took a reasonable amount of time to complete 0.06
• Overall, it was easy for me to apply [program] 0.06
• Determine the amount of time each phase of the application process is anticipated to take 0.06
• It was easy to access service in a language I could speak and understand well 0.05
• Completing the budget document 0.05
• It was clear what to do if I had a problem or question 0.04
• Meeting the requirements of the application 0.04
• I was able to move smoothly through all of the steps related to the [program] application 0.03
• I received consistent information 0.03
• Determine when the application period for [program] takes place 0.02
• I was confident that my personal information was protected 0.02
• Completing the project timeline 0.02
• Understanding the requirements about the [program] 0.00
• I needed to explain my situation only once 0.00

R2 = 0.63

Change in Importance of Drivers of Satisfaction- Year 1 vs. Year 2

• When comparing the drivers of satisfaction between Year 1 and Year 2, there has been some notable changes in the importance of certain service attributes. The helpfulness of Service Canada phone representatives has increased in importance and become the primary driver of satisfaction, while timeliness of service has decreased in importance but remains the second most prominent driver overall.
• Other aspects of service which have increased in importance in driving satisfaction include the ease of getting help when needed and the ease of finding out what information needs to be provided when applying and putting together the information needed to apply. Aspects of service that have decreased in importance include needing to explain your situation only once, overall ease of applying and ease of completing the project timelines.

Priority Matrix – Overview

READER’S NOTE: This slide was intended to assist the reader in interpreting data shown in a priority matrix. A priority matrix has been used to identify priority improvement areas with respect to service interactions with applicants.

A priority matrix allows for decision makers to identify priorities for improvement by comparing how well applicants feel you have performed in an area with how much impact that area has on applicants’ overall satisfaction. It helps to answer the question ‘what can we do to improve satisfaction’. Each driver or component will fall into one of the quadrants explained below, depending on its impact on overall satisfaction and its performance score (provided by survey respondents).

Click for larger view

Overall Priority Matrix – Impact vs. Performance

• The greatest opportunities to improve the service experience are improving the helpfulness of Service Canada phone representatives and the timeliness of service. The most prominent secondary areas for improvement include the ease of getting assistance, clarity of process and confidence in issue resolution.
• The provision of service in either Official Language, the ease gained from completing steps online, ease of determining eligibility and ease of finding information about the program are relative strengths for the organization and areas that should be maintained.

Click for larger view

*factors with standardized coefficients below 0.05 were excluded

Pre-Application

Information Gathering about the Program 

Channel Use Pre-Application to Learn About the Program

• At nearly six in ten (57%, +6 pts), applicants were most likely to have received an email from the GoC, ESDC, or the program they applied to during the aware stage, followed by half (48%) who went to the Government of Canada website for the program. One quarter (25%, -35 pts) went to the general Government of Canada website, or talked to peers/community network (23%, -6 pts).
• Compared to Year 1, a greater proportion received an email from the GoC, ESDC, or the program, while fewer talked to peers/community network or their MP.

Which of the following did you use to find out about [PROGRAM] before you applied?

Click for larger view

Figure 28: Channel use pre-application- to learn about the program

This horizontal bar chart shows responses to a question about which channels the applicant used to find out about the program before they applied and presents results for Year 1 and Year 2. All the 1942 respondents from Year 2 answered as follows:

• Received an email from the funding program directly: Year 2 57%. Year 1 51%.
• Went online to the Government of Canada website for the [program]: Year 2 48%.
• Went online to the Government of Canada website: Year 2 25%. Year 1 60%.
• Talked to my peers / community network: Year 2 23%. Year 1 29%.
• Talked to my local Member of Parliament (MP): Year 2 15%. Year 1 18%.
• Participated in Government of Canada info session / webinar. Year 2 11%. Year 1 10%.
• Emailed a Program Officer for [program] directly: Year 2 11%. Year 10%.
• Went online to websites for other levels of government: Year 2 5%. Year 1 8%.
• Used social media to get information: Year 2 5%. Year 1 5%.
• Emailed a Service Canada office: Year 2 4%. Year 1 5%.
• Called a Service Canada office directly: Year 2 3%. Year 1 5%.
• Went online to other websites: Year 2 3%. Year 1 4%.
• Called 1 800 O Canada phone line: Year 2 2% Year 1 2%.
• Went to a Service Canada office: Year 2 1% Year 1 1%.
• NONE OF THESE: Year 2 4%. Year 1 4%.

Q2. Which of the following did you use to find out about [INSERT PROGRAM] before you applied? Consider all the methods you used to learn about the program before filling out the application. Please select all that apply.
Base: All respondents
Note: Year 1 wave had the following answer choice wording that did not mention the specific program applied to: “Emailed a program officer directly” and “Received an email from the funding program directly”.

Significantly higher/lower than Year 1

Channel Use Pre-Application to Learn About the Program by Program

• CSJ applicants were more likely to have received an email from the GoC, ESDC, or program directly (+10 pts), while those applying to EAF, SDPP, and SDG were less likely. Applicants to all programs but EAF, NHSP and CSJ were more likely to have gone to the general GoC website, talked to peers/ community network, participated in a GoC website, emailed a program officer directly, used social media or went online to other websites. YESS applicants were also more likely to go to the Government of Canada website for the program. Compared to Year 1, CSJ and NHSP applicants were more likely to receive an email from the GoC.

Which of the following did you use to find out about [PROGRAM] before you applied?

Q2. Which of the following did you use to find out about [INSERT PROGRAM] before you applied? Consider all the methods you used to learn about the program before filling out the application. Please select all that apply.
Base: All respondents
Note: Year 1 had the following answer choice wording that did not mention the specific program applied to: “Emailed a program officer directly” and “Received an email from the funding program directly”.

* small sample size

** very small sample size

Significantly higher / lower than total

Significantly higher/lower than Year 1

Ease of Use of Government of Canada Website

• Of those who used the Government of Canada website at the aware stage, the vast majority felt nearly all aspects of learning about the program were easy. Applicants were most likely to feel it was easy to determine if their organization was eligible for funding (84%), determining when the application period takes place (83%), and finding general information about the program they applied for (82%). Ratings were lowest for the determining the amount of time each phase of the application process is anticipated to take (58%).
• Compared to Year 1, applicants were more likely to feel it was easy to determine the steps to apply and understand information about the program.

How difficult or easy was it to find the following information about [PROGRAM] on the Government of Canada website?

Click for larger view

Figure 29: Ease of Use of Government of Canada website

This horizontal bar chart shows responses to a question about how difficult or easy it was for the applicant to find different types of information about the program they applied for on the Government of Canada website and presents results for Year 1 and Year 2. Respondents were asked to provide ratings on a 5-pt scale. All 1,071 respondents from Year 2 who used government of Canada website responded as follows:

• Determine if your organization is eligible for [program] funding: Year 2 55% very easy, 30% somewhat easy, 12% neutral, 3% somewhat difficult, 1% very difficult, 0% Don’t know. 84% easy. Year 1 83% easy.
• Determine when the application period for [program] takes place: Year 2 53% very easy, 30% somewhat easy, 13% neutral, 4% somewhat difficult, 1% very difficult, 0% Don’t know. 83% easy. Year 1 n/a easy.
• Find general information about the [program]: Year 2 46% very easy, 36% somewhat easy, 14% neutral, 3% somewhat difficult, 1% very difficult, 0% Don’t know. 82% easy. Year 1 82% easy.
• Determine the steps to apply for funding: Year 2 42% very easy, 39% somewhat easy, 14% neutral, 4% somewhat difficult, 1% very difficult, 0% Don’t know. 81% easy. Year 1 78% easy.
• Understand the information about the [program]: Year 2 41% very easy, 39% somewhat easy, 15% neutral, 5% somewhat difficult, 1% very difficult, 0% Don’t know. 80% easy. Year 1 76% easy.
• Find out what information they need to provide when applying for the [program]: Year 2 41% very easy, 37% somewhat easy, 16% neutral, 4% somewhat difficult, 1% very difficult, 0% Don’t know. 79% easy. Year 1 78% easy.
• Determine the amount of time each phase of the application process is anticipated to take: Year 2 27% very easy, 31% somewhat easy, 27% neutral, 9% somewhat difficult, 5% very difficult, 2% Don’t know. 58% easy. Year 1 n/a.

Q5. On a scale from 1 to 5, where 1 is very difficult and 5 is very easy, how difficult or easy was it to find the following information about [INSERT PROGRAM] on the Government of Canada website? Select one response per item.
Base: Used Government of Canada website

Note: values less than 3% not labelled

* small sample size

** very small sample size

Significantly higher / lower than total

Significantly higher/lower than Year 1

Ease of Use of Government of Canada Website by Program

• Of those who used the GoC website, applicants to all programs but EAF, NHSP, and CSJ were less likely think it was easy to find all types of information related to their program. EL&CCI, SDPP and SDG applicants, in particular, were less likely to feel it easy to find all types of information.
• Compared to Year 1, CSJ applicants were more likely to feel it was easy to determine the steps to apply and UT&IP applicants were more likely to feel it was easy to determine if they were eligible for funding.

How difficult or easy was it to find the following information about [PROGRAM] on the Government of Canada website?

Q5. On a scale from 1 to 5, where 1 is very difficult and 5 is very easy, how difficult or easy was it to find the following information about [INSERT PROGRAM] on the Government of Canada website? Select one response per item.

* small sample size

** very small sample size

Significantly higher / lower than total

Significantly higher/lower than Year 1

Application Process

Applying for Funding

Channel Use for Application Preparation

• Applicants were most likely to use the Government of Canada website to prepare and complete their application (43%), though this represents a six-point decrease from Year 1. Approximately a quarter (24%) say they talked to their peers/community network (-2 pts) or emailed a program officer directly (-1 pt).
• Two in ten (21%, -2 pts) applicants say they did none of the things mentioned.

To prepare and complete your application (up until when you submitted) did you consult with any of the following?

Click for larger view

Figure 30: Channel Use for Application Preparation

This horizontal bar chart shows responses to a question about which channels the applicant used to prepare and complete their application (up until when they submitted) and presents results for Year 1 and Year 2. All 1942 respondents from Year 2 answered as follows:

• Went online to the Government of Canada website: Year 2 43%.Year 1 49%.
• Talked to my peers / community network: Year 2 24%. Year 1 26%.
• Emailed a Program Officer directly: Year 2 24%. Year 1 23%.
• Participated in a Government of Canada information session or webinar: Year 2 17%Year 1 11%.
• Talked to my local Member of Parliament (MP): Year 2 11%.Year 1 15%.
• Called a Service Canada office directly: Year 2 8%. Year 1 11%.
• Emailed a Service Canada office: Year 2 5%. Year 1 11%.
• Called 1800 O Canada phone line: Year 2 3%. Year 1 4%.
• Used social media to get information: Year 2 3%. Year 1 3%.
• Worked with a private consultant: Year 2 2%. Year 1 n/a.
• Went to a Service Canada office : Year 2 0%. Year 1 1%.
• NONE OF THESE: Year 2 21%. Year 1 19%.

Q6. To prepare and complete your application (up until when you submitted) did you consult with any of the following? Please select all that apply.
Base: All respondents
Note: Year 1 wave had the following answer choice wording that did not mention the specific program applied to: “Emailed a program officer directly”. 

Significantly higher/lower than Year 1

Channel Use for Application Preparation by Program

• Applicants to NHSP as well as all programs but EAF, NHSP and CSJ were more likely to have talked to peers, emailed a program officer directly, or participated in a GoC info session/webinar and were less likely to have visited the GoC website. Applicants to all programs but EAF, NHSP and CSJ were also less likely to have talked to their local MP. Applicants to EAF were more likely to emailed a program officer directly or went to other websites and less likely to have participated in a webinar.
• Compared to Year 1, applicants to CSJ were more likely to have participated in a webinar and less likely to have talked to peers, their local MP or call or email a Service Canada office. Applicants to EAF were more likely to have emailed a program officer directly and less likely to have talked to their local MP or emailed a Service Canada office, while applicants to NHSP as well as all programs but EAF, NHSP and CSJ were less likely to have visited the GoC website or participated in a webinar.

Which of the following did you use to find out about [PROGRAM] before you applied?

Q6. To prepare and complete your application (up until when you submitted) did you consult with any of the following? Please select all that apply.
Note: Year 1 wave had the following answer choice wording that did not mention the specific program applied to: “Emailed a program officer directly”.

* small sample size

** very small sample size

Significantly higher / lower than total

Significantly higher/lower than Year 1

Ease of Application Process

• Improvement has been made on most aspects of the ease of completing the application compared to Year 1. Applicants were most likely to feel it was easy to meet the requirements of the application (80%, +3 pts), understand the requirements of the application (76%, +3 pts), complete the project timeline (75%, unchanged), and put together the information needed to apply (74%, +5 pts). Comparatively to the ease dimensions noted above, ratings were lower for completing the narrative questions (70%, +6 pts) and completing the budget document (67%, unchanged).

How would you rate the following elements of the application for [PROGRAM]?

Click for larger view

Figure 31: Ease of Application Process

This horizontal bar chart shows responses to a question about how difficult or easy the applicant found different elements of the application process in Year 2. Respondents were asked to provide ratings on a 5-pt scale. All 1942 respondents from Year 2 answered as follows:

• Meeting the requirements of the application: Year 2 37% very easy, 43% somewhat easy, 15% neutral, 3% somewhat difficult, 2% very difficult, 1% Don’t know. 80% easy. Year 1 77% easy.
• Understanding the requirements of the application: Year 2 36% very easy, 40% somewhat easy, 17% neutral, 5% somewhat difficult, 2% very difficult, 0% Don’t know. 76% easy. Year 1 73% easy.
• Completing the project timeline: Year 2 35% very easy, 40% somewhat easy, 18% neutral, 4% somewhat difficult, 2% very difficult, 2% Don’t know. 75% easy. Year 1 75% easy.
• Putting together the information they needed to apply for the program: Year 2 7431% very easy, 44% somewhat easy, 19% neutral, 5% somewhat difficult, 1% very difficult, 0% Don’t know. % easy. Year 1 69% easy.
• Completing the narrative questions: Year 2 28% very easy, 42% somewhat easy, 22% neutral, 6% somewhat difficult, 2% very difficult, 0% Don’t know. 70% easy. Year 1 64% easy.
• Completing the budget document: Year 2 28% very easy, 40% somewhat easy, 22% neutral, 6% somewhat difficult, 3% very difficult, 2% Don’t know. 67% easy. Year 1 67% easy.

Q7. On a scale of 1 to 5, where 1 is very difficult and 5 is very easy, how would you rate the following elements of the application for [INSERT PROGRAM]? Select one response per item.
Base: All respondents

Note: values less than 3% not labelled

* small sample size

** very small sample size

Significantly higher / lower than total

Significantly higher/lower than Year 1

Ease of Application Process by Program

• Applicants to all programs but EAF, NHSP, and CSJ experienced more difficulty with all elements of the application process. EAF applicants had more difficulty with nearly all aspects of the process except for completing the narrative questions, while NHSP applicants provided lower ratings for the ease of meeting the requirements of the application and putting together the information they needed to apply. Those applying to CSJ were more likely to find it easy to complete the project timeline and put together the information needed to apply.
• Compared to Year 1, CSJ applicants were more likely to find it easy to understand the requirements of the application, put together the information they needed to apply and complete the narrative questions (along with NHSP applicants). NHSP applicants were also more likely to say it was easy to meet the requirements of the application). EAF applicants were less likely to feel it was easy to complete the project timeline.

How would you rate the following elements of the application for [PROGRAM]?