EXECUTIVE SUMMARY - Evaluating Various Components of EAS Client Services 2015 HC-POR-14-08

POR Registration Number: POR 048-14
PWGSC Contract number: HT372-146021/001/CY
Contract award date:2015-03-12
Delivery date: March 2015

Prepared for:
Health Canada

Ce rapport est aussi disponible en français.

For more information on this report, please e-mail: por-rop@hc-sc.gc.ca.

Prepared by:
Elemental Data Collection
170 Laurier Avenue West, Suite 100
Ottawa ON K1P 5V5

Table of contents

• Introduction
  • Background
  • Research Objectives
• Methodological Summary
  • Survey Methodology
  • Call Disposition
• Use of the Results
• Expenditures
• Appendix - Research Instruments

Introduction

Background

Employee Assistance Services (EAS) is a full service Employee Assistance Program (EAP) provider that offers confidential, professional, bilingual counselling services on a cost-recovery basis. EAS offers both preventive and remedial solutions to employers, employees, and family members who have concerns that could affect their personal well-being and/or work performance.

In order to be in full compliance with the Employee Assistance Society of North America (EASNA) accreditation standards, and respecting the fundamental need for a bias-free evaluation of the professional and confidential services provided by EAS, a specific number of audits must be completed to capture valuable quality assurance information from users of EAS' services.

This specific contract will allow EAS to meet this significant requirement and provide valuable information to its customer organizations and to further evaluate its impact in supporting clients in need of assistance.

Research Objectives

The goal of this research is to collect detailed employee feedback about the quality of services received from EAP such as timeliness of service delivery, accessibility of service, effectiveness, level of well-being pre and post EAP, work productivity, etc. Because it is administered only once per year, it evaluates a broader range of areas related to the services provided. Overall, it allows employees and/or their family members to voice their level of satisfaction with the help received from EAP. This information is also used to provide feedback to customer organizations in the annual statistical report. The contract for this project is for data collection only. No design or analysis is provided by the consultant.

As such, Health Canada requires the use of public opinion research (traditional telephone interviews) to evaluate the services EAS provides to its clients. Specifically, this will involve the completion of approximately 750 interviews. The data will be provided in Excel format along with frequencies and cross-tabulation analysis, with the first set of results being delivered by March 31, 2015.

The highest level of respect and flexibility will be given to former clients and their willingness to answer questions related to the services provided by EAS.

Methodological Summary

Survey Methodology

A quantitative research approach was used to obtain a statistically valid picture of the knowledge, opinions, perceptions and attitudes of former EAS clients. This research was conducted via a telephone survey. Below is a detailed description of the methodology.

• 750 telephone interviews were conducted randomly from a list of 3,627 former clients of EAS. The sample source used was provided by EAS and respondents were randomly selected from this list to be contacted.
• To ensure proper regional representation of the sample, soft quotas were implemented by province/region in proportion to the sample we received from EAS. The results can be considered accurate to within ± 4.4%, 19 times out of 20.
• Each sample record was called up to eight times in attempt to reach the target respondent, with the average number of call attempts being 4 per record.
• Calling was conducted weekdays from 9:00 am to 9:00 p.m. (local time), and on weekends from 11:00 am to 9:00 p.m. (local time).
• Interviewing was conducted in both English and French.
• The average interview length was 17.8 minutes, while individual interviews ranged in length from 9 to 33 minutes.
• Sponsorship of the study was revealed during the introduction of the interview as Employee Assistance Program (EAP).
• The survey was registered with the Marketing Research and Intelligence Association's (MRIA) Research Registration System.
• The response rate¹ for the survey was 32%.
• The fieldwork for this study was conducted from March 13^th to March 25^th, 2015.

The research was conducted in accordance with the standards set out by the MRIA, as well as applicable federal legislation (PIPEDA).

The response rate refers to the number of people that participated in the survey compared to the actual number sampled from the target population. The response rate was calculated using the MRIA's Empirical Method of Response Rate Calculation.

Call Disposition

The following table presents the call disposition information, including the response rate using the standard MRIA Empirical Method of Response Rate Calculation²:

Use of the Results

The results of this research will be used by Health Canada to: 1) comply with the requirement for accreditation by COA and to demonstrate best practice in the area of quality assurance, as EAS is mandated to develop measures to assess the various components of the services provided to clients, as well as to assess the impact of the counselling services provided to clients in need of psychosocial support; 2) allow EAS, which is delivered by Health Canada to all employees of the federal government in need, to meet this significant requirement and provide valuable information to its customer organizations and to further evaluate its impact in supporting clients in need of support; and 3) continue to meet its intended objectives as there are no foreseen identifiable risks associated with information gathering and dissemination. Failure to get feedback on client satisfaction could lead to a federal approach that does not resonate well with creating a healthy workplace that supports mental health.

Expenditures

The total expenditures for the research were $19,972.75 (HST included).

The response rate [R=R/(U+IS+R)] is calculated as the number of responding units [R] divided by the number of unresolved [U] numbers plus in-scope [IS] non-responding households and individuals plus responding units [R].

Appendix - Research Instruments

ENGLISH QUESTIONNAIRE:

EMPLOYEE ASSISTANCE SERVICES BUREAU - CLIENT TELEPHONE SURVEY

Before we start, I’d like to thank you for agreeing to participate in this confidential survey. The information you share with us will help improve our services without identifying participants in any way.

You will be asked to respond to a series of statements or questions that fall under 6 sections. Your honest opinions are what interest us most. Please note that EAP as a generic name includes: Employees from the Employee Assistance Program, Members from the Canadian Forces and Retirees from Veteran Affairs Canada and their immediate family members.

Please let me know if you’re having any trouble understanding me or if you’d like me to repeat a question. Do you have any questions before we start?

I. YOUR PRIMARY REASON FOR USING EMPLOYEE ASSISTANCE PROGRAM SERVICES

I will list a number of reasons employees might have for consulting their EAP. I’ll then ask you to indicate which one of the following best describes your primary reason for contacting your EAP:

• a) To assist with family-related problems (such as difficulties related to the couple, children, conflict, parenting).
• b) To assist in dealing with psychological issues (such as depression or anxiety).
• c) To assist in dealing with physical health issues.
• d) To assist in dealing with substance abuse issues (such as alcohol or drugs).
• e) To assist with work-related problems (such as stress, frustration, workload, work performance, organizational change).
• f) To assist in dealing with career decisions (such as re-orientation and redirection).
• g) To assist in dealing with workplace discrimination (such as age, gender, race).
• h) To assist in dealing with workplace conflict (such as with a supervisor, employee or peer).
• i) To assist in dealing with workplace harassment (such as sexual harassment, abuse of authority).
• j) To assist with gambling-related problems.
• k) To assist with financial problems (such as budget, bankruptcy).
• l) To assist with legal problems (for example child custody, divorce).
• m) Others (please elaborate)

_____________________________________________________________________________________________________________________________________________

II. YOUR LEVEL OF SATISFACTION WITH EAP SERVICES.

In this set of questions, I will ask your opinion regarding the services you received. For each question, please choose the response that best describes how you feel between: very satisfied, satisfied, dissatisfied, very dissatisfied.

1. In general, how satisfied were you with the counsellor who initially responded to your telephone call (to 1-800-268-7708)?

• 1) Very satisfied
• 2) Satisfied
• 3) Dissatisfied
• 4) Very dissatisfied

2. How satisfied were you with how promptly the counsellor to whom you were referred scheduled your first appointment?

• 1) Very satisfied
• 2) Satisfied
• 3) Dissatisfied
• 4) Very dissatisfied

3. How satisfied were you with the counsellor’s ability to understand your concerns?

• 1) Very satisfied
• 2) Satisfied
• 3) Dissatisfied
• 4) Very dissatisfied

4. Did you need to be referred externally? (If not, skip this question.) If so, how satisfied were you with your EAP counsellor’s ability to refer you to additional services outside of the EAP?

• 1) Very satisfied
• 2) Satisfied
• 3) Dissatisfied
• 4) Very dissatisfied
• 5) Not applicable

5. Overall, how satisfied are you with the quality of services you’ve received?

• 1) Very satisfied
• 2) Satisfied
• 3) Dissatisfied
• 4) Very dissatisfied

6. How satisfied are you with how our EAP has met your needs?

• 1) Very satisfied
• 2) Satisfied
• 3) Dissatisfied
• 4) Very dissatisfied

7. How satisfied are you with the help you received in dealing with your concerns?

• 1) Very satisfied
• 2) Satisfied
• 3) Dissatisfied
• 4) Very dissatisfied

For the next few questions, responses are limited to YES or NO

8. Did your EAP counsellor:

• a) provide you with general information regarding the EAP services ?
  • Yes
  • No
  • Uncertain
• b) provide you with information regarding the confidentiality of our EAP services ?
  • Yes
  • No
  • Uncertain
• c) inform you about what to do with any problems or concerns you have regarding the EAP services you received ?
  • Yes
  • No
  • Uncertain
• d) provide you with a voluntary confidential survey to assess the EAP services with pre-paid postage envelope)
  • Yes
  • No
  • Uncertain
• e) provide you with a telephone follow-up within 2 weeks of your last counselling session?
  • Yes
  • No
  • Uncertain
  • Not applicable

If 'Not Applicable' was chosen, please ask the client to choose from the following two answers:

• I completed my sessions and it has not yet been two weeks.
• I declined a follow-up to be conducted.

9. If you were to seek help again, would you contact our EAP?

• Yes
• No
• Uncertain

If No, please elaborate:

_____________________________________________________________________________________________________________________________________________

10. If you thought someone else was in need of EAP services, would you recommend our EAP?

• Yes
• No
• Uncertain

If No or Uncertain, please elaborate:

_____________________________________________________________________________________________________________________________________________

III. YOUR REASONS FOR ENDING EAP COUNSELLING

What is the main reason your counselling sessions ended?

I will list a series of reasons, please indicate the main reason for ending your EAP counselling.

• a) You accomplished what you wanted to in counselling.
• b) You reached the limit of sessions available to you.
• c) You could no longer fit counselling into your work schedule.
• d) Your counsellor referred you to other services that would address your concerns more adequately (such as mental health counsellors, community services).
• e) You weren't confident in your counsellor’s ability to help you.
• f) You were uncomfortable talking about your issue with your counsellor.
• g) You felt that your counselling was going nowhere.
• h) You decided to go elsewhere for counselling services.
• i) Other (please elaborate)

_____________________________________________________________________________________________________________________________________________

IV. THE IMPACT & USEFULNESS OF EAP SERVICES RECEIVED

In Question #1, your answer was ___________.

Did the EAP services you received make a difference in that aspect of your life?

• Yes
• No

Sometimes counselling has an impact on other areas of our lives than the issue we were initially addressing. Did you notice an impact on other areas of your life?

• Yes
• No

If No, go to next section.

If Yes, in which other area of your life did you notice the most impact?

• a) To assist with family-related problems (such as difficulties related to the couple, children, conflict, parenting).
• b) To assist in dealing with psychological issues (such as depression or anxiety).
• c) To assist in dealing with physical health issues.
• d) To assist in dealing with substance abuse issues (such as alcohol or drugs).
• e) To assist with work-related problems (such as stress, frustration, workload, work performance, organizational change).
• f) To assist in dealing with career decisions (such as re-orientation and redirection).
• g) To assist in dealing with workplace discrimination (such as age, gender, race).
• h) To assist in dealing with workplace conflict (such as with a supervisor, employee or peer).
• i) To assist in dealing with workplace harassment (such as sexual harassment, abuse of authority).
• j) To assist with gambling-related problems.
• k) To assist with financial problems (such as budget, bankruptcy).
• l) To assist with legal problems (for example child custody, divorce).
• m) Others (please elaborate)

V. YOUR WELL-BEING IN TERMS OF WORK PRODUCTIVITY, RELATIONSHIPS, ABSENTEEISM AND HEALTH STATUS

In this section each of the questions has two parts. The first part relates to difficulties you were experiencing four weeks (30 days) prior to seeking EAP counselling sessions. The second part relates to difficulties 30 days after the last EAP session. For each question, please choose the response that best describes how you feel between the choices provided.

1.a Presenteeism - In the 30 days prior to contacting EAP, how often did emotional problems (such as feeling depressed or anxious) limit your concentration, performance and the kind or amount of work you could do?

• 1) none of the time
• 2) a little of the time
• 3) some of the time
• 4) most of the time
• 5) all of the time

1.b And 30 days after the last session?

• 1) none of the time
• 2) a little of the time
• 3) some of the time
• 4) most of the time
• 5) all of the time

2.a Productivity as Affected by Physical Health - In the 30 days prior to contacting EAP, how much difficulty did you have doing your work or other regular daily activities as a result of your physical health?

• 1) none at all
• 2) a little bit
• 3) moderately
• 4) quite a bit
• 5) Could not do daily work

2.b And 30 days after the last session?

• 1) none at all
• 2) a little bit
• 3) moderately
• 4) quite a bit
• 5) Could not do daily work

3.a Social Relationships - In the 30 days prior to contacting EAP, to what extent had your problems (whether physical or emotional) interfered with your normal social activities with family, friends, or co-workers?

• 1) none at all
• 2) slightly
• 3) moderately
• 4) quite a bit
• 5) Extremely

3.b And 30 days after the last session?

• 1) none at all
• 2) slightly
• 3) moderately
• 4) quite a bit
• 5) Extremely

The responses to these next questions differ from the previous responses.

4.a Perceived Health Status - In the 30 days prior to contacting EAP, how would you say your overall health was on a scale of 1-5, where 1 is excellent and 5 is poor?

• 1) Excellent
• 2) Very good
• 3) Good
• 4) Fair
• 5) Poor

4.b And 30 days after the last session?

• 1) Excellent
• 2) Very good
• 3) Good
• 4) Fair
• 5) Poor

5.a Absenteeism - In the 30 days prior to contacting EAP, how many days did you miss an entire day of work because of your physical, personal or mental health problems (average number of days)? ____

5.b And 30 days after the last session? ____

6. Work Engagement - The following statements reflect what you may do or feel on the job. Please indicate the degree to which you agree with each of the statements for the 30 days prior to accessing the EAP and 30 days after the last session.

1. I felt stimulated by my work.
2. I often thought about work on my way to the work site.
3. I felt passionate about my job.
4. I was often eager to get to the work site to start the day.
5. I often found myself thinking about my work at home.

7. Resiliency - Please indicate the extent to which you agree with each of the following statements using the following scale:

• 1= strongly disagree
• 2= disagree
• 3= neutral
• 4= agree
• 5= strongly agree

Since the end of my EAP sessions:

1. I tend to bounce back quickly after a hard time. ____
2. I have a hard time making it through stressful events. ____
3. It does not take me long to recover from a stressful event. ____

8. Self-Coping Abilities - Since the end of your EAP sessions, has there been any change with:

1. your ability to deal with stressful circumstances?
2. your capacity to put things into perspective?
3. your ability to act positively when addressing difficulties?

1. none at all
2. slightly
3. moderately
4. quite a bit
5. extremely
6. N/A

VI. EAP SERVICES STRENGTHS AND WEAKNESSES / HOW THE EAP SERVICES CAN BE IMPROVED.

In the last 4 questions I will ask for your general impressions of our EAP.

1. What do you believe are the strengths of our EAP?

_____________________________________________________________________________________________________________________________________________

2. What do you believe are the weaknesses of our EAP?

_____________________________________________________________________________________________________________________________________________

3. How might our EAP services be improved?

_____________________________________________________________________________________________________________________________________________

4. Did you receive the kind of EAP services you expected?

• Yes
• No

If no, what were you expecting?

_____________________________________________________________________________________________________________________________________________

5. You’ve completed the survey. Before ending this call, is there anything else you would like to say about your experience with EAP counselling services?

_____________________________________________________________________________________________________________________________________________

*********************

Thanks again for taking the time to participate in this survey!
If you have any questions or concerns, please feel free to call the 24-hour Employee Assistance Service phone line at 1-800-268-7708.