Number 18 - A Framework for Developing, Implementing and Reviewing Diversity Training Programs

The Public Service of Canada is serving an increasingly diverse Canadian population. In addition to being responsive to the public, this diversity must be reflected in the compostion of its workforce. Departments and agencies want to become more competent in the management of a diverse workforce. In response, the demand for diversity training has been steadily increasing across the Public Service.

The purpose of the SMIF proposal was to develop a tool to help mangers and project leaders make the necessary decisions and choices regarding the development, implementation and review of diversity training programs. "A Framework for Developing, Implementing and Reviewing Diversity Training Programs - A Guide for Public Service Managers and Project Leaders" describes the general stages managers and project leaders undertake to determine that a diversity training program is required and identifying a suitable program that meets the organization's needs.

The Guide is available in both official languages and will be available on the TBS Web Site. The Interdepartmental Committee on Employment Equity will be requested to inform their departmental managers of the availability of the Guide. The Public Service Commission has agreed to market this Guide as part of their marketing strategy for diversity and employment equity tools.

Employment Equity in the federal public service is a Treasury Board Secretariat (TBS) priority. To achieve it, a fundamental shift must take place in the public service’s corporate culture. One way to create this shift is through the implementation of training programs that make employees more aware of the value of a diverse workforce and helps them to learn the skills they need to work effectively with different types of people. In the case of the federal public service, this involves the four designated Employment Equity Program groups: Aboriginal peoples, visible minorities, persons with disabilities and women.

This guide helps managers and project leaders make the necessary decisions and choices regarding the development, implementation and review of diversity training programs. It describes the four general stages of this decision-making process and presents them in straightforward terms for non-experts. The stages are:

Determining Needs: Identifying workplace problems or needs and determining whether they would best be addressed by diversity training or another management strategy.

Selecting / Developing a Program: Determining training objectives, intended audience and whether to choose an existing program or develop and validate a new one.

Implementation and Monitoring: Monitoring the programs as they proceed to ascertain outcomes.

Decision Review and Evaluation: Periodically reviewing ongoing programs to determine whether they should be maintained, modified, replaced or discontinued based on quality and continuing benefits.

Annex A presents a case study, loosely based on a real situation, that puts the contents of the guide in concrete terms. Annex B is a checklist of steps to take and factors to consider in implementing and reviewing a diversity training program.

Note: The framework is broadly applicable and not designed to restrict the nature or scope of co-funding proposals to the Special Measures Innovative Fund of TBS's Special Measures Initiatives Program. However, departments seeking co-funding should demonstrate that relevant stages of the framework have been applied in developing the initiative, particularly in terms of identifying needs and how the program addresses them. Proposals for new programs should include reasons why existing programs are not acceptable.

Identifying Needs and Sources: The decision to pursue diversity training starts with the recognition that a workplace problem exists. The problem might be under-representation of a designated group, cross-cultural antagonisms, poor-quality work, low productivity or any matter that interferes with a smooth-running environment.

The next step is to identify the source of the problem and whether or not it is diversity related. For example, low productivity could be a matter of job skills rather an diversity, and poor-quality work may reflect low employee morale which may or may not be diversity related.

Diversity-related problems tend to arise from one of + or a combination of + three sources:

• external pressures
• the job system (physical environment, equipment, job descriptions, staff size, etc.)
• employees (skills, attitudes, prejudices, etc.).

Diversity training is most effective when the workplace problem is due to employee matters. Needs that arise from external pressures or from the nature of the job itself are usually best addressed by other management strategies. It is important therefore to compile separate lists of the needs that are attributable to each of these sources.

To Train or Not to Train: Diversity training programs typically heighten employee awareness, engender an appreciation of workforce diversity, generate more positive attitudes and foster better interaction across groups. If your workplace would benefit from one of these advances, then diversity training would be appropriate.

But it is also important to remember that training programs are only one answer. Other practices such as employment equity strategies, equitable pay and promotion policies, and fair performance evaluations + when implemented as part of a comprehensive human resources management strategy + can be highly effective in addressing diversity-related workplace needs. Often, the work environment can be made more conducive to diversity by ensuring that staff members understand program and policy objectives, that management is accountable and that key players such as unions lend support.

In the end, training is not a substitute for good communication and other strategies. It is most effective when it is implemented as part of a broader coherent human resources management strategy. Its outcomes are durable only when the program is carried out in concert with other measures and reinforced by senior management with concrete illustrations of the appreciation of diversity.

Having decided that a diversity training program is required, the task is to find a program that meets your needs, comes close enough that it can be suitably modified or, failing that, to design one that specifically addresses your workplace's particular problems. It is significantly cheaper and faster to choose a program that already exists and, as a general rule, this should be your first option. The key is to know what to look for when choosing among the myriad programs available.

How to Select a Program: The first thing to look for is a match between your workplace needs and the program's objectives. Program objectives are often expressed as "expected outcomes", that is the measurable benefits that trainees derive from the program. These may include awareness (of self and/or the similarities and differences of others), attitudes (fewer biases, more positive attitudes, enhanced sensitivity), knowledge (information about other groups), behaviours (more productive, harmonious interpersonal interactions) and performance (greater productivity).

It is also important that the program be at an appropriate level of difficulty or complexity for the intended audience, and in a format that appeals to these individuals. As well, you want a program that is well designed + that is, it must meet its objectives, have reliable learning outcome measures, be accompanied by an implementation manual, etc. You can assess the design of a program and its suitability in part by answering the questions in the accompanying box. You can also check for documentation regarding the qualifications of the people who designed the program (knowledge and experience in diversity issues as well as training program design/implementation), a written summary of proposed outcome measures and previous results, and an independent evaluation.

Developing a New Program: If no existing program meets your needs or comes close enough to be modified, you will have to create your own program. The technical complexities of this task are beyond the scope of this guide but, in general, the process entails:

• working with subject-matter experts to develop a plan
• assembling a project team and designing the training package
• testing the program on a limited scale, evaluating it and refining it
• producing a manual detailing the program design and validation, proposed measurement instruments, and instructions for implementing the program as intended by its designers and evaluators.

Measuring Immediate Outcomes + Validation: Whether you are assessing the merits of an existing program or are in the process of having one developed for you, an important element to consider is the measurement of results or outcomes.

Validation is considered an important part of the design process. It is encumbant on the program designers to demonstrate that refinements, based on this evaluation, were introduced. Whether you are assessing a program specifically developed for your workplace or making a decision among existing programs, the following questions can help.

• Were learning outcomes measured at the end of the program? What measures were used, do they reflect program objectives and does information as to their validity and reliability exist?
• According to the above outcome measures, what proportion of the trainees benefited from the program and by how much?
• Has the program been validated by independent evaluators so that it is now ready for widespread implementation?

If documented material addresssing this type of information is not available, it would suggest the program has not been adequately evaluated or validated.

After validation, the program is ready for widespread use. To achieve the learning outcomes intended, the course must be implemented as the designers and evaluators recommend and taught by competent trainers. The success of the program's implementation is gauged by immediate measured outcomes (that is, its direct impact on trainees) and eventual outcomes (that is, its effectiveness in changing the workplace).

The program is inititially implemented as designed with no alterations and according to the training manual. The manual details program objectives, intended trainees, size of training groups, content and pedagogical approach, expected characteristics and skills of the trainers, etc. As the program continues, it should be monitored either directly by you or by a representative to ensure it is being implemented according to the manual.

Immediate Measured Outcomes + Impact on Trainees: It can never be assumed that a program is achieving its expected results. Its impact on the trainees must be measured for reasons of accountability and cost-effectiveness. Instruments used to measure outcomes should reflect the program's written objectives and expected learning outcomes. Among the most popular measures of trainee progress are tests, questionnaires or surveys of knowledge, skills and attitudes. These should be administered both before and after the program to assess how much change has occurred and how much of this change is due to the program.

Eventual Outcomes + Impact on the Workplace: The true measure of a program's success is whether or not it has resulted in measurable changes in the workplace. Longer-term outcomes such as inter-cultural harmony, less harassment, fewer and less intense interpersonal conflicts, reduced employee turnover, more equitable performance appraisals, greater productivity, etc., are the best indicators of a program's success. While practical and technical challenges often stand in the way of such measurements, managers are encouraged to seek relevant information through discussions with staff, opinion surveys or direct analysis of employee performance over time.

It is important to periodically revisit your decision to implement diversity training to determine if the program should be maintained, modified, replaced or discontinued.

A formal evaluation, conducted by experts, provides valuable information about the quality and outcomes of the program. It is, in essence, a "report card". Often called a summative evaluation at this stage, it assesses the program's outcomes which helps managers determine if the program is cost-effective and of benefit to participants.

Like everything else, diversity issues change over time. In the workplace, this could be due to several factors + employee turnover, changes in social climate, modified policies, job shifts and other factors. A significant determining factor is, of course, the successful completion of a diversity training program. It is important therefore for managers to review (yearly, if possible) their decision to implement diversity training and, based on the results, decide if the program should continue. The following questions should be addressed.

• Do the employees still need what the program is designed to offer?
• Are the employees still motivated to take this program?
• Have other programs been developed that are better or more cost-effective in relation to current employee needs?
• Has the successful application of non-training strategies reduced or eliminated the need for diversity training?
• Should a program be maintained, modified, replaced or discontinued? This question calls for management decision. In essence:
• If a program still yields its intended benefits, is cost-effective, has trained some but not all candidates, is in demand and responds to workplace needs that are still considered important, it is reasonable to continue it.
• If feedback from program evaluators, trainers and/or trainees suggests that the program is acceptable but requires minor adjustments, it is reasonable to modify it for further use.
• If a demonstrably better or more cost-effective program is now available, it would be reasonable to replace the program.
• If the program no longer demonstrates value, or is no longer needed for whatever reason, it should be discontinued.

Determining Needs: The setting involved some 3,000 employees, 200 of whom were women. The work was physically demanding and, for safety reasons, required tight teamwork. The problem was excessive male-female conflicts.

The Director General (DG) met with senior managers in two small groups and, after examining complaint files, concurred that management action was warranted. They brainstormed to list the important problems, those they considered to be diversity related and what the most likely underlying needs were (what must change to address the problem). They categorized the needs separately according to probable sources: external pressures, the job itself or employee characteristic.

The DG created four focus groups representing management, staff and both designated and non-designated groups. Each group met for a day to review and revise the list of needs. A professional associate summarized the information and highlighted the needs considered important by the DG, senior managers and more than one focus group.

Two problems surfaced. (1) An unusually large number of employees, mostly women, had been hired the previous year and, because they had little prior experience, their performance for the first few months was substandard (a job system source). (2) Interpersonal male/female conflicts were frequent because the men refused to entrust their safety to inexperienced female colleagues (an employee source and therefore diversity related). The underlying needs were job skills for the women and a trusting attitude for the men.

The DG and senior managers decided to (1) modify the recruitment campaign to hire more experienced applicants; (2) provide more on-the-job training and supervision during the first three months for all new employees; and (3) run a diversity training program for the men, within a specified budget and time frame.

Choosing a program: A three-person task force was set up to define objectives and trainee characteristics, and find a matching program. Each task force member had experience in either diversity, training or the work setting.

Characteristics of intended trainees: English-speaking males; average age, 38; high school education; average 12 years' experience in that job; a preference for tough outdoor work.

Objectives: to promote awareness of women's strengths in dealing with stressful situations and to provide information about women performing competently in physically demanding jobs.

Immediate learning outcomes: in answering opinion questionnaires, the men would demonstrate views that both sexes can deal with psychologically and physically stressful situations; and, they would accurately answer factual questions on a multiple-choice test about physically demanding jobs successfully performed by women.

The task force, with the help of a consultant in program evaluation and diversity issues, found 12 potentially appropriate programs. It chose one that was designed to promote awareness and valuing of women's competence and trustworthiness. It included an appropriate attitude questionnaire as an immediate outcome measure and, from the available description, seemed to have been developed in a professional manner and in a format suitable for the intended trainees. Although it did not address the second objective, the DG in consultation with senior managers agreed with the task force's recommendation and purchased the program for a six-month trial period.

Implementation and monitoring: The four-week program was given in three-hour sessions to groups of up to 15 men, by a male/female training team. It used films, brief lectures, case studies and discussion. Attendance was mandatory. The vendor agreed to provide one training session per month for six months and to provide feedback on immediate outcomes and overall employee program acceptance. The vendor also agreed, if requested, to provide an intensive three-day workshop for the government employees who would do the subsequent training.

Participants took two separate 30-minute attitude surveys + both immediately before and after the program + to measure immediate outcomes.

Five seniors managers monitored the programduring the trial period.They:

• verified that the program was adequately advertised and that participants were given the necessary time off work to attend
• requested and received résumés of the trainers and a detailed training schedule with times, places and participants
• sent a form letter to all trainees thanking them for agreeing to participate and inviting feedback
• at the end of the six months, requested and received a report from the vendor including immediate outcome statistics and comments concerning the six-month implementation. In this case, it was decided to provide the outcome statistics anonymously, without indicating the scores of individual employees. The statistics showed that 90 per cent of the trainees showed better attitudes and, according to comments, enjoyed the program.

The managers therefore recommended that the program be fully implemented. The DG accepted this recommendation. The vendor provided workshops for the trainers as originally agreed, with the understanding that experienced trainers would "spot visit" the workshops during the first three weeks. The DG appointed a senior manager as overall coordinator and requested a written report within a year. The program proceeded without incident.

Review and Evaluation: The DG received a written report from the program coordinator 18 months after the start of the program. Some 585 employees had participated; the cost was 20 per cent higher than budgeted; 83 per cent of participants showed significant gains; and, as yet, there was no evidence of impact on the workplace.

The DG decided to:

• continue the program for two years, given the immediate positive results and number of employees still to be trained;
• approve the extra 20 per cent in spending;
• hire external program evaluators to do a summative evaluation of the program, including an assessment of workplace impact, over the next 12 months; and,
• before the end of the two-year period, to determine in consultation with all persons concerned and taking into account the summative evaluation results, whether the program should be maintained, modified, replaced or discontinued.

• Identify problems that may be diversity related
• Identify probable underlying needs
• Classify problems/needs according to source: external pressures, the job itself or employee issues
• Based on consultations, establish list of potential non-training management decisions
• Envisage non-training action plans
• Consider if remaining employee problems/needs might be addressed by diversity training
• State concrete purposes of the proposed diversity training in relation to employee problems/needs.

• Program objectives are stated concretely, preferably as expected learning outcomes.
• The program is expected to influence trainees:

• awareness
• attitudes
• performance
• knowledge
• behaviours

• Target audience is determined.
• A review of existing programs is conducted regarding program objectives, expected learning outcomes and intended trainees.
• A list of potentially suitable existing diversity training programs is drawn up.
• Each program is considered in terms of:
• objectives in relation to employee needs
• previous application to similar trainees
• documentation about design and validation
• available instruments for measuring immediate outcomes
• An existing program is selected or a new program must be developed. Detailed plans are made in consultation with experts.

• The program has a manual or equivalent document which provides a detailed description of the program, concrete objectives, target audience, evidence that the program meets its objectives, implementation instructions and proposed outcome measures.
• The program is presented by qualified trainers.
• What the trainees are learning from the program is measured routinely.
• The measured outcomes of the program are monitored from time to time and are acceptable.
• In the workplace, indications exist that the problems which led to the diversity training have been reduced or resolved.

After a program has been in use for a time, the following should be verified periodically:

• That the program has been evaluated recently.
• That the program benfits the trainees, is cost-effective, continues to address relevant employee needs, is still in demand and does not need updating

That the most appropriate decision is to:

• continue the program
• modify the program and then continue its use.
• replace the program by another which is better and/or more cost-effective.
• discontinue the program.

[ Table of Contents ]