Resource Library

5. Project Results

5.1 Definitions

Outputs are defined in the project funding application form as "the products and events that will result from the project." They may include:

• one-time or time-limited events, such as
  • meetings (for consultation, educational, planning and/or training purposes);
  • training sessions;
  • campaigns (public education, fundraising);
  • displays; and
• products, such as
  • resources (manuals, pamphlets, newsletters, directories, videos);
  • reports;
  • planning/strategy tools (action plans/strategies, needs assessments, literature reviews, surveys, evaluations);
  • models/approaches (e.g., peer education programs)

Outcomes are defined in the funding application form as "the changes/improvements expected to occur as a result of the project." They may include:

• changes - for example:
  • in health status, knowledge/awareness, attitudes, behaviours;
  • at the level of the determinants of health - e.g., in the social, physical or service environments;
• mechanisms (e.g., a steering committee, a strategy).

In 2000-2001, these concepts were still relatively new to some community groups, and a certain amount of confusion still surrounded their interpretation. (For example, a support group resulting from project activities was classified in some reports as an output and in others as an outcome.) However, in an effort to enhance the quality and value of project evaluation activities, a standard reporting tool will be introduced across all regions in fiscal year 2001-2002. This tool will help to ensure that groups have a common understanding of key evaluation reporting terms.

5.2 Project Outputs

The volume of outputs rose appreciably in 2000-2001, compared with 1999-2000. This was due to stepped-up activity in the Program as it moved into the second year of funding. A wide assortment of outputs were recorded.

Consultation and planning sessions showed a marked increase in year two, as more groups began meeting and mobilizing their communities, carrying out formal and informal studies of hepatitis C populations and their needs and concerns, and planning how to address these through long-term strategies, action plans, etc.

Information, education and training-related outputs also rose in year two, boosted by a host of capacity-building projects. For example, training and educational sessions (e.g., for stakeholders, staff and volunteers) showed a six-fold increase, and educational events for client populations rose by a factor of 16. The latter included:

• theatre training for at risk youth;
• community workshops for street populations on nutrition, safer sex and other practical issues;
• suppers and bingo sessions, providing an occasion for hepatitis C-related presentations;
• one-on-one counselling sessions with staff for homeless persons.

Support groups and their development were a main focus of activity, with almost half of the projects initiating or strengthening such groups. Many projects used the support group setting as an opportunity to inform/educate participants on issues ranging from compensation claims to nutrition.

The number of media campaigns did not register a similar increase. This is not surprising. Information from needs assessments has provided project organizers with a clearer picture of their audiences' needs and concerns, and a better idea of the tools and resources required to gain the attention and confidence of groups once considered "hard to reach" (e.g., because of low literacy).

Even so, a number of projects recognized the value of using the media to catch the audience's attention. This resulted in an assortment of press articles, press releases/advertisements, radio and television interviews, and a media roundtable on cable television (taped for future use).

The volume of resources produced in year two increased sharply (from 19 in year one to more than 150). An analysis of these resources - encompassing print/audio-visual materials, web sites, displays/booths in shopping malls, and dramatic presentations - reveals that many were carefully geared to particular audiences. In the print category, by far the largest resources included the following:

• a "Hepatitis C Handbook for Injectors";
• a pamphlet on safe tattooing and body-piercing for residents of a detoxification centre;
• various games (e.g., for youth);
• street language materials for homeless persons;
• listings of services for hepatitis C-infected/affected persons, injection drug users, and street populations.

Various electronic resources were also developed, including:

• hepatitis C web sites (including a bilingual one with links to Charity Village, HepCBC, HepCure and Hepatitis International);
• moderated e-mail discussion groups;
• a regular hepatitis C network "Updater" providing members of one national NGO with e-mailed information on living with liver disease, local programs, services and activities.

An assortment of models, approaches and programs were developed (or were in development) for specific purposes during the period under review:

• education programs, such as "Living with Hepatitis C", "L iving with Liver Disease" (modified version), and a model educational strategy based on adult learning principles for use in prisons;
• peer-based training programs (e.g., for the rave community, inmates, young Aboriginal residents of treatment facilities, and volunteers providing support to affected groups);
• a medicine wheel teaching model;
• harm reduction programs.

Some groups took on the challenge of creating comprehensive models to meet the awareness, education and support needs of particular populations (homeless persons, inner-city dwellers).

Less easy to define, but of clear significance, was a sprinkling of actual and planned outputs that are perhaps best described as capacity-building tools. These outputs included the following:

• three databases developed by organizations for their own use (identifying relevant contacts/resources and other links, potential funders, and a mailing list for e-mail/internet information);
• a conference report;
• a software tool to facilitate data analysis;
• a policy and procedures manual;
• volunteer guidelines and recruitment tools (applications, position descriptions, volunteer agreements);
• a fund raising manual.

5.3 Project Outcomes

a) Intended Outcomes

Intended outcomes are those changes that occur as planned or anticipated over the short, intermediate or long term. They are the positive changes of a continuing nature that are intentionally sought by groups initiating projects. Most projects funded under the Program are aimed at bringing about short- and intermediate-term changes. On their own, such changes can seem small and insignificant. However, taken together over time they can assume great weight and eventually, through their combined effect, lead to the accomplishment of longer-term community health goals.

A review of outcomes sought and/or achieved in year two reveals three main groupings:

• those that are ameliorative, operating to relieve the distress, isolation and lack of support experienced by infected/affected populations;
• those that are preventative/harm-reducing, helping to halt or reduce spread of hepatitis C to populations at risk;
• those that are instrumental in the sense that they provide the means by which much of the desired change can be effected, especially over the long term.

i) Ameliorative outcomes⁷

A large number of initiatives sought to lessen the impact of an hepatitis C diagnosis on individuals, their families and friends through education, skills-building and support, by increasing their sense of control over their own health (empowerment) and improving their health practices. A number of projects also sought to effect positive changes in the social environment to make it more supportive of those affected by hepatitis C.

Knowledge and skills

Intended outcomes in this grouping included a better understanding on the part of the client populations about many aspects of the disease. For example, knowledge gains were reported relating to:

• hepatitis C itself, and the access points for community services;
• treatment options and research into hepatitis C and hemophilia for members of a hemophilia organization;
• the disease process and healthy behaviours, for hepatitis C patients and their families.

Skills-related outcomes identified in the project reports included:

• improved coping skills in hepatitis C-positive individuals with bleeding disorders and their families;
• improved ability to communicate with physicians;
• improved self-care skills for hepatitis C-positive persons.

Support

Client outcomes related to support included:

• increased ability to participate in a support group;
• enhanced links among several infected/affected inner-city clients who might form the core of a future support group (i.e., in stage 2 of project);
• increased sharing and mutual support among those infected and their "immediate circle";
• an increased sense of community (i.e., decreased social isolation) among infected or co-infected street-involved youth.

Social environment

A supportive social environment is crucially important for those whose lives are affected by hepatitis C. Perhaps because of its frequent association with injection drug use, hepatitis C appears to stir prejudice and fear in people, and hepatitis C-positive individuals - especially those living in small communities - can be too embarrassed or ashamed to seek help. Accordingly, some projects developed initiatives to change the social environment. Their aim was to increase the sensitivity of community groups/agencies and their personnel to the often overwhelming social barriers faced by persons already disadvantaged because of their illness. With more sensitive staff and approaches, clients can at least find a "safe zone". Among the positive effects reportedly observed were:

• improved agency attitudes to street-involved youth in a large city;
• a more supportive environment, including more sensitive, non-judgmental, compassionate attitudes towards inner-city populations struggling with homelessness, poverty and substance use;
• the provision of "accessible care and treatment without stigma or discrimination"; and
• increased understanding and sensitivity within community health and service groups/agencies concerning the reality of being infected with hepatitis C, and living as an injection drug user.

Empowerment

Yet another set of outcomes was noted in the reports. These outcomes appear to reflect the net gains derived from any or all of the above, in empowerment terms - i.e., the positive overall effect that is experienced by people whose knowledge and skills have increased, and who feel more accepted and supported. Groups for whom such outcomes were reported included:

• hepatitis C-positive individuals living in the urban core, who reported an increased ability to assess their own situation and cope, act and plan for the future;
• residents of a large rural/small town district, who reportedly experienced an improved quality of life and lifestyle gains with respect to nutrition, hygiene, medical compliance, and drug and alcohol consumption;
• hepatitis C-infected/affected residents of a remote rural region, whose stress and anxiety levels appeared to decrease;
• members of a hepatitis C "Healing Circle", who reported some improvements in their health practices.

ii) Prevention and harm reduction outcomes⁸

The outcomes in this category followed a similar overall pattern to those falling under the rubric of Support, in that they ranged from reports of enhanced knowledge and skills, through an increased sense of effectiveness and self-confidence, to improved health attitudes/practices. Positive environmental outcomes related to prevention and harm reduction were also reported.

Knowledge and skills

This group of reported outcomes included the following:

• increased awareness of hepatitis C and prevention strategies among hepatitis C-infected/co-infected inmates (federal and provincial);
• increased knowledge of injection drug risks and harm reduction techniques among injection drug users and homeless, under-housed and marginalized people living in the downtown area of a large city;
• increased knowledge of hepatitis C transmission (risk factors, consequences, etc) among non professional tattoo artists/body piercers who engage in potentially risky practices.

Sense of effectiveness

Outcomes illustrating a greater sense of effectiveness were the raised self-esteem and self-concept reported among inner-city youth who took part in an educational hepatitis C theatre project. These gains reportedly resulted from the new skills and knowledge they had acquired in the course of the experience.

Health practices

Following are some examples of improved client health practices, as reported by project groups:

• adoption of safer practices (i.e., use of clean injecting equipment and reduced needle-sharing) by an injection drug-using population;
• the safe disposal of used needles;
• higher rates of hepatitis C testing and by implication detection rates for at risk, street-involved youth in a mid-sized city; and
• positive behaviour changes" observed in male and female inmates of correctional institutions.

iii) Instrumental outcomes⁹

As discussed under Capacity-building (section 4.I.ii) many interventions sought to develop more effective mechanisms, systems, structures and skills to combat hepatitis C. The short- and medium-term outcomes of such initiatives are not ends in themselves, but instruments for achieving long-term population health outcomes. The following examples were noted.

Mechanisms

Intended outcomes in this category included the development of new community partnerships and the strengthening of existing ones. "Partnership" in this context is loosely construed to include collaborative groupings of all kinds, including project advisory committees, working groups and teams formed to conduct project-related activities, as well as broader (local and regional) service-provider networks and intersectoral partnerships.

As various projects demonstrated, collaborative mechanisms such as these provided a means whereby members could contribute their varied knowledge, skill sets and perspectives, as well as their expertise with specific populations (e.g., improving audience reach, ensuring informed input, and increasing the effectiveness of dissemination efforts). One Alberta project report noted that its workers had increased their knowledge in two new areas (methadone maintenance and the provincial process for organ transplantation) as a result of expanded collaboration. In Atlantic Canada, the close working relationship that evolved between project organizers and a needle exchange dramatically increased the project's ability to reach injection drug users.

The inclusion of clients in collaborative bodies provided an ongoing means of tapping into their worlds (for insights, project approaches, feedback on resources) as well as a chance to overcome stereotypes and prejudices on all sides. For example, a project in Saskatchewan reported that youth formerly in trouble with the law in Prince Albert had "forged a new relationship with adults [including law enforcement officials], based on respect and power-sharing, which now informs the project's direction". Their collaboration with the police had resulted in invitations to make 20 additional school presentations ("Who ever thought the police would listen to us?").

Increased coordination (as a result of committee work) also helped to reduce duplication of effort. One project group collaborating with a local health authority in Alberta was pleasantly surprised to find that the latter could supply needed data (on hepatitis C incidence in the region). This saved the group both time and effort. Another project commented, "Participating in the activities of other non-profit groups and organizations enabled us to learn their mandates and work together for the benefit of citizens without duplicating services ....we each enhance the other's work". With one notable exception (in which tensions arose between groups concerning the modes whereby their respective clienteles had become infected) working together also helped to ensure that the partners were in agreement on key issues, reducing the potential for community members to receive mixed or confusing messages. One report offered a blunt but succinct assessment of collaborative work: "Though cumbersome, the partnerships were effective."

When sectors and service providers work together, bolder initiatives become possible. Illustrating this were two initiatives, both involving a wide spectrum of agencies and organizations, that attempted to develop "comprehensive" and "whole-person" approaches. The breadth of knowledge and expertise at the table enabled these groups to cluster the issues rather than tackling them piecemeal.

Increased knowledge and awareness

Knowledge gains reportedly came in several areas: for example, various groups increased their basic understanding of hepatitis C, client needs, harm reduction principles, and ways to locate resources and assistance. Likewise, they and their workers acquired skills in outreach, communication, adult education, prevention and harm reduction, among others.

Changes in structures and systems

The reports identified a range of positive outcomes in this category, including:

• improved co-ordination, planning and referral systems;
• better overall accountability to clients;
• improved staff organization and arrangement of physical space; and
• streamlining of services.

By tailoring their interventions to community needs, many groups were able to facilitate client access to improvements in:

• services (more locations, easier accessibility, more convenient times);
• educational events (weekend dates, no charge for attendance);
• information (more understandable, more relevant, appropriate language, available confidentially, etc);
• delivery modes (peer education, community-based events, often including food, held in locations frequented by the client populations);
• formats (games, mobile displays, bingo sessions, suppers, and film discussions).

b) Unintended Outcomes

What happens unexpectedly often carries within it a seed of learning - a hint, possibly, that a different approach may be needed, a different resource tried, or another population included.

There were project groups that found they had miscalculated with respect to intended outcomes. While several groups were more successful than expected, this sometimes placed unforeseen demands on their resources. Initiatives of this kind included:

• a BC/Yukon project that reported higher than expected demand for its newsletters, as well as telephone-call and referral volumes;
• an Ontario project that was able to cancel plans for a telephone survey as a result of the high turnout at its public forum;
• a Saskatchewan group that reported receiving an unexpectedly high volume of requests for information and training from the public and, to its reported surprise, from medical professionals as well.

Other projects either overestimated or underestimated the level of effort or resources needed, or misjudged where the demand would come from. For example,

• in one project, organizers found that their involvement with a support group stretched out for longer than planned - the clients needed more time to become comfortable participating in a "group support system";
• projects in two regions reported that they unexpectedly had to dedicate resources to assisting clients with compensation claims (related to infection through the blood supply);
• another project received unexpected requests for collaboration with community agencies (e.g., the Salvation Army, and the province's alcohol and drug abuse commission) to support injection drug users.

Some projects identified "down sides" to increased networking and professional development. In one case, it was the continual need to "keep up" with leading-edge information and disseminate this to client populations. A similar concern was echoed by a group that found itself under pressure to revise information for staff, trainers and partners.

As one project report noted, "more partners means more visions are involved". This was an initiative in which stakeholders had constantly pressed for the content to be expanded to include their particular areas of expertise - abuse, street life, gender inequalities and cultural issues. In another project, a consortium met some resistance to a plan to expand its mandate that would include hepatitis C issues - one consortium member feared that a diluted mandate would reduce the organization's ability to focus on HIV issues.

On the whole, however, the unexpected results yielded by involvement with networks were considered beneficial. Two regional networks in Ontario collaborated on a previously unscheduled "mini hepatitis C conference", and a Manitoba project was pleasantly surprised by the number of individuals and groups across the country involved in hepatitis C issues at different levels, and by the willingness of the hepatitis community to share information. An inner-city project received offers of help from unexpected quarters (e.g., individuals and groups not infected, at risk or directly affected by hepatitis C who showed an interest in volunteering).

Unpredictability in their client populations stymied some projects. One Ontario group said it had experienced unexpected difficulty in attracting and retaining hepatitis C-positive individuals for its advisory committee. In Alberta, project workers reported that although clients had initially shown great interest in being trained as peer workers, and provided good input, they had failed to turn up at subsequent sessions. In contrast, the enthusiasm of their client populations caught some project workers off guard (for example, the receipt of unsolicited written input from an inmate in Atlantic Canada who had participated in an initial work group meeting, and the unexpectedly strong referral role played by peer workers in an Ontario initiative.)

[Previous] [Table of Contents] [Next]