Canadian Medical Association Journal 1995; 153: 1233-1236
Paper reprints may be obtained from: Dr. Renaldo N. Battista, Division of Clinical Epidemiology, Montreal General Hospital, 1650 Cedar Ave., Montreal QC H3G 1A4; fax 514 934-8293
[Table of Contents]
Clinical practice guidelines are being developed by a variety of organizations, including
physician associations, specialty societies and those who pay for health care. Nevertheless, relatively little information is available about how clinical topics for guidelines development are selected.
Given the resources required to produce and disseminate guidelines, a focus on the areas of highest priority is critically important. However, with the wide range of interests in guidelines development it is unlikely, and perhaps even undesirable, that all guidelines developers agree on a common set of priorities. For example, as part of its interest in establishing a professional domain for its members a specialty society may have a strong interest in guidelines for the use of a particular therapy, whereas payers' interest in guidelines for the same technology may stem from a desire to optimize spending on it or to reduce queues.
In light of the enormous potential of clinical practice guidelines we review the largely US literature on priority setting and consider its applicability to Canadian efforts. We also summarize the results of a survey mailed to workshop participants that pertain to priority setting. (The complete survey findings were reported in the first article of this workshop series.1) Last, recognizing the possibility of conflict among various developers' priorities, we propose some general principles to establish a shared framework for priority setting to encompass the possible interests of all groups.
[Table of Contents]
Literature review
The published material on priority setting for clinical practice guidelines is minimal. The peer-reviewed literature is scanty and generally from US organizations active in the related area of technology assessment. We searched Index Medicus electronically for articles published from 1985 to 1994 and identified 655 referring to clinical practice guidelines, of which 2 were also identified through searches on priority setting. A similar search for publications on technology assessment yielded 1003 articles, of which 14 were also identified through searches on priority setting. We then examined these 16 articles, discarding 5 as unrelated to guidelines development. Of the remaining 11, 3 were pertinent to our discussion of priority setting in guidelines development. We identified additional relevant sources from the references of published material and from the files of the Council for Health Technology Assessment, in Quebec. Last, we approached knowledgeable people regarding other sources of material on priority setting; none identified additional material.
Six of the sources we reviewed described factors used in setting priorities, but none provided a practical example of how the process is used in topic selection. The report of the US Institute of Medicine5 (IOM) was the most comprehensive. After examining the available evidence, it recommended that analytic, data-intensive approaches, however useful in technology assessment or guidelines development, should not be substituted for the process of building consensus among stakeholders. The IOM recommended that, regardless of the method used, explicit criteria should be identified and recorded so that priority setting is not solely an implicit judgement process. Table 1 lists such criteria from various groups.
[Table of Contents]
Synthesis of findings
Drawing on these sources we defined two main criteria: the prevalence and health burden associated with a condition, and the cost implications of the practice for which a guideline is contemplated. We also defined two associated criteria: the guideline's potential effect on the costs of the practice, and its potential to reduce the burden of morbidity and to enhance the quality of care associated with a condition.
Given the focus on technology assessment, some data or information required in these analytic priority-setting processes may be difficult or almost impossible to obtain for practice-related activity. For example, "management of diabetes" includes diverse services often not individually identifiable, whereas "CT scanning of the head" is more likely to be uniquely identifiable from available data sources, particularly claims data.
Nevertheless, we identified several themes and associated questions for consideration when priorities are set for guidelines development. They form the basis for the priority-setting process we propose.
[Table of Contents]
Feasibility
[Table of Contents]
Survey results
Questionnaires were sent to 107 potential workshop participants, 55 of whom completed
and returned it. Regarding priority setting, respondents were asked to rate the importance of each of seven criteria on a 5-point scale (from "very important" to "not at all important"); to describe priority setting in the organizations they represented; and to list who was consulted in these priority-setting processes.
The seven criteria were health burden on the population, economic burden of disease on society, the cost of treatment to the health care system, the extent of practice variation, the state of scientific knowledge, the cost of guidelines development and practitioners' interest in having guidelines developed. Respondents were not required to rank the criteria relative to each other and could thus rate all seven the same if they wished.
The respondents were classified as belonging to one of five groups: physician licensing authorities, governments and paragovernmental organizations, provincial and territorial medical associations or other organizations representing physicians at the provincial level, national specialty societies, and others. For each group, average scores for each of the seven items were calculated.
The survey findings were reported in detail in the first article of this workshop series. In brief, between-group differences in item scores were not significant. Table 2 in the first article of the series presents the criteria that were deemed to be very important or important by the various respondent groups. All of the groups identified health burden on the population as a very important factor in setting priorities for guidelines development. Also, the respondents felt that the state of scientific knowledge was an important factor, perhaps reflecting concerns about feasibility. They gave relatively lower scores to the cost of guidelines development and to practitioners' interest in having guidelines developed. Last, although respondents identified the people or groups responsible for setting priorities in their organizations, they gave relatively little information about how priorities were actually set.
In addition to rating the seven criteria for their importance in priority setting, the respondents were invited to specify other criteria. They had seven suggestions, reflecting three themes: risks associated with inappropriate treatment, existence of a champion-led group with enough resources for guidelines development and implementation, and evidence for the effectiveness of guidelines in changing physician behaviour.
[Table of Contents]
Thinking about priority setting
Several key messages emerged from our review of the literature on priority setting and the survey results. First, concern with the health of the population and, by extension, the quality of care delivered appears to be foremost in Canadian thinking about guidelines. Cost concerns, whether measured in terms of the cost of treatment to the health care system or the economic burden of illness on society, received varying levels of consideration, depending on the nature of the organization rating the factors. Costs of care and of illness appear to be greater concerns for governments than for other stakeholders, particularly provincial and territorial medical associations.
Second, practice variation may provide information for priority setting, although the methods used to evaluate the extent of variation must be appropriate. Variation may expose patients to substantial harm or lost benefit if the practice in question has a particularly complex risk-benefit profile. Also, regardless of the effects of the practice on patients, variation usually has cost implications that may be difficult to measure, even if they are recognized. For example, underuse of preventive screening services of demonstrated effectiveness represents a narrowly defined financial saving to the health care system that may vanish if guidelines lead to 100% participation. In practice, variation associated only with low-risk "unnecessary" care appears to have been thought less important than variation that may directly affect people receiving medical attention. This relative discounting of variation in low-risk or low-cost services may mirror respondent groups' different ratings of factors for priority setting and should be examined with a view to removing impediments to collaboration in guidelines development.
In terms of what is actually happening in Canada, there is no reported use of analytic, data-intensive approaches. Consultation with members of guidelines-developing organizations was, however, frequently mentioned by the survey respondents. In contrast to the heavy emphasis on community and patient participation advocated by the IOM in the United States, the survey respondents appeared less concerned with community participation. Given the increasing casting of members of the public as partners and clients of the health care system, this absence of concern may undermine the impact of guidelines in Canada. The development of guidelines for which there is no receptive practitioner or patient audience will be of no long-term benefit, regardless of one's position in the health care system. Although differences in the health care systems and cultures of the two countries may limit the transferability of US experience to Canada, the IOM's framework, in which structured priority setting informed by quantitative analysis involves significant roles for community and patient input, is a useful one for Canadian developers.
Given the nature of the Canadian health care system and of the links that already exist among stakeholders through organizations such as the National Partnership for Quality in Health, there may be little gain from huge, new investments in the data collection necessary to implement a strictly analytic approach. Existing data, particularly administrative claims data, should be seriously considered for use in providing quantitative inputs for priority setting. Furthermore, purely analytic approaches appear to grow in importance only when it is impossible to include all involved parties in setting priorities.
In light of these observations, we propose the following framework for priority setting by Canadian groups developing clinical practice guidelines.
If macro-level mechanisms do extend to the level of individual physician practice (e.g., through internal markets for services or fund-holding physicians, as in Britain) an established system of guidelines would be an important forum for developing bases for funding and reimbursement, rather than directly intervening in the physician-patient relationship. In short, while our neighbours to the South struggle to get the "health-care-costs horse" back in the fiscal barn, we in Canada may still have time to work with the horses to ensure that they do what we want.
[Table of Contents]
References
CMAJ: Nov. 1, 1995
"Guidelines Workshop" articles