Setting priorities and selecting topics for clinical practice guidelines

Renaldo N. Battista, MD, ScD, FRCPC; Matthew J. Hodge, BA, PhD

Canadian Medical Association Journal 1995; 153: 1233-1236

This article is the third in a series of six that appeared in the October, November and December issues of CMAJ.

Paper reprints may be obtained from: Dr. Renaldo N. Battista, Division of Clinical Epidemiology, Montreal General Hospital, 1650 Cedar Ave., Montreal QC H3G 1A4; fax 514 934-8293

Abstract

Setting priorities and selecting topics are important steps in guidelines development, but they have received relatively little attention to date. Responses from a survey of guidelines stakeholders in Canada suggest that the health burden of a clinical condition on the population is an important factor in priority setting. Economic considerations, cast as either costs of treatment to the health care system or the economic burden of illness to society, are given varying importance by different stakeholder groups. Drawing on the literature and the survey results, the authors propose a framework for priority setting. Important issues requiring consideration include the role of public and community participation, the need for and appropriate emphasis on quantitative data regarding current practice and its variation, and mechanisms to link guidelines to health-policy development and management of the health care system.

[Table of Contents]

Clinical practice guidelines are being developed by a variety of organizations, including physician associations, specialty societies and those who pay for health care. Nevertheless, relatively little information is available about how clinical topics for guidelines development are selected.

Given the resources required to produce and disseminate guidelines, a focus on the areas of highest priority is critically important. However, with the wide range of interests in guidelines development it is unlikely, and perhaps even undesirable, that all guidelines developers agree on a common set of priorities. For example, as part of its interest in establishing a professional domain for its members a specialty society may have a strong interest in guidelines for the use of a particular therapy, whereas payers' interest in guidelines for the same technology may stem from a desire to optimize spending on it or to reduce queues.

In light of the enormous potential of clinical practice guidelines we review the largely US literature on priority setting and consider its applicability to Canadian efforts. We also summarize the results of a survey mailed to workshop participants that pertain to priority setting. (The complete survey findings were reported in the first article of this workshop series.1) Last, recognizing the possibility of conflict among various developers' priorities, we propose some general principles to establish a shared framework for priority setting to encompass the possible interests of all groups.

[Table of Contents]

Literature review

The published material on priority setting for clinical practice guidelines is minimal. The peer-reviewed literature is scanty and generally from US organizations active in the related area of technology assessment. We searched Index Medicus electronically for articles published from 1985 to 1994 and identified 655 referring to clinical practice guidelines, of which 2 were also identified through searches on priority setting. A similar search for publications on technology assessment yielded 1003 articles, of which 14 were also identified through searches on priority setting. We then examined these 16 articles, discarding 5 as unrelated to guidelines development. Of the remaining 11, 3 were pertinent to our discussion of priority setting in guidelines development. We identified additional relevant sources from the references of published material and from the files of the Council for Health Technology Assessment, in Quebec. Last, we approached knowledgeable people regarding other sources of material on priority setting; none identified additional material.

Six of the sources we reviewed described factors used in setting priorities, but none provided a practical example of how the process is used in topic selection. The report of the US Institute of Medicine5 (IOM) was the most comprehensive. After examining the available evidence, it recommended that analytic, data-intensive approaches, however useful in technology assessment or guidelines development, should not be substituted for the process of building consensus among stakeholders. The IOM recommended that, regardless of the method used, explicit criteria should be identified and recorded so that priority setting is not solely an implicit judgement process. Table 1 lists such criteria from various groups.

[Table of Contents]

Synthesis of findings

Drawing on these sources we defined two main criteria: the prevalence and health burden associated with a condition, and the cost implications of the practice for which a guideline is contemplated. We also defined two associated criteria: the guideline's potential effect on the costs of the practice, and its potential to reduce the burden of morbidity and to enhance the quality of care associated with a condition.

Given the focus on technology assessment, some data or information required in these analytic priority-setting processes may be difficult or almost impossible to obtain for practice-related activity. For example, "management of diabetes" includes diverse services often not individually identifiable, whereas "CT scanning of the head" is more likely to be uniquely identifiable from available data sources, particularly claims data.

Nevertheless, we identified several themes and associated questions for consideration when priorities are set for guidelines development. They form the basis for the priority-setting process we propose.

[Table of Contents]

Feasibility

Population

Costs

Effects on population, costs and practice

[Table of Contents]

Survey results

Questionnaires were sent to 107 potential workshop participants, 55 of whom completed and returned it. Regarding priority setting, respondents were asked to rate the importance of each of seven criteria on a 5-point scale (from "very important" to "not at all important"); to describe priority setting in the organizations they represented; and to list who was consulted in these priority-setting processes.

The seven criteria were health burden on the population, economic burden of disease on society, the cost of treatment to the health care system, the extent of practice variation, the state of scientific knowledge, the cost of guidelines development and practitioners' interest in having guidelines developed. Respondents were not required to rank the criteria relative to each other and could thus rate all seven the same if they wished.

The respondents were classified as belonging to one of five groups: physician licensing authorities, governments and paragovernmental organizations, provincial and territorial medical associations or other organizations representing physicians at the provincial level, national specialty societies, and others. For each group, average scores for each of the seven items were calculated.

The survey findings were reported in detail in the first article of this workshop series. In brief, between-group differences in item scores were not significant. Table 2 in the first article of the series presents the criteria that were deemed to be very important or important by the various respondent groups. All of the groups identified health burden on the population as a very important factor in setting priorities for guidelines development. Also, the respondents felt that the state of scientific knowledge was an important factor, perhaps reflecting concerns about feasibility. They gave relatively lower scores to the cost of guidelines development and to practitioners' interest in having guidelines developed. Last, although respondents identified the people or groups responsible for setting priorities in their organizations, they gave relatively little information about how priorities were actually set.

In addition to rating the seven criteria for their importance in priority setting, the respondents were invited to specify other criteria. They had seven suggestions, reflecting three themes: risks associated with inappropriate treatment, existence of a champion-led group with enough resources for guidelines development and implementation, and evidence for the effectiveness of guidelines in changing physician behaviour.

[Table of Contents]

Thinking about priority setting

Several key messages emerged from our review of the literature on priority setting and the survey results. First, concern with the health of the population and, by extension, the quality of care delivered appears to be foremost in Canadian thinking about guidelines. Cost concerns, whether measured in terms of the cost of treatment to the health care system or the economic burden of illness on society, received varying levels of consideration, depending on the nature of the organization rating the factors. Costs of care and of illness appear to be greater concerns for governments than for other stakeholders, particularly provincial and territorial medical associations.

Second, practice variation may provide information for priority setting, although the methods used to evaluate the extent of variation must be appropriate. Variation may expose patients to substantial harm or lost benefit if the practice in question has a particularly complex risk-benefit profile. Also, regardless of the effects of the practice on patients, variation usually has cost implications that may be difficult to measure, even if they are recognized. For example, underuse of preventive screening services of demonstrated effectiveness represents a narrowly defined financial saving to the health care system that may vanish if guidelines lead to 100% participation. In practice, variation associated only with low-risk "unnecessary" care appears to have been thought less important than variation that may directly affect people receiving medical attention. This relative discounting of variation in low-risk or low-cost services may mirror respondent groups' different ratings of factors for priority setting and should be examined with a view to removing impediments to collaboration in guidelines development.

In terms of what is actually happening in Canada, there is no reported use of analytic, data-intensive approaches. Consultation with members of guidelines-developing organizations was, however, frequently mentioned by the survey respondents. In contrast to the heavy emphasis on community and patient participation advocated by the IOM in the United States, the survey respondents appeared less concerned with community participation. Given the increasing casting of members of the public as partners and clients of the health care system, this absence of concern may undermine the impact of guidelines in Canada. The development of guidelines for which there is no receptive practitioner or patient audience will be of no long-term benefit, regardless of one's position in the health care system. Although differences in the health care systems and cultures of the two countries may limit the transferability of US experience to Canada, the IOM's framework, in which structured priority setting informed by quantitative analysis involves significant roles for community and patient input, is a useful one for Canadian developers.

Given the nature of the Canadian health care system and of the links that already exist among stakeholders through organizations such as the National Partnership for Quality in Health, there may be little gain from huge, new investments in the data collection necessary to implement a strictly analytic approach. Existing data, particularly administrative claims data, should be seriously considered for use in providing quantitative inputs for priority setting. Furthermore, purely analytic approaches appear to grow in importance only when it is impossible to include all involved parties in setting priorities.

In light of these observations, we propose the following framework for priority setting by Canadian groups developing clinical practice guidelines.

  1. Before selecting topics for guidelines, developers should consider consulting with members of their respective organizations, potential end users and other stakeholders. Meaningful patient and community involvement should be carefully considered.
  2. During these consultations thought should be given to what sort of topic is feasible given the resources of the developers, the conditions of practice into which the guidelines will be introduced and the likelihood that the guidelines will improve the health of the population and limit costs to the health care system. When available, quantitative data should be incorporated.
  3. Documentation of the process leading to a particular guidelines topic should be maintained, made available to members and other stakeholders and form the basis for evaluating the guidelines development process.
Canada's current macro-level regulation of health care spending provides an opportunity to stress quality of care over a pure cost-containment motive in guidelines development. Nevertheless, stakeholders with access to policy levers, particularly reimbursement, are likely to accelerate guidelines development and use, particularly if efforts to implement guidelines receive short shrift.

If macro-level mechanisms do extend to the level of individual physician practice (e.g., through internal markets for services or fund-holding physicians, as in Britain) an established system of guidelines would be an important forum for developing bases for funding and reimbursement, rather than directly intervening in the physician-patient relationship. In short, while our neighbours to the South struggle to get the "health-care-costs horse" back in the fiscal barn, we in Canada may still have time to work with the horses to ensure that they do what we want.

[Table of Contents]

References

  1. Carter AO, Battista RN, Hodge MJ et al: Report on activities and attitudes of organizations active in the clinical practice guidelines field. CMAJ 1995; 153: 901-907
  2. Woolf SH: Practice guidelines: a new reality in medicine. Arch Intern Med 1990; 150: 1811-1818
  3. Phelps CE, Parente ST: Priority setting in medical technology and medical practice assessment. Med Care 1990; 28: 703-723
  4. Phelps CE, Mooney C: Correction and update on "priority setting" in medical technology assessment. Med Care 1992; 30: 744-751
  5. Donaldson MS, Sox HC (eds), Institute of Medicine: Setting Priorities for Health Technology Assessment, National Academy Press, Washington, 1992
  6. Eddy DM: Selecting technologies for assessment. Int J Tech Assess Health Care 1989; 5: 485-501
  7. White LJ, Ball JR: The Clinical Efficacy Assessment Project of the American College of Physicians. Int J Technol Assess Health Care 1985; 1: 169-174
  8. Technology Advancement Center, University Hospital Consortium: Technology Alert [newsletter] 1994; 2 (5)
  9. Canadian Task Force on the Periodic Health Examination: The periodic health examination. CMAJ 1979; 121: 1193-1254
  10. Woolf SH: Practice guidelines, a new reality in medicine. II: Methods of developing guidelines. Arch Intern Med 1992; 152: 946-952
  11. Fisher AA, Miller R: Technical and other considerations in identifying operations research problem areas, selecting topics, and designing studies. Prog Clin Bio Res 1991; 371: 367-375

[Table of Contents]

Disclaimer

This guideline is for reference and education only and is not intended to be a substitute for the advice of an appropriate health care professional or for independent research and judgement. The CMA relies on the source of the CPG to provide updates and to notify us if the guideline becomes outdated. The CMA assumes no responsibility or liability arising from any outdated information or from any error in or omission from the guideline or from the use of any information contained in it.
GO TO CMAJ: Nov. 1, 1995   GO TO "Guidelines Workshop" articles
This page was last updated Jan. 8, 1996.