|
Canadian
Medical Association Journal Home |
Table of Contents Free eCMAJ TOC
Back issues |
Evidence-based medicine: a commentary on common criticisms
Abstract Discussions about evidence-based medicine engender both negative and positive reactions from clinicians and academics. Ways to achieve evidence-based practice are reviewed here and the most common criticisms described. The latter can be classified as "limitations universal to the practice of medicine," "limitations unique to evidence-based medicine" and "misperceptions of evidence-based medicine." Potential solutions to the true limitations of evidence-based medicine are discussed and areas for future work highlighted.
Evidence-based medicine has been defined1 as "the process of systematically finding, appraising, and using contemporaneous research findings as the basis for clinical decisions." The reader will immediately recognize that this is not a new process — clinicians have always striven to combine their clinical expertise and their patients' values with the best available evidence. However, interest in evidence-based medicine has grown exponentially since the coining of the term in the early 1990s1,2 (from 1 MEDLINE citation in 1992 to 2957 in February 2000) and has led to calls to increase the teaching of evidence-based medicine at the undergraduate and postgraduate levels.3 Evidence-based medicine is a multistep process (Table 1).4 Clinicians can incorporate evidence into their practices in 3 ways. First is the "doing" mode, in which at least the first 4 steps in Table 1 are carried out before an intervention is offered. Second is the "using" mode, in which searches are restricted to evidence sources that have already undergone critical appraisal by others, such as evidence-based guidelines or evidence summaries (thus skipping step 3 in Table 1). Third is the "replicating" mode, in which the decisions of respected opinion leaders are followed (abandoning at least steps 2 and 3). Of course, even clinicians trained to the "doing" level move back and forth between these modes, typically depending on whether they are dealing with clinical problems they encounter frequently or only rarely. Discussions about evidence-based medicine engender both negative and positive reactions from clinicians and academics, and this paper describes our efforts to categorize and respond to the most common criticisms.
Literature search
Criticisms of evidence-based medicine were systematically sought through an electronic literature search, from published surveys of front-line clinicians,56,7,8,9,10 and from the written records of questions posed during seminars held around the world from 1994 to 1999 by the director of the NHS Research and Development Centre for Evidence-Based Medicine in Oxford, United Kingdom (http://cebm.jr2.ox.ac.uk) (Dr. David Sackett, personal communication, 1999). MEDLINE was searched (without language restrictions) for articles published from 1966 to 1999 using the following search strategy: "evidence-based medicine" [MH] OR ("evidence-based" [TW] AND "medicine" [TW]) OR ("evidence" [TW] and "based" [TW] and "medicine" [TW]) AND "limitations" [MH] OR "criticisms" [MH] OR "limitations" [TW] OR "criticisms" [TW]. The titles and abstracts of the 95 articles identified (and the full text of the 47 felt to be potentially relevant) were reviewed by both of us for potential criticisms. Discrepancies (3 cases) were resolved by consensus. The reference lists of the retrieved articles were searched and experts in the field contacted in order to identify other relevant articles.
Criticisms expressed more than once (i.e., in 1 or more articles or at 1 or more seminars) were identified by content or qualitative analysis and included in this paper. The classification of criticisms was developed by both of us after a review of the criticisms that had been identified, with input from several members of the Evidence-Based Medicine Working Group (see Acknowledgements).
Limitations Our classification of the commonly cited limitations of evidence-based medicine appears in Table 2. The first 3 limitations outlined here are not unique to evidence-based medicine but are universally encountered in the practice of medicine. Shortage of coherent, consistent scientific evidence
Clinicians frequently encounter situations in which there is no relevant evidence from either basic or applied research.11 The exponential growth in clinical research, coupled with international efforts to identify, sort and rationalize this evidence systematically, will eventually close many of these gaps. However, until that time, clinical experience and reasoning (based on principles derived from basic scientific research) "must be applied to traverse the many grey zones of practice."11
Even when evidence exists, difficulties arise when it is inconclusive, inconsistent with previous studies, irrelevant to clinical realities or of poor quality.12 Indeed, in few research studies are the results reported in the context of the totality of available evidence.13 Although systematic reviews are a potential solution to this problem, inadequate attention to their methodology may lead to surprising variation in results and recommendations.14 Thus, steps must be taken to improve and standardize the methodology and reporting of systematic reviews.15,16
In our view, these problems, far from constituting a limitation of evidence-based medicine, highlight the importance of training clinicians to appraise research critically, to recognize the indeterminacy represented by confidence intervals and to apply the evidence, taking into account their patients' unique risks and values. Difficulties in applying evidence to the care of individual patients
The universal occurrence of biological variation hampers attempts to extrapolate evidence, whether from basic or applied research, to individual patients. Thus, we disagree with the criticism that this problem is unique to evidence-based medicine.17
To address this concern, researchers increasingly use particular study architectures (such as "N-of-1" and large, simple trials) and carry out judicious subgroup analyses that are intended to improve our ability to extrapolate research results to individual patients in the "real world."18,19,20 Furthermore, novel formats that enable clinicians to describe evidence to each other and to individual patients have been developed. For example, the number needed to treat and the number needed to harm have gained acceptance as useful means to make the evidence relevant to the individual patient.21,22 Patient values can be incorporated into these expressions by means of formal decision analysis or bedside simplifications such as the likelihood of being helped or harmed.23 Barriers to the practice of high-quality medicine
The gap between the demand for health care and the resources available to meet that demand is growing and results in clinicians having to care for more patients in less time.24 This pressure impairs the ability of clinicians to apply any evidence, whether from basic or applied science, to their patients.
Per capita health expenditures have more than doubled over the past 2 decades, and over one-third of this rise is owing to the increased intensity of services.25 Thus, it is not surprising that purchasers have increasingly attempted to control escalating health care costs by setting priorities and rationing services (explicitly as in the case of efficacious but expensive drugs like sildenafil in the United Kingdom26 or implicitly as in proposals to provide antihypertensive drugs only to patients deemed to be at high risk27).
Some have criticized evidence-based medicine for this curtailing of clinical freedom.28 However, this process was well underway before the elucidation of evidence-based medicine. Indeed, increased attention to the principles of evidence-based medicine among policy-makers and purchasers should lead to the preservation of funding for proven efficacious therapies and the elimination only of interventions that have been shown to be harmful or ineffective. The need to develop new skills
Unquestionably the practice of evidence-based medicine requires the acquisition and development of new skills (in literature searching and critical appraisal). Their mastery and application are formidable tasks and should not be underestimated. However, the assertion of some critics that clinicians are not interested in learning such skills is contradicted by surveys of practising clinicians.5,6,7,8,9,10
Evidence-based medicine skills can be acquired at any stage in clinical training. Incorporating their acquisition into the routine of grand rounds, postgraduate and undergraduate seminars, and "morning report" integrates them with the other skills being developed in these settings.29 Members of clinical teams at various stages of training can collaborate by sharing the searching and appraising tasks. The different skills required for practising in the "using" and "doing" modes can be learned in sequence, thus avoiding learner overload. Indeed, for many clinicians the most appropriate means to achieving evidence-based practice may be through the "using" mode outlined earlier in this article. Limited time and resources
Critics of evidence-based medicine have correctly pointed out that its practice may require time and resources unavailable to the busy clinician.30
Important developments to help overcome this barrier include the systematic reviews generated by the Cochrane Collaboration, the growing numbers of evidence-based journals (such as ACP Journal Club) containing abstracts of quality- and relevance-filtered studies, and the creation of "best evidence" sections in a number of established journals. Moreover, electronic searching is increasingly being made available at the point of care, cutting time of access to the evidence to a few seconds.31 The generation of databases of critically appraised topics (1-page summaries of evidence relevant to common clinical questions),32 which can be quickly accessed at the point of care,31 represent another time- and energy-saving solution for busy clinicians, as is the division of labour between members of the clinical team noted previously. Finally, although we can generate several questions for each patient we see (and become paralyzed by trying to address them all), we can pare them down to just one by balancing the question that would be most important to our patient's well-being against that which may be answered most easily, that which is most interesting to us and that which is most likely to be raised by subsequent patients.4 Paucity of evidence that evidence-based medicine "works"
Although agreeing that evidence-based medicine makes good sense in theory, its critics have quite appropriately demanded evidence for whether it improves patient out-comes.33
No such evidence is available from randomized trials because no investigative team has yet overcome the problems of sample size, contamination and blinding that such a trial raises. Moreover, it is questionable whether withholding access to evidence from the control arm in such a trial would be ethical. However, outcomes researchers consistently document that patients who receive proven efficacious therapies have better outcomes than those who do not.34,35,36
Given this evidence, the focus has shifted from whether to teach evidence-based medicine to how to do so, and recent randomized trials have compared alternative strategies for enhancing evidence-based practice. These trials have both discredited traditional approaches such as didactic lectures and validated newer approaches such as academic detailing (one-on-one educational sessions with a content expert) and seeking advice from local opinion leaders.37
Misperceptions
Many criticisms of evidence-based medicine stem from misperceptions or misrepresentations and may be answered by careful consideration of the definition of evidence-based medicine and the 5 steps outlined in Table 1.4 We include these misperceptions in Table 2 in order to clarify that they represent only pseudolimitations of evidence-based medicine.
For example, criticisms that evidence-based medicine denigrates clinical expertise,17,38 ignores patients' values39 or promotes "cookbook medicine"38,40 arise because of a failure to appreciate step 4 in Table 1.41 Moreover, because evidence-based medicine is cost-indifferent and directed toward maximizing the quality of life of individual patients, it may (and often does) result in policies that will increase, rather than decrease, costs (consider the provision of statin drugs for normocholesterolemic patients following myocardial infarction).42,43 The most commonly cited pseudolimitation is that evidence-based medicine is an ivory-tower concept;44 however, surveys and audits of front-line clinicians clearly refute this claim.5,6,7,8,9,10,45,46,47,48,49,50 Furthermore, a common misperception is that evidence-based medicine is limited to doing, as opposed to using, clinical research.17 Although a minority of practitioners of evidence-based medicine also do research, its practice is a method for providing care for patients, not a method for performing research.
The final misperception is that only randomized trials or systematic reviews constitute the "evidence" in evidence-based medicine.44,51 Even the most vehement protagonist of evidence-based medicine would acknowledge that several sources of evidence may inform clinical decision-making. However, the practice of evidence-based medicine stresses finding the best available evidence to answer a question, and hierarchies of evidence have been developed to help describe the quality of evidence that may be found to answer various questions. Thus, randomized clinical trials are usually considered the "gold standard" for establishing the effects of an intervention, but they are not the best sources for answering questions about diagnosis, prognosis or harm. Although this hierarchy has been criticized for devaluing the basic sciences,51 we would submit that numerous studies over the past 4 decades have demonstrated the potential fallibility of extrapolating directly from the bench to the bedside, without the intervening step of proving the assumptions to be valid in human subjects.52,53,54
Conclusion Evidence-based medicine, like other models of care,55 has limitations, and further innovation and study are required to resolve the issues raised in this paper. In particular, efforts need to be directed toward improving clinicians' access to evidence at the point of care; developing better methods of describing evidence to patients in order to facilitate shared decision-making; and conducting studies to test whether and how evidence-based medicine affects processes of care and patient outcomes.
Competing interests: None declared for Dr. Straus. Dr. McAlister has received speaker fees from Merck Frosst Canada & Co.
Contributors: Both authors took part in the conception, design and analysis of the study and wrote the first draft of this paper. Dr. McAlister wrote subsequent drafts, and both authors revised them for their intellectual content.
Acknowledgements: We thank Drs. David Sackett, Ian Chalmers, Scott Richardson, William Rosenberg, Brian Haynes and Gordon H. Guyatt for their helpful comments on earlier versions of this manuscript.
Financial support was received from the Alberta Heritage Foundation for Medical Research, Edmonton, Alta.
Dr. Straus is with the Division of General Internal Medicine, Mount Sinai Hospital, Toronto, Ont., and Dr. McAlister is with the Division of General Internal Medicine, University of Alberta Hospital, and is Population Health Investigator, Alberta Heritage Foundation for Medical Research, Edmonton, Alta. This article has been peer reviewed. Correspondence to: Dr. F. McAlister, Division of General Internal Medicine, 2E3.24 WMC, University of Alberta Hospital, 8440 112 St., Edmonton AB T6G 2R7; fax 780 407-2680; Finlay.McAlister@ualberta.ca
References
Copyright 2000 Canadian Medical Association or its licensors |
