Canadian Medical Association Journal 1996; 155: 1277-79
Paper reprints may be obtained from: Dr. Patricia Huston, CMAJ, PO Box 8650, Ottawa ON K1G 0G8; fax 613 523-0937; pubs@cma.ca
© 1996 Canadian Medical Association
One of the most sobering findings of research that synthesizes and analyses the work of others is that much research is flawed and poorly reported. Anyone who has attempted to review medical literature knows that one is often comparing apples and oranges. It is very difficult to combine the results of research when they are reported incompletely and in noncomparable ways. Even the most powerful research design, the randomized controlled trial (RCT), has been plagued with design and reporting difficulties.[1]
Trialists may counter this criticism by noting that there has been little consensus on how best to report a trial and that existing recommendations are based on opinion, not evidence. Until recently this has been true. Now, evidence-based recommendations for the reporting of RCTs are available. The Consolidation of Standards for Reporting Trials (CONSORT) statement[2] represents years of research and consensus-building by epidemiologists, biostatisticians, trialists and medical editors. Initially their task was addressed by two independent working groups;[3,4] it is much to the credit of members of both groups and the facilitator of the blended group, David Moher of the Ottawa Civic Hospital Loeb Research Institute, that these separate initiatives were combined.
The core contribution of the CONSORT statement consists of a flow diagram (Fig. 1) and a checklist (Table 1). The flow diagram enables reviewers and readers to quickly grasp how many eligible participants were randomly assigned to each trial arm, how many withdrew or were lost to follow-up and how many actually completed the trial. Such information is frequently difficult or impossible to ascertain from trial reports as they are currently presented. The checklist identifies 21 items that should be incorporated in the title, abstract, introduction, methods, results or conclusion of every RCT.
The goal of the CONSORT statement is to improve the internal validity of RCTs. The CONSORT group believes that the quality of trials can be revealed and eventually improved by a standardized reporting process if standards for reporting are based on evidence. Accordingly, it conducted a review of "research on research" to identify factors that influence trial results. Thus, almost 75% (15/21) of the items on the CONSORT checklist are based on empirical evidence. For example, Schulz and associates have shown that trials in which the method of concealing treatment allocation was inadequate or unclear reported exaggerated treatment effects compared with trials in which concealment was adequate and clearly reported.[15] Although the precise method of concealment has often not been reported in the past, it will be now -- and greater attention will undoubtedly be given to this element of blinding during the conduct of RCTs as well.
Items included in the CONSORT checklist for which no empirical evidence was available were selected on the basis of common sense, using a modified Delphi process to establish consensus. One example of items in this category is the recommendation to state, whenever feasible, absolute numbers rather than percentages. This is already generally accepted practice.
The editors of CMAJ commend and endorse the CONSORT statement. Along with many other journals, including The Lancet and the Journal of the American Medical Association, we will be adopting its recommendations as editorial policy. The checklist and diagram are not simply for editors to endorse; they are primarily intended for authors to use. A flow diagram and completed checklist noting the manuscript page on which each item is addressed should be enclosed with every RCT report submitted for publication.
The CONSORT statement is not a panacea for all the reporting difficulties of RCTs. Although it takes a huge step toward the improvement of the internal validity of single trials it does not address important difficulties caused by multiple publication of the results of multicentre trials. Therefore we propose a further recommendation: that all authors reporting the results of an RCT note whether their trial is related to a larger, multicentre trial and if their trial (or any portion thereof) has been reported elsewhere. It is through standardized and transparent reporting of findings that the closest approximation of the truth will be gleaned from current research.
We have begun to work with authors of completed trials to help them follow the CONSORT standards to the greatest extent possible; the report by Nickel and colleagues of their trial findings (see pages 1251 to 1259 of this issue [abstract]) represents the first such effort. Conformation to all requirements of the CONSORT statement checklist will take time as adjustments are made to trial protocols and data management; we acknowledge and accept this. Nonetheless, we anticipate that the efforts of researchers to apply the CONSORT standards will be well rewarded. As major medical journals adopt these standards, not only is the quality of investigation and reporting in RCTs bound to improve, but the development of guidelines and the conduct of systematic reviews and meta-analyses will be facilitated. The exemplary initiative of the CONSORT group deserves not only wide-based support but also rigorous follow-up and evaluation.