Canadian Medical Association Journal 1995; 153: 783-786
[résumé]
Klein and associates' conscientious and thoughtful report on the relation between physician beliefs and the use of episiotomy (see pages 769 to 779 in this issue) augments their earlier work on episiotomy and postpartum outcomes.[1,2] Their results constitute a persuasive argument for more restricted use of episiotomy. Their studies also present fascinating methodologic implications.
The original study[1] was a randomized controlled trial (RCT). Unfortunately, its usefulness was compromised by poor physician compliance with the trial protocol. Depending upon the random allocation of patients to the "treatment" or "control" study arms, contributing physicians were to have followed a policy of either liberal or restrictive use of episiotomy. However, Klein and associates detected that many did not alter their use of episiotomy as required and employed episiotomy approximately 90% of the time for participants in either trial arm. Not surprisingly, the culprits turned out to be those who viewed episiotomy favourably.
Advances in health care depend upon the knowledge gained from unbiased studies. Because RCTs provide the only hope of eliminating selection biases from investigations, they serve as the foundation for advancing medical science.
However, Klein and associates' work exemplifies another side of RCTs: they are anathema to the human spirit. Researchers need to realize that, given the opportunity, trial implementers will frequently subvert the intended aims of random assignment. Subversion can be prevented or deflected, however, with painstaking attention to design and implementation.
Their RCT was, however, seriously affected by physician noncompliance with the randomly assigned therapy. Physicians who viewed episiotomy favourably or very favourably performed the procedure with similar frequencies regardless of the trial arm to which the participants had been assigned. Obviously, trials in which so much therapeutic contamination occurs are of limited value. Hence, Klein and associates subjected their results to further research and analysis. Because the results of their post-hoc studies are perhaps more meaningful than those of the original trial, one might surmise that doing an RCT under such circumstances wastes time, energy and money. Why not just begin with a nonrandomized, observational study?
Part of the answer resides in the fact that the investigators did not expect to encounter such levels of noncompliance. This point aside, however, we should be circumspect about reflex reactions to nonrandomized studies. They have certain characteristics that lead to biases of some degree. Moreover, lack of empirical research into the quality standards for observational studies makes their conduct and interpretation fraught with difficulty. In contrast, RCTs are demonstrably less susceptible to bias than nonrandomized studies[3-10] and apply quality standards that have been extensively evaluated.[3,4,7-20] Granted, observational studies are useful and necessary when an RCT proves impossible. When RCTs are feasible, however, they should be preferred. The conditions under which Klein and associates conducted their trial strained the limits of feasibility.
Whether selection bias of this kind is introduced hinges largely upon the adequacy of the random-allocation process. Random allocation in a trial should involve both generating an unpredictable assignment sequence and concealing that sequence until participants are actually allocated. Many medical researchers inaccurately regard the sequence-generating process as "random allocation" and overlook the allocation concealment process, which is perhaps the more important of the two.[13]
Allocation concealment should prevent anyone involved in the trial from having foreknowledge of a treatment assignment up to the point of allocation.[14] Crucially, concealment prevents those who admit patients to a trial from knowing the upcoming assignments.
Breaches of allocation concealment probably happen more often than we suspect. Only about one quarter of trials report even the most minimal of allocation concealment procedures.[14,16] Moreover, my colleagues and I have found that trials in which the allocation sequence had been inadequately concealed yielded larger estimates of treatment effects (odds ratios were exaggerated by 30% to 40% on average) compared with trials in which authors reported adequate allocation concealment.[13] This result provides empiric evidence of bias in trials with inadequate allocation concealment. Furthermore, many residents and junior faculty will admit to deciphering, or witnessing someone else deciphering, an assignment scheme.[21] Although most published RCTs probably provide reliable results, I believe allocation breaches to be more than a rare occurrence.
Klein and associates' description of allocation concealment[1] surpassed the concealment descriptions provided in most of the reports my colleagues and I have reviewed.[14,16] Yet they used envelopes that arguably are, in general, susceptible to deciphering.[22] "The envelopes were opaque, sequentially numbered, and contained instructions printed on opaque cards."[1] The opaque envelopes and opaque cards would have effectively prevented transillumination of the envelopes, a frequently cited problem.[21] However, Klein and associates did not specify that the envelopes were sealed, and unsealed envelopes have led to deciphering in other trials.[21]
Even with adequate concealment, some deciphering of sequences can occur. Klein and associates' trial could not be blinded, and thus the assignments became known after allocation. Unblinded treatments, in particular, require an unpredictable assignment sequence. If a sequence is predictable (e.g., following an ABAB pattern or consisting of short, fixed blocks), a sequence can be deduced from the order of the past assignments, and physicians can foresee all or some of the upcoming assignments. Because Klein and associates did not report how they generated the assignment sequence we do not know whether it was unpredictable.[1] A random-number table or generator should have played a role. We may perhaps infer that some form of blocking was used, since "study envelopes were prepared separately by parity in each hospital."[1] If the blocks were of fixed size, and especially if they were small (six patients or fewer), the sequence may have been too predictable. In this case, the block size could have been deciphered and selection bias introduced, whatever the effectiveness of the allocation concealment.
Although these considerations may appear picayune, seemingly trivial issues have been implicated in the sabotaging of trials.[21] The methodologic details of random assignment require diligent attention. I presume that the assignment envelopes in Klein and associates' trial were sealed. Given the other potential problems with allocation concealment and sequence generation, however, I do not know, and perhaps the investigators themselves do not know, whether their assignment sequence was partly deciphered. My guess is that any biases that entered their trial favoured episiotomy. If this is the case, it only strengthens their conclusions as to the lack of benefit of episiotomy.
Under the threat of physician noncompliance several actions could increase the likelihood of obtaining reliable results. First, investigators could conduct preliminary research on whether physician compliance might be a problem. The results of this preparatory work could prompt changes to the trial design, foster enhancements to trial implementation or scuttle the use of a randomized design. Second, investigators need to give participating physicians and other collaborators extensive training in the trial protocol. Although this is frequently done, investigators may not devote enough time and effort to protocol training. Third, investigators should educate those involved in implementation on the fundamentals of RCT methodology. People who implement trials sometimes undermine them without knowing it.[21] Last, more effort should be devoted to monitoring compliance with the research protocol once allocation has occurred. Recalcitrant contributors could be instructed to adhere to the protocol or be excluded from further involvement (although their data would be used in the analysis up to that point, of course). This step would have minimized the impact of noncompliance on Klein and associates' results.
The type of monitoring I am espousing could be carried out when data come into a central trial office. None of these recommendations would entail expensive on-site monitoring. Rather, I advocate that, in general, trials be simplified in order to permit larger samples to be used. Nevertheless, some on-site auditing may be necessary.[23] With regard to random allocation, we must acknowledge the human factors that influence this important scientific process. Educating those who implement trials in the rationale for random allocation is an important first step, but a surprisingly difficult one. Importantly, the scientific community, particularly granting agencies and journals, should insist upon adequate sequence generation, adequate allocation concealment and the reporting of both.[11,14,16,21] No excuse should be accepted for a failure to meet these requirements.
If investigators use envelopes for allocation concealment, these should be sequentially numbered, opaque and sealed. Moreover, the investigators should ensure that the envelopes are opened sequentially, and only after they are inscribed with the participating patient's name and other details.24 I also recommend using pressure-sensitive or carbon paper inside the envelope to copy the details onto the allocation record and thus create a valuable audit trail.