What can comparisons of mortality rates tell us about waiting lists?

David C. Hadorn

CMAJ 2000;162:794-5

To monitor performance and quality of care, it is necessary to know whether or not the mortality among patients waiting for CABG is excessive in view of the considerable and continuing mortality among patients with severe cardiac problems, even after CABG. One reasonable approach to answer such a question would be to ask well-informed, independent clinicians to scrutinize the circumstances surrounding each patient's death, using standardized review criteria, and to determine on a case-by-case basis the likelihood of survival (for some defined period of time) had the patient received the treatment "in time." Reviews of this kind are in fact the gold standard from a clinical perspective, but they are cumbersome and expensive and seem rather subjective from the perspective of most laypeople, including politicians.

In this issue (page 775) David Naylor and colleagues explore a potentially more efficient method for assessing the "reasonableness" of mortality rates among patients on CABG waiting lists.¹ The authors borrow an approach used in mortality studies involving patients in other clinical contexts, including those on general medical wards or in intensive care units. To my knowledge, this is the first attempt to apply such a method to the assessment of mortality rates among patients on waiting lists.

The premise is simple: details of individual cases are subsumed into group-level mortality rates, which are then compared with rates obtained in a comparison group. Provided adequate control is maintained over other possible relevant differences between the 2 groups, such comparisons can be meaningful and illuminating.

However, opportunities for inappropriate and misleading comparisons abound, as officials in the US Medicare program discovered in 1987 when they released raw mortality data for hospitals participating in the program.^2,3 Mortality rates, which were widely reported in the press, ranged from 0% to 100%, with no allowance for differences in case mix. The hospice facilities whose names were associated with 100% mortality were rightly upset over the attending publicity.

Since then, the science of comparing mortality rates has advanced considerably, particularly with respect to techniques for adjusting for differences in case mix. These techniques largely depend on statistical models that incorporate clinical factors known to be significantly correlated with mortality risk, such as mental status, blood pressure and serum urea nitrogen level. Data on these factors are collected for all patients, and raw group mortality rates are adjusted for case-mix differences detected by these factors. In this way, apples can be compared with apples. Unfortunately, as with case-by-case reviews by experts, extracting information from individual patient records is both costly and time consuming.

Perhaps for this reason Naylor and colleagues took a small leap of faith and dispensed with adjustments at the individual patient level (except for age and sex), relying instead on the perceived reasonableness of a carefully selected comparison group. The idea was to find a comparison group that is sufficiently similar to the index group to warrant the comparison without the need to engage in case-by-case data abstraction or analysis.

To this end, the authors compared patients waiting for CABG with a group of patients who had survived 6 months after an acute myocardial infarction (AMI). By 6 months the post-AMI group had reached a plateau in mortality risk but were still at increased risk relative to their age and sex cohorts. As such, both groups of patients were deemed to have clinically significant coronary artery disease that was relatively stable at the time. Whether the 2 groups were therefore comparable in mortality risk is unknown, although it would perhaps be surprising if they differed greatly. And indeed they did not. The standardized mortality ratio among the patients waiting for CABG indicated that they were almost 3 times as likely to die as age- and sex-matched control subjects in the general population. Patients in the post-AMI cohort were almost 4 times as likely to die as those in the general population. From this perspective, the mortality rate among patients queuing for CABG does not seem too excessive. Indeed, fans of the Canadian health care system will be comforted to learn that the risk of death in the CABG-queuing cohort is lower than that in the group of relatively stable post-AMI patients, although this difference could turn the spotlight on post-AMI care throughout the province.

Is the comparison valid? I think so, but only insofar as the standardized mortality ratios of the 2 groups show that patients in Ontario with moderate to severe heart disease have a 3- to 4-fold increase in the risk of death relative to age- and sex-matched cohorts in the general population. Although the large number of patients in the study (21 220 in each group) ensured that virtually any difference in observed mortality rates between the 2 groups would be statistically significant, it is unclear whether the observed difference is clinically meaningful. Nevertheless, perhaps the most important finding for beginning the benchmarking process is that the difference in mortality between the 2 groups was not large. As more comparison data become available, the resulting distributions of mortality rates will indicate where the benchmarks of quality should be set.

It seems worthwhile to explore further the idea that comparing mortality rates among patients waiting for CABG with observed rates in other groups might provide useful information. It would of course make sense to compare rates between groups waiting for CABG, both within Canada and internationally; however, more far-reaching comparisons might also be instructive: for example, rates among patients waiting for cataract surgery or hip replacement. Such patients have a certain background level of heart disease, so including them could indicate the extent to which coronary artery disease amenable to CABG is responsible for excess mortality. This sort of comparison might be more of an "apples and pomegranates" comparison, but I for one would be interested in seeing the data.

The extent to which mortality rates emanating from such studies can be meaningfully interpreted without patient-level adjustment remains to be seen. Certainly practitioners and hospitals are not going to accept criticism based on elevated mortality rates until and unless adequate consideration is given to case mix. But the difficulties and expense inherent in making patient-level case-mix adjustments, as I have described, are reason enough to continue pursuing other options. Naylor and colleagues are to be commended for pointing us in an intriguing direction.

Competing interests: None declared.

Send a letter to the editor Envoyez une lettre à la rédaction

Correspondence to: Dr. David C. Hadorn, 1801 Shasta St., Richmond CA 94804, USA; fax 510 527-6565; hadorn@dnai.com

1. Naylor CD, Szalai JP, Katic M. Benchmarking the vital risk of waiting for coronary artery bypass surgery in Ontario. CMAJ 2000;162(6):775-9.
2. Report — Medicare hospital mortality rates [press release]. Washington (DC): Health Care Financing Administration; 1987 Dec 16.
3. Dubois RW, Rogers WH, Moxley JH, Draper D, Brook RH. Hospital inpatient mortality: Is it a predictor of quality? N Engl J Med 1987;317:1674-80. [MEDLINE]

© 2000 Canadian Medical Association or its licensors