Bias inherent in retrospective waiting-time studies: experience from a vascular surgery waiting list [CMAJ

Bias inherent in retrospective waiting-time studies: experience from a vascular surgery waiting list

Boris Sobolev, Peter Brown, David Zelt, Samuel Shortt

CMAJ 2000;162(13):1821-2

See also:

Calculating waiting times retrospectively C. De Coster; A. Mariotto [Letters]

In the absence of waiting-time registries, administrative health care databases are often used to study how long it takes to receive medical services.^1,2 In these studies waiting time is ascertained retrospectively, from the date of service to some preceding moment.

In waiting-time analysis, however, retrospective and prospective approaches to data collection do not yield the same information. In retrospective design, the sampling unit is "patient received service." In prospective design, where waiting is evaluated in a cohort of patients added to a waiting list and followed forward in time, the sampling unit is "patient added to the list."

If every wait ended in the receipt of the service, the 2 designs would generate equivalent data. However, for a variety of reasons, some patients are removed from waiting lists without receiving the service.³ Any patient removed from the list before receiving the service would not be sampled in a retrospective study. In a prospective study an observation for a patient removed from the list without access to the service is considered "right censored," which indicates that the waiting time for that patient was less than it otherwise would have been.

If the "censored" observations are not accounted for, as in a retrospective design, the estimated probabilities of receiving the service may be biased toward a higher rate, and the median and mean waiting time may be underestimated.

To assess the magnitude of this bias, we used data for surgical waiting times collected prospectively at an acute care hospital in Ontario. We conducted waiting-time analysis in 2 ways: first, with data for all patients, as would be done in a prospective study, and then, with data only for those who actually underwent the surgery, as would be done in a retrospective study.

All patients accepted for elective vascular surgery in the Department of Surgery, Queen's University, Kingston, Ont., between 1994 and 1998, were eligible for the study. The surgical procedures included repair of abdominal aortic aneurysm, carotid endarterectomy, peripheral vascular bypass surgery and other surgery involving blood supply to the legs. The follow-up period ended 6 months after the last patients were added to the waiting list. Of 1084 consecutive cases, 985 patients received surgery, 14 were still waiting at the end of the follow-up period, and 85 were removed from the list without surgery for a variety of reasons: the patient's condition improved (19 patients), death occurred while the patient was awaiting surgery (3), the surgical risk became too great (38), or the patient decided against surgery (25).

To calculate the mean number of admissions from the list per week (i.e., the admission rate) we divided the total number of admissions by the total number of patient-weeks for the list. The probability of receiving surgery as a function of waiting time was estimated by the product–limit method.⁴

Fig. 1 shows the estimated probabilities of receiving surgery with data for the patients who had surgery (retrospective design) and for all patients added to the list (prospective design). The difference between these estimates, as measured by the log-rank test, was statistically significant (p < 0.001). As expected, the retrospective analysis produced estimates of the probability of undergoing surgery that were biased upward and underestimated the median waiting time. For this analysis, the mean weekly admission rate was 11.6 (95% confidence interval [CI] 10.9–12.3) per 100 patients, and the median time to admission was 6 (95% CI 5–6) weeks. In contrast, for the prospective design, the mean weekly admission rate was 9.8 (95% CI 9.5–10.1) per 100 patients, and the median time to admission was 7 (95% CI 6–7) weeks. These differences were statistically significant (p < 0.001). The mean (and standard error) time to admission was 8.6 (0.3) for the retrospective design and 10.2 (0.3) for the prospective design.

In other time-to-event studies, it has been suggested that retrospective design can lead to serious problems with inferences.⁵ Retrospective designs can similarly bias the results of studies of waiting times. For instance, coexisting illnesses could cause delay in surgery in a subpopulation and have no effect on the other patients in the cohort. If sicker patients are routinely removed from the waiting list before surgery, the impact of comorbid conditions might be missed by such a design.

Partial salary support for Dr. Sobolev came from the Institute for Clinical Evaluative Sciences, Toronto.

Competing interests: None declared.

Send a letter to the editor
Envoyez une lettre à la rédaction

Drs. Sobolev, Brown, Zelt and Shortt are with Queen's Health Policy Research Unit and Surgery Department, Queen's University, Kingston, Ont.

This article has been peer reviewed.

Reprint requests to: Dr. Boris Sobolev, Queen's Health Policy, 3rd floor Abramsky Hall, Kingston ON K7L 3N6; fax 613 533-6353; bs9@post.queensu.ca

References

DeCoster C, Carrière KC, Peterson S, Walld R, MacWilliam L. Waiting times for surgical procedures. Med Care 1999;37(6 Suppl):JS187-205.
Mackillop WJ, Groome PA, Zhang-Solomons J, Zhou Y, Feldman-Stewart D, Paszat L, et al. Does a centralized radiotherapy system provide adequate access to care? J Clin Oncol 1997;15:1261-71. [MEDLINE]
Lee A, Don B, Goldacre MJ. Waiting list statistics. II: An estimate of inflation of waiting list length. BMJ (Clin Res Ed) 1987;295(6607):1197-8.
Hosmer DW, Lemeshow S. Descriptive methods for survival data. In: Applied survival analysis: regression modeling of time to event data. New York: John Wiley & Sons; 1998; p. 27-86.
Weinberg CR, Baird DD, Rowland AS. Pitfalls inherent in retrospective time-to-event studies: the example of time to pregnancy. Stat Med 1993;12(9):867-79. [MEDLINE]