Checking random assignment with claims data
Online posting: March 11, 1997
Published in print: May. 1, 1997 (CMAJ 1997;156:1269)
Re: The review of randomization in the Canadian National
Breast Screening Study: Is the debate over?, by Dr. Norman F.
Boyd, CMAJ 1997;156:207-9 [full text / résumé].
Dr. Boyd refers to our analysis of health insurance data on
the Manitoba women enrolled in the National Breast Screening
Study (NBSS).[1] One of the objectives of our study, which was
funded by the National Cancer Institute of Canada, was to
determine whether there was independent evidence supporting
criticism of the random assignment procedures used in the NBSS.
In Manitoba, we could construct a medical history for each woman
before her entry into the trial with the use of a database
generated as part of that province's health insurance system. The
database includes claims data from the billing cards sent to
Manitoba Health by fee-for-service physicians. Each card must
include the reason for the visit (diagnosis) for the claim to be
accepted and the physician paid.
The NBSS provided identifying information for each of the 9477
women who attended the screening centre in Manitoba, which we
then linked with each woman's claims data. We retrieved all
procedures and diagnoses related to the breast in the 24 months
before each woman's entry into the NBSS. As Boyd notes, we
identified 9 women who had a claim for breast cancer during that
period. Of these 9 women, 3 had no other breast-related claims, 2
were referred for a mammogram, 2 had a biopsy and 2 had both a
mammogram and a biopsy. None had surgery or other form of cancer
treatment during the 24 months before their enrolment. Only 1 of
these women subsequently had breast cancer diagnosed and treated,
several years after the closure of the NBSS centre. We verified
this with files provided by the NBSS, which documented all cases
with a cancer diagnosis during the trial or the follow-up period.
We also determined that these women had continuous coverage under
Manitoba Health from 1980 to 1995, suggesting that none of them
had left the province or had been diagnosed or treated elsewhere.
Eight of the 9 women with a prior claim for breast cancer (4
of whom were in their 40s and 4 of whom were in their 50s) were
randomly assigned to the mammography arm of the NBSS. We reported
a probability value of 0.06 for this comparison, based on
Fisher's exact test. Boyd reanalyzed the data and found a
p value of 0.05, but he used the chi-square test. It
should be recognized, when interpreting these results, that the
study examined a large number of comparisons between women in the
2 arms of the trial. The finding of a p value close to
0.05 did not control for multiple comparisons and should be
treated with caution.
In interpreting these results, one needs to be aware of the
nature of the health insurance system in Manitoba. A tentative
diagnosis may be entered on a claims card as the reason for the
visit and any further diagnostic procedures. Even if the results
are negative -- as was presumably the case for these 9 women -- the
original claims card remains in the system. The physicians of
these women probably wrote "query breast cancer" or "rule out
breast cancer" on the claims cards.
Despite the limitations of the claims data, they were
generated independently from the NBSS and provide a unique method
of checking the randomization process used in this trial.
Marsha Cohen, MD
Clinical Epidemiology Unit
Sunnybrook Health Science Centre
North York, Ont.
Department of Community Health Sciences
University of Manitoba
Manitoba Centre for Health Policy and Evaluation
Winnipeg, Man.
Department of Health Administration
University of Toronto
Toronto, Ont.
mmcohen@ices.on.ca
Pat Kaufert, PhD
Department of Community Health Sciences
University of Manitoba
Winnipeg, Man.
Leonard MacWilliam, MNM, MSc
Manitoba Centre for Health Policy and Evaluation
Winnipeg, Man.
Robert Tate, MSc
Department of Community Health Sciences
University of Manitoba
Winnipeg, Man.
Reference
- Cohen MM, Kaufert PA, MacWilliam L, Tate RB. Using an
alternative data source to examine randomization in the Canadian
National Breast Screening Study. J Clin Epidemiol
1996;49:1039-44.
CMAJ email
CMAJ Readers' Forum