Disease-simulation models and health care decisions [CMAJ

Disease-simulation models and health care decisions

J. Jaime Caro

CMAJ 2000;162:1001-2

See also:

The clinical burden of prostate cancer in Canada [Research]
The economic burden of prostate cancer in Canada [Research]

Not long ago medical decisions were made by physicians — often in consultation with their patients — unfettered by external demands to consider whatever scientific evidence there might be and even less by economic constraints beyond those of the patients themselves. Shifting of the financial burden through insurance, coupled with explosive growth in the availability of medical interventions, the number of publications about them and the costs of health care, has disrupted the traditional model, raising concerns among administrators and other stakeholders. For these new parties to medical decision-making, ad hoc informal consideration of fragmentary evidence and outcomes by individual practitioners is insufficient; the best available data must be fully integrated to arrive at a comprehensive set of outcomes.¹

The 2 articles in this issue by Steven Grover and colleagues addressing the clinical and economic burden of prostate cancer are examples of an approach taken in an attempt to satisfy these new demands (page 977 and page 987).^2,3 The authors use the Montreal Prostate Cancer Model, a mathematical model they have created into which relevant aspects of the disease process are entered. A disease-simulation model is merely a quantitative framework that expresses the relations between the factors and outcomes of interest. At its simplest the model can be just a single expression: the risk of disease A increases by X in the presence of factor B. This very basic model cannot answer many questions, however. What if there are other factors? A different time period? What happens to life expectancy? Quality of life? Costs? Thus, models tend to get more complicated as they are developed to combine data from many sources to produce detailed estimates of the clinical and economic consequences of one particular management strategy or another.

Can this approach meet the new needs? In principle, it can do this quite well for populations (less so for individual patients). As Grover and colleagues note, these models provide a means of using imperfect data to inform decisions and can guide future research. A major weakness of this approach is in its reporting. Despite splitting their material into 2 articles, the authors do not describe the model in much detail, even though they acknowledge that it is so complex that confidence intervals could not be provided. This is not the authors' fault; the journal limits article texts to about 2500 words and is not alone in doing so. The British Medical Journal used more words to provide reporting guidelines than it allots to the reports themselves.⁴

Why is this a weakness? Is parsimony not one of the hallmarks of good science? The problem is that these models do not reflect an individual study with a single objective but, rather, a collection of investigations, each with its own purpose and methods. Grover and colleagues have to report numerous studies: of the rate of progression of prostate cancer in relation to various (mostly tumour) determinants; of the effect of various treatments on these rates; of the survival under several conditions; of the current prevalence of disease states and treatments; of the resource use associated with various management strategies; of the unit costs of those resources; and so on. In addition, the model structure and analytic techniques themselves must be explained. Each of these investigations could well occupy a full paper on its own, but even the most munificent editors would be disinclined to dedicate an entire issue to a model. In the absence of such detail, it is impossible to fulfil an even more important requirement for good science: reproducibility.

What, then, should a prudent decision-maker do? Believe in the model and its associated investigations (bolstered, perhaps, by its ability to come close to the results of other studies)? Reject it and demand that the issues be settled by a clinical trial, however impossible its conduct would be? Ignore it and trust intuition?

I don't think any of these are acceptable choices in the long run. Instead, models must become public domain, open for all to scrutinize. This would prove difficult to put into effect, because these models generally reflect the investment of many individual and corporate interests who will be loathe to part with the asset. Although rarely sold to others, these models are an asset because they are the basis for extensive publication, which would attract more research funding and would further academic — or commercial in corporate circles — promotion. Moreover, unrestricted access alone will not solve the problem. The modellers would have to invest additional effort to render the models evaluable, and evaluators would have to invest considerable effort (measured in months, not hours) to assess properly what has been done. Thus, a registry or central repository of models will not be the remedy.

A more productive alternative might be for society to invest in these models collectively. Thus, instead of the Montreal Prostate Cancer Model, for example, there would be the Medical Research Council of Canada (or other such organization) Model. Such models could be created by the same investigators but with the explicit purpose of the models becoming a public tool reflecting the most up-to-date knowledge. Documentation would not be a brief article in a medical journal but, rather, a full report (book length) published by the sponsoring organization. The model, not the report, would be expressly evaluated by other experts and over time would be added to and modified by others (overseen by an appropriate task force) as new data or concepts emerge. Many investigators, including those with commercial interests, could use the up-to-date model for diverse purposes — some of which would justify publication in medical journals, where the focus would be the specific narrower topic rather than the entire model and its objective.

Not only would this eliminate the plethora of irreproducible models often making conflicting claims for decision-makers' attention, but it would greatly increase the likelihood of bringing accumulated evidence to bear in the improvement of the efficiency of medical practice.

Competing interests: None declared.

Send a letter to the editor
Envoyez une lettre à la rédaction

Dr. Caro is Scientific Director of Caro Research in Concord, Mass.

References

Gold MR, Siegel JE, Russell LB, et al, editors. Cost-effectiveness in health and medicine. New York: Oxford University Press; 1996.
Grover SA, Coupal L, Zowall H, Rajan R, Trachtenberg J, Elhilali M, et al. The clinical burden of prostate cancer in Canada: forecasts from the Montreal Prostate Cancer Model. CMAJ 2000;162(7):977-83.
Grover SA, Coupal L, Zowall H, Rajan R, Trachtenberg J, Elhilali M, et al. The economic burden of prostate cancer in Canada: forecasts from the Montreal Prostate Cancer Model. CMAJ 2000;162(7):987-92.
Dummond MF, Jefferson TO, on behalf of the BMJ Economic Evaluation Working Party. Guidelines for authors and peer reviewers of economic submissions to the BMJ. BMJ 1996;313:275-83. [MEDLINE]