Development and validation of the Osteoporosis Risk Assessment Instrument to facilitate selection of women for bone densitometry

Suzanne M. Cadarette,*†‡ Susan B. Jaglal,*†§¶** Nancy Kreiger,*††† Warren J. McIsaac,‡‡ Gerarda A. Darlington,‡§§ Jack V. Tu,†¶**¶¶

CMAJ 2000;162(9):1289-94

• Abstract
• Introduction
• Methods
• Results
• Interpretation
• References

Background: Although mass screening for osteoporosis is not recommended among postmenopausal women, there is no consensus on which women should undergo testing for low bone mineral density. The objective of this study was to develop and validate a clinical tool to help clinicians identify which women are at increased risk for osteoporosis and should therefore undergo further testing with bone densitometry.

Methods: Using Ontario baseline data from the Canadian Multicentre Osteoporosis Study, we identified all cognitively normal women aged 45 years or more who had undergone testing with dual-energy x-ray absorptiometry (DXA) at both the femoral neck and the lumbar spine (L1–L4). Participants who had a previous diagnosis of osteoporosis or were taking bone active medication other than ovarian hormones were excluded. The main outcome measure was low bone mineral density (T score of 2 or more standard deviations below the mean for young Canadian women) at either the femoral neck or the lumbar spine. Logistic regression analysis and receiver operating characteristic (ROC) analysis were used to identify the simplest algorithm that would identify women at increased risk for low bone mineral density.

Results: The study population comprised 1376 women, of whom 926 were allocated to the development of the tool and 450 to its validation. A simple algorithm based on age, weight and current estrogen use (yes or no) was developed. Validation of this 3-item Osteoporosis Risk Assessment Instrument (ORAI) showed that the tool had a sensitivity of 93.3% (95% confidence interval [CI] 86.3%–97.0%) and a specificity of 46.4% (95% CI 41.0%–51.8%) for selecting women with low bone mineral density. The sensitivity of the instrument for selecting women with osteoporosis was 94.4% (95% CI 83.7%–98.6%). Use of the ORAI represented a 38.7% reduction in DXA testing compared with screening all women in our study.

Interpretation: The ORAI accurately identifies the vast majority of women likely to have low bone mineral density and is effective in substantially decreasing the need for all women to undergo DXA testing.

[Contents]

DXA is the fastest growing single test in medicine. In Ontario the number of bone density tests increased from 34 402 in 1992 to 165 630 in 1997.¹⁵ At present there is no clear method for deciding who should undergo DXA testing¹⁶ because guidelines are not sufficiently precise to allow clinicians to decide which women warrant further testing.^8,9,11,17 In effect, physicians may have little choice but to test all women around the time of menopause. The objective of this study was to develop and validate a simple screening tool to assist physicians in selecting patients for bone densitometry.

[Contents]

The Canadian Multicentre Osteoporosis Study (CaMos) is a 5-year cohort study evaluating the relation between risk factors for osteoporosis, measures of bone integrity and osteoporotic fracture.¹⁸ In brief, an age-, sex- and region-stratified random sample of the Canadian population was selected using a telephone-based sampling frame. Included in the CaMos population were noninstitutionalized people 25 years of age and older residing within 50 km of one of the 9 study centres (Vancouver, Calgary, Saskatoon, Hamilton, Toronto, Kingston, Quebec City, Halifax and St. John's). Native populations residing in the northern regions of the country were excluded. Participants had to be fluent in at least English or French or, in Toronto and Vancouver, Chinese. The collection of baseline data began in February 1996 and ended in September 1997. Eligible subjects were invited to meet with a trained interviewer to complete a standardized questionnaire and visit the clinic for bone densitometry. Data were collected on risk factors for osteoporosis, including age, race or ethnic background, personal and family history of bone fragility or fractures, medical history (e.g., comorbid conditions, reproductive history), current and past medication use, anthropometric information and lifestyle factors (dietary intake of calcium and vitamin D, alcohol consumption, smoking, physical activity and sunlight exposure).

For our study we used CaMos baseline data from the 3 Ontario centres. Cognitively normal women (Mini-Mental State score greater than 20¹⁹) who had undergone DXA testing at both the femoral neck and the lumbar spine (L1–L4) were eligible. We excluded those with a diagnosis of osteoporosis or taking bone active medications other than ovarian hormones (calcitonin, bisphosphonates or fluoride) at the time of study. About two-thirds of the eligible women were randomly allocated to the development of the assessment tool and the remainder to its validation.²⁰ The preliminary analysis revealed that the prevalence of osteoporosis was less than 1% among women aged less than 45 years; therefore, we restricted our study population to women aged 45 or more.

BMD was measured using QDR 1000 DXA machines (Hologic, Inc., Bedford, Mass.) in Kingston and Toronto and a Lunar DPX Alpha machine (Lunar Corporation, Madison, Wis.) in Hamilton. To permit pooling of BMD measurements across study sites, CaMos standardized BMD values to Hologic equivalents.²¹ Hologic-equivalent BMD values were used by CaMos to determine the normal BMD values for young Canadian women (unpublished observations). These data served as the reference T score in our study. A low BMD at either the femoral neck or the lumbar spine is clinically relevant for deciding about prophylactic treatment to prevent osteoporosis and fragility fractures.¹¹ Most patients with osteoporotic fractures have a BMD T score that is 2 standard deviations (SDs) below the mean for young adults.^17,22 A value of 2 or more SDs below the mean has been referred to as the fracture threshold.^22,23 In addition, the US National Osteoporosis Foundation promotes pharmacological therapy to reduce the risk of fracture among women with a BMD value more than 2 SDs below the mean for young normal adults.⁸ We chose a BMD value of 2 or more SDs below the mean for young Canadian women at either the femoral neck or the lumbar spine as the main outcome measure for our study.

Information on risk factors was obtained from responses to the CaMos questionnaire. Logistic regression analysis was used to evaluate the relation between each risk factor and a low BMD at the femoral neck and at the lumbar spine separately. Backward selection and stepwise approaches were used in model building.²⁴ An effort was made to maximize predictive performance using variables that can be easily determined in clinical practice.^20,25 Variables that best predicted low BMD of the femoral neck or lumbar spine separately were considered for inclusion in a model to predict low BMD at either of these 2 sites. Scores were assigned by rounding the odds ratio estimates to the nearest integer, assigning a score of zero to the reference group.^26–28 Receiver operating characteristic (ROC) curves were plotted for each model to determine the area under the ROC curve and the sensitivity, specificity and positive predictive value (PPV) at each threshold score.^29–31 Exact binomial confidence intervals were calculated. To ensure that few subjects with a BMD T score of 2 or more SDs below the mean would be missed, threshold scores for recommending testing with DXA were chosen to yield 90% sensitivity or greater. Selection of final variables for inclusion in the Osteoporosis Risk Assessment Instrument (ORAI) was based on comparison of the sensitivity, specificity and PPV when different numbers of clinical variables were included. Sensitivity analyses were performed by comparing the predictive ability of different models to identify the complement of normal BMD (T score more than 1 SD below the mean) and osteoporosis (T score of 2.5 or more SDs below the mean).⁹ Analysis of the development cohort was used to select the final criteria for inclusion in the ORAI. ROC analyses were then used to assess the discriminatory performance of the ORAI in the validation cohort.

[Contents]

The Ontario sample of CaMos comprised 1930 women aged 45 or more years, of whom 1553 (80.5%) had undergone DXA testing at both the femoral neck and the lumbar spine. Of these, 9 were excluded because of cognitive impairment, and a further 168 were excluded because they had a diagnosis of osteoporosis or were taking bone active medications other than ovarian hormones. Of the remaining 1376 women, 926 were randomly allocated to the development of the tool and 450 to its validation. There were no significant differences in demographic characteristics between the development and validation cohorts (Table 1).

In bivariate analysis, age, race (white v. non-white), age at menarche, menopause, weight, sunlight exposure in past year, no alcohol consumption, previous minimal trauma fracture (wrist or forearm, hip, back, pelvis or rib) at age 45 or older, current estrogen use and current progesterone use were associated with low BMD at one or both sites (data not shown). After adjustment, age, weight and current estrogen use were common independent correlates of low BMD at both the femoral neck and at the lumbar spine (Table 2). Other significant correlates of low BMD varied between the 2 bone sites: current physical activity and previous minimal trauma fracture for the hip, and menopause for the lumbar spine. The 6 predictors of low BMD at either the femoral neck or the lumbar spine (age, weight, current estrogen use, menopause, physical activity, and previous minimal trauma fracture at age 45 or older) were considered for inclusion in the ORAI. The discriminatory performance of a scoring system with all 6 variables and of models with fewer variables (eliminated through backward selection) is presented in Table 3. The discriminatory performance of the one-variable model (weight < 70 kg) was significantly less than that of the model with at least 3 variables (weight, age and current estrogen use) (p < 0.05 for difference between areas under ROC curves). In fact, no threshold score using weight alone permitted 90% sensitivity. Models with at least 4 items (age, weight, current estrogen use and menopause) selected all women for DXA testing who were aged 65 years or more and not currently taking estrogen. Models with 3 and 2 items selected all women aged 65 years or more for DXA testing regardless of their estrogen use. Whereas the sensitivity, specificity and PPV of the 3-item instrument was similar to more complex models (those with 4 to 6 items), the specificity of the 2-item scoring algorithm was lower. As a result, 3 items (age, weight and current estrogen use) were included in the final model (i.e., the ORAI) (Table 4). A score of 9 or greater identified 90% of women with a BMD T score of 2 or more SDs below the mean and was therefore chosen as the threshold to recommend further testing with bone densitometry.

Table 5 provides summary statistics of the ORAI's performance in identifying women with the complement of normal BMD, low BMD and osteoporosis in the development and validation cohorts. The discriminatory performance of the ORAI did not differ significantly between the development and validation cohorts (areas under ROC curves 0.79 and 0.77 respectively, p > 0.05). At the recommended threshold score of 9, the 3-item ORAI had a sensitivity of 90.0%, a specificity of 45.1% and a PPV of 32.5% for identifying women with low BMD in the development cohort; the corresponding values in the validation cohort were 93.3%, 46.4% and 34.6%. In addition, the sensitivity was 97.0% in the development cohort and 94.4% in the validation cohort for selecting women with osteoporosis. With a T score of less than 1 SD below the mean, specificity increased to 56.8% and 58.2% in the development and validation cohorts respectively; in other words, the ORAI selected about 43% of those with normal BMD values for further testing with DXA. Overall, use of the ORAI represented a 38.7% reduction in DXA testing compared with screening all women in our study.

[Contents]

Screening all women for osteoporosis using bone densitometry is not recommended,^12–14 and even selective use has been criticized in "well women".³² However, practice guidelines recommend bone densitometry for the diagnosis of osteoporosis and for deciding on treatment.^8–10,17 As a result, clinicians must use clinical factors to determine who is at increased risk for osteoporosis and should thus undergo further testing for low BMD. No clear consensus exists as to which factors clinicians should use to guide this decision.¹⁶ We found that a simple assessment using 3 factors that can be easily determined during a clinical interview successfully identified over 90% of women at increased risk for osteoporotic fractures. At the same time, fewer than half of those with normal BMD would be selected for testing.

The ORAI supports selective DXA testing in women aged 65 years or more, in women aged 45 years or more who weigh less than 60 kg and in women aged 55–64 years who weigh 60–70 kg and are not taking estrogen. Although selection of women aged 65 years or more is consistent with the US National Osteoporosis Foundation guidelines,⁸ the latter also recommend selection of postmenopausal women under 65 years of age who have one or more risk factors for osteoporosis other than menopause. The ORAI provides more specific recommendations for selection based on current weight and estrogen use; the result would be substantially less use of bone densitometry. For example, the selection of all women aged 65 years or more, in addition to postmenopausal women who currently smoke, weigh less than 127 lb (57.6 kg), have a history of fracture as an adult or have a history of fracture in a first-degree relative (the 4 risk factors highlighted by the National Osteoporosis Foundation), had a sensitivity of 92% but a specificity of only 21% for identifying low BMD in our sample.

Other approaches to selecting women for bone densitometry have been proposed.^33–35 Michaëlsson and colleagues³³ suggested body weight as the sole criterion for osteoporosis screening. Our results also showed that weight less than 70 kg was the single best indicator of low BMD. However, because of low sensitivity, weight alone is insufficient to select women for bone densitometry. The Simple Calculated Osteoporosis Risk Estimation (SCORE), created by Lydick and colleagues,³⁴ uses an index based on age, race, rheumatoid arthritis, history of nontraumatic fracture after age 45, estrogen use and weight. Although evaluations of SCORE in a Toronto study revealed good sensitivity, selecting 90% of women with low BMD, SCORE also selected 68% of women with a BMD value not more than 2 SDs below the mean.³⁶ Finally, Weinstein and colleagues³⁵ identified factors associated with osteoporosis among 1346 postmenopausal women referred for DXA testing. They suggested that postmenopausal women aged 61 years or more who weigh 165 lb (74.8 kg) or less be screened and that, among postmenopausal women who have never received oral estrogen therapy (oral contraceptives or hormone replacement therapy), either age or weight be used as the screening criterion. When applied to our postmenopausal women, the sensitivity was similar to that of the ORAI (89%); however, the specificity was lower (32%).

This is the largest Canadian study to provide a comprehensive evaluation of osteoporosis risk factors and "gold-standard" testing with DXA. However, the generalizability of the ORAI may be limited because of the low response rate (42%) in the CaMos Ontario sample. Further validation of the ORAI is required to evaluate its discriminatory performance using data from different populations and to explore results using current manufacturer norms.²⁵ Although the ORAI may help to guide decisions about the need for bone densitometry, clinical judgement in individual cases is always important. The ORAI is not intended for use in women at high risk for secondary osteoporosis (e.g., long-term corticosteroid use and primary hyperparathyroidism). In addition, the ORAI does not address follow-up DXA testing for future diagnostic evaluation, or the follow-up efficacy of treatment strategies. Physicians can refer to published Canadian guidelines for recommendations about these issues.¹¹ Furthermore, physicians should promote bone health independent of decisions about DXA testing, including physical activity, adequate nutrition (calcium and vitamin D in particular) and avoidance of tobacco and excessive alcohol.^8,11

Incidence rates of hip fracture are projected to double within the next 15 years if preventive measures are not taken.^5,6 DXA testing for low BMD identifies which women are most in need of preventive treatment¹¹ and significantly increases the use of hormone replacement therapy.^37,38 The 3-item ORAI provides a simple method for making clinical decisions about the need for DXA testing. Targeting high-risk populations is important for achieving cost-effective interventions.³⁹ In Ontario, DXA testing costs $81–$101, as compared with $41–$65 for a mammogram.⁴⁰ Use of the ORAI may help to limit unnecessary costs. The ORAI identifies the majority of women at risk for osteoporosis, yet it limits the need for unnecessary testing among those with normal BMD.

We thank the Canadian Multicentre Osteoporosis Study (CaMos) site directors for permission to use their data for this study: Dr. Timothy Murray (Toronto site), Dr. Jonathan D. Adachi (Hamilton site), Dr. Tassos Anastassiades (Kingston site) and Dr. Alan Tenenhouse (CaMos principal investigator).

This study was supported by a grant from the University of Toronto Dean's Fund. CaMos was funded by the Seniors Independence Research Program through the National Health Research and Development Program (project no. 6605-4003-OS), the Medical Research Council of Canada (MRC), the MRC–PMAC (Pharmaceutical Manufacturers Association of Canada) Health Program, Merck Frosst Canada Inc., Eli Lilly Canada Inc., Procter and Gamble Pharmaceuticals Canada Inc. and Dairy Farmers of Canada. Dr. Susan Jaglal is a career scientist of the Ontario Ministry of Health; Dr. Jack Tu is supported by an MRC Scholar award.

This article was presented as a plenary poster at the meeting of the American Society for Bone and Mineral Group, Sept. 30-Oct. 1, 1999, in St. Louis, Mo.

Competing interests: None declared.

[Contents]

Reprint requests to: Ms. Suzanne M. Cadarette, Population Health Sciences, Research Institute, Hospital for Sick Children, 555 University Ave., Toronto ON M5G 1X8; s.cadarette@utoronto.ca

1. Jaglal S. Osteoporotic fractures: incidence and impact. In: Williams JI, Badley EM, editors. Patterns of health care in Ontario: arthritis and related conditions. Toronto: Institute for Clinical Evaluative Sciences; 1998. p. 143-56.
2. Johnell O. The socioeconomic burden of fractures: today and in the 21st century. Am J Med 1997;103(2A Suppl):20S-26S.
3. Lindsay R. The burden of osteoporosis: cost. Am J Med 1995;98(2A Suppl):9S-11S.
4. Singer BR, McLauchlan GJ, Robinson CM, Christie J. Epidemiology of fractures in 15 000 adults: the influence of age and gender. J Bone Joint Surg Br 1998;80B:243-8. [MEDLINE]
5. Jaglal SB, Sherry PG, Schatzker J. The impact and consequences of hip fracture in Ontario. Can J Surg 1996;39(2):105-11. [MEDLINE]
6. Millar WJ, Hill GB. Hip fractures: mortality, morbidity and surgical treatment. Health Rep 1994;6(3):323-37. [MEDLINE]
7. Papadimitropoulos EA, Coyte PC, Josse RG, Greenwood CE. Current and projected rates of hip fracture in Canada. CMAJ 1997;157(10):1357-63. [MEDLINE]
8. National Osteoporosis Foundation. Physician's guide to prevention and treatment of osteoporosis. Belle Mead (NJ): Excerpta Medica; 1998.
9. Kanis JA, Delmas P, Burckhardt P, Cooper C, Torgerson D. Guidelines for diagnosis and management of osteoporosis. The European Foundation for Osteoporosis and Bone Disease. Osteoporos Int 1997;7(4):390-406. [MEDLINE]
10. The prevention and management of osteoporosis. Consensus statement. Australian National Consensus Conference 1996. Med J Aust 1997;167(Suppl 1):1-15.
11. Scientific Advisory Board, Osteoporosis Society of Canada. Clinical practice guidelines for the diagnosis and management of osteoporosis. CMAJ 1996;155(8):1113-33. [MEDLINE]
12. Canadian Task Force on the Periodic Health Examination. The Canadian guide to clinical preventive health care. Ottawa: Minister of Supply and Services Canada; 1994.
13. US Preventive Services Task Force. Guide to clinical preventive services. 2nd ed. Baltimore: Williams & Wilkins; 1996.
14. Marshall D, Johnell O, Wedel H. Meta-analysis of how well measures of bone mineral density predict occurrence of osteoporotic fractures. BMJ 1996;312:1254-9. [MEDLINE]
15. Jaglal SB, Hawker G, Jaakkimainen L, McIsaac W. Bone density testing in Ontario, Canada: patterns of use and implications for health care costs [abstract]. J Bone Miner Res 1999;14(Suppl 1):S251.
16. Kazanjian A, Green C, Bassett K. Bone densitometry: Does the emperor have clothes? [letter]. CMAJ 1999;160(13):1822. [MEDLINE]
17. Joseph P, Hughes D. Osteoporosis: guidelines for general practitioners. Aust Fam Physician 1997;26(10):1181-96. [MEDLINE]
18. Kreiger N, Tenenhouse A, Joseph L, Mackenzie T, Poliquin S, Brown J, et al. The Canadian Multicentre Osteoporosis Study (CaMos): background, rationale, methods. Can J Aging 1999;18(3):376-87.
19. Folstein MF, Folstein SE, McHugh PR. "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975;12:189-98. [MEDLINE]
20. Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction rules. Application and methodological standards. N Engl J Med 1985;313(13):793-9. [MEDLINE]
21. Genant HK, Grampp S, Glüer CC, Faulkner KG, Jergas M, Engelke K, et al. Universal standardization for dual x-ray absorptiometry: patient and phantom cross-calibration results. J Bone Miner Res 1994;9(10):1503-14. [MEDLINE]
22. Jergas M, Glüer C. Assessment of fracture risk by bone density measurements. Semin Nucl Med 1997;27(3):261-75. [MEDLINE]
23. Meema HE, Meema S. Postmenopausal osteoporosis: simple screening method for diagnosis before structural failure. Radiology 1987;164:405-10. [MEDLINE]
24. Kleinbaum DG, Kupper LL, Muller KE, Nizam A. Selecting the best regression equation. In: Applied regression analysis and other multivariable methods. 3rd ed. Toronto: Duxbury Press; 1998. p. 386-422.
25. Laupacis A, Sekar N, Stiell IG. Clinical prediction rules: a review and suggested modifications of methodological standards. JAMA 1997;277(6):488-94. [MEDLINE]
26. Tu JV, Mazer D, Levinton C, Armstrong PW, Naylor CD. A predictive index for length of stay in the intensive care unit following cardiac surgery. CMAJ 1994;151(2):177-85. [MEDLINE]
27. Harrell F. Regression coefficients and scoring rules [letter]. J Clin Epidemiol 1996;49(7):819. [MEDLINE]
28. Tu JV, Naylor CD. Clinical prediction rules [letter]. J Clin Epidemiol 1997;50(6):743-4. [MEDLINE]
29. Metz CE. Basic principles of ROC analysis. Semin Nucl Med 1978;8(4):283-98. [MEDLINE]
30. Hanley JA, McNeil BJ. The meaning and use of the area under receiver operating characteristic (ROC) curve. Radiology 1982;143:29-36. [MEDLINE]
31. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837-45. [MEDLINE]
32. Green CJ, Basette K, Foerster V, Kazanjian A. Bone mineral density testing: Does the evidence support its selective use in well women? Vancouver: British Columbia Office of Health Technology Assessment, University of British Columbia; 1997. Report no BCOHTA 97:2T.
33. Michaëlsson K, Bergström R, Mallmin H, Holmberg L, Wolk A, Ljunghall S. Screening for osteopenia and osteoporosis: selection by body composition. Osteoporos Int 1996;6:120-6. [MEDLINE]
34. Lydick E, Cook K, Turpin J, Melton M, Stine R, Byrnes C. Development and validation of a simple questionnaire to facilitate identification of women likely to have low bone density. Am J Manag Care 1998;4(1):37-48. [MEDLINE]
35. Weinstein L, Ullery B, Bourguignon C. A simple system to determine who needs osteoporosis screening. Obstet Gynecol 1999;93(5 pt 1):757-60.
36. Cadarette SM, Jaglal SB, Murray TM. Validation of the Simple Calculated Osteoporosis Risk Estimation (SCORE) for patient selection for bone densitometry. Osteoporos Int 1999;10(1):85-90. [MEDLINE]
37. Papaioannou A, Parkinson W, Adachi J, O'Connor A, Jolly EE, Tugwell P, et al. Women's decisions about hormone replacement therapy after education and bone densitometry. [abstract/résumé] CMAJ 1998;159(10):1253-7.
38. Torgerson DJ, Thomas RE, Campbell MK, Reid DM. Randomized trial of osteoporosis screening. Use of hormone replacement therapy and quality-of-life results. Arch Intern Med 1997;157:2121-5. [MEDLINE]
39. Jönsson B. Targeting high-risk populations. Osteoporos Int 1998;8(Suppl 1):S13-6.
40. Changes to the physicians' schedule of benefits and technical fee adjustments for October 1, 1999 [Bull no 4346]. Toronto: Ontario Ministry of Health and Long-Term Care; 1999.

© 2000 Canadian Medical Association or its licensors