Chronic Diseases and Injuries in Canada

Volume 31, no. 3, June 2011

Estimates of the treated prevalence of bipolar disorders by mental health services in the general population: comparison of results from administrative and health survey data

A.G. Bulloch, PhD (1); S. Currie, PhD (2); L. Guyn, BSc, BA (2); J.V. Williams, MSc (1); D.H. Lavorato, MSc (1); S.B. Patten, MD, PhD (1)

Abstract

Introduction: Informed provision of population mental health services requires accurate estimates of disease burden.

Methods: We estimated the treated prevalence of bipolar disorders by mental health services in the Calgary Zone, a catchment area in Alberta with a population of over one million. Administrative data in a central repository provides information of mental health care contacts for about 95% of publically funded mental health services. We compared this treated prevalence against self-reported data in the 2002 Canadian Community Health Survey: Mental Health and Well-Being (CCHS 1.2).

Results: Of the 63 016 individuals aged 18 years plus treated in the Calgary Zone in 2002–2008, 3659 (5.81%) and 1065 (1.70%) were diagnosed with bipolar I and bipolar II disorder, respectively. The estimated treated population prevalence of these disorders was 0.41% and 0.12%, respectively. We estimated that 0.44% to 1.17% of the Canadian population was being treated by psychiatrists for bipolar I disorder from CCHS 1.2.

Discussion: For bipolar I disorder the estimate based on local administrative data is close to the lower end of the health survey range. The degree of agreement in our estimates reinforces the utility of administrative data repositories in the surveillance of chronic mental disorders.

Keywords: bipolar disorder, administrative data, health surveys, prevalence.

Introduction

Accurate estimates of the disease burden of mental disorders in the population are necessary to provide adequate mental health services. Traditionally, estimates of the prevalence of mental disorders in the general population have used data from health surveys carried out either in person and/or by telephone. However, such health surveys suffer from a number of shortcomings. For example, the 2002 Canadian Community Health Survey: Mental Health and Well-Being (CCHS 1.2),¹ which estimated the prevalence of mental disorders and the use of health services, relies on self-report data rather than on professional diagnosis. Though this data is obtained by trained personnel through face-to-face interviews, it is subject to recall bias; hence the possible value of estimates based on other sources of data.

In Canada, the public health care sector provides the majority of health services, including treatment for addictions and mental disorders. Detailed information on the recipients of health services are captured in various administrative datasets. This information is easily accessible, and its use for research purposes is cost effective.² Such databases provide a “real-world” perspective on treatment of mental disorders that generalize to the actual practice of providing mental health services. Further, administrative datasets can provide precise estimates of treated prevalence and avoid the recall bias of health surveys.³ As such, they can contribute significantly towards increasing the capacity for national health surveillance.⁴

Administrative data on mental health has been used to research the effects of system changes on service use and quality of care,⁵ variations in treatment practices across settings,⁶ performance measurement including adherence to best practices,⁷ predictors of service utilization,⁸ determining the proportion of the general population with mental disorders who receive treatment,^{9, 10} the cost effectiveness of mental health services,¹¹ place-based population health research² and long-term evaluation of changes in the use of psychiatric emergency services.¹²

The Calgary Zone is one of five defined catchment areas for the province of Alberta. All public health services in Alberta are under a single governing body called Alberta Health Services (AHS). The Calgary Zone covers a geographic area of 39 000 square kilometres and has a population of over 1.3 million inhabitants. It includes one large urban city (Calgary) and several smaller cities and towns including Banff, Airdrie, Okotoks and Canmore. The Calgary Zone provides a wide range of adult addiction and mental health services including specialized inpatient treatment in three large urban hospitals, day hospital services, outpatient programs including one clinic that specializes in bipolar disorder, and community outreach programs. People with bipolar disorder can access any of these services at no personal cost.

The use of a central data repository created by linking administrative data from separate information systems is an innovative way of deriving period prevalence estimates for treated mental health conditions. It is a different approach to that taken by most record linkage studies in Canada, for which family doctor visits or hospitalizations are the primary patient encounters. The data repository maintained in the Calgary Zone links data from the entire spectrum of psychiatric services, including inpatient, day hospital, outpatient, and community outreach programs. As such, this data repository is unique, although it does resemble the now defunct Kingston Psychiatric Record Linkage System.¹³ The majority of research using administrative data is conducted on acute care service users. However, many people with mental disorders never require hospitalization or emergency psychiatric care. Physician billing records are also limited for estimating the prevalence of specific mental disorders; in Alberta, physicians are required to submit only the first three digits of the ICD-9* code that identifies the patient as having either a depressive or bipolar mood disorder, for example. In addition, alternative relationship plans may preclude access to physician billing data since these plans replace fee-for-service billings. For example, in a multidisciplinary setting physicians may be paid through sessional arrangements that do not require submission of a diagnostic code as part of a fee-for-service submission or they may not be required to submit a fee-for-service billing at all.

Bipolar disorders can be devastating; they usually begin in early life and are associated with a high risk of suicide.¹⁴ Bipolar I disorder is characterized by one or more manic or mixed episodes that may or may not be accompanied by one or more episodes of major depression.¹⁵ Symptoms of mania include flight of ideas or racing thoughts, inflated self-esteem, decreased need for sleep, talkativeness and irritability. Bipolar II disorder is characterized by hypomanic episodes that, in contrast to manic episodes, are not severe enough to cause marked impairment in social or occupational functioning, or result in hospitalization. In order to meet DSM-IV-TR^† diagnostic criteria for bipolar II disorder, there must also be one or more episodes of major depression.

Whereas it is often proposed that bipolar disorders are underdiagnosed, some authors postulate the opposite.¹⁶ One controversial proposal is to lower the threshold for diagnosis of bipolar disorder, which would substantially increase estimates of its prevalence.¹⁷ Either way, it is apparent that there is a need to evaluate the actual prevalence in real world treatment.

The purpose of our study is to compare estimates of the treated prevalence of bipolar disorders from CCHS 1.2 and the mental health service data repository of the Calgary Zone.

Methods

This study is based on data from two sources. National estimates of the treated prevalence of bipolar disorder I in the general population came from CCHS 1.2. We compared these estimates to the calculated treated prevalence for both bipolar I and II disorders from administrative data in the Calgary Zone. In terms of physician type, the administrative data covers various mental health services (see below), but not general physicians (GPs). To be able to compare the 2 datasets, we restricted our analysis of CCHS 1.2 to psychiatrists alone.

National mental health survey 

CCHS 1.2 has been described in detail elsewhere.¹⁸  Briefly, conducted in 2002, CCHS 1.2 was a  population-based, cross-sectional survey designed to monitor the mental health of Canadians and their need and use of mental health services. Statistics Canada obtained a nationally representative sample of individuals aged 15 years or older in 2002 that did not include individuals from the three territories, armed forces, Aboriginal populations, or living in institutions or in some remote areas; the response rate was 77% (n = 36 984). In the majority of cases, trained personnel conducted face-to-face interviews, with telephone interviews being conducted when this was not possible.

We received approval to access the CCHS 1.2 Master File from the Social Sciences and Humanities Research Council, and accessed these data at the Statistics Canada Prairie Regional Research Data Centre at the University of Calgary. Ethical approval for access was acquired from the University of Calgary Conjoint Health Research Ethics Board.

Assessment of bipolar I disorder in CCHS 1.2 is based upon the diagnosis of manic or mixed episodes in accordance with DSM-IV-TR diagnostic criteria.¹³ The specific questions on mania were based on a World Mental Health version of the Composite International Diagnostic Interview (WMH-CIDI)¹⁹ modified for CCHS 1.2. and were delivered by trained interviewers. Respondents were not asked if they have bipolar disorder. Instead, they were asked series of questions. Algorithms were then used to assess this disorder depending on the answers received. Two algorithms were used to determine if manic episodes occurred in either the last year (12-month prevalence) or during the respondents’ lifetime (lifetime prevalence). Separate questions asked whether a GP or psychiatrist was treating their disorder.

To calculate the treated prevalence of bipolar I disorder in the Canadian population, we cross-tabulated the raw CCHS 1.2 data and calculated population estimates with 95% confidence intervals (CIs). (Note that CCHS 1.2 did not survey bipolar II disorder.) These estimates and CIs were both weighted and bootstrapped, using sampling weights and replicate bootstrap weights provided by Statistics Canada, to compensate for complex sampling procedures. For example, small provinces were oversampled so the impact of these results on the national estimate has to be reduced accordingly, i.e. given less weight. Since the sample size of bipolar cases in CCHS 1.2 was insufficient to create a separate, reliable estimate for the province of the Alberta, we used the national prevalence estimates as a surrogate. While there is no reason to believe that prevalence estimates vary substantially across Canada, research has shown regional differences in mental health service use,20 and these would influence estimates of treatment prevalence.

Administrative data

 The administrative records of mental health service users in the Calgary Zone are maintained in a central data repository. All these users had been seen by a mental health professional (psychiatrist, psychiatric nurse, psychologist or social worker) licensed in Alberta to conduct diagnostic evaluations. For each service user there is a minimum dataset consisting of a unique lifetime identifier (ULI), referral source, admission and discharge dates, length of stay, program enrolment, age, gender, postal code, mental health diagnoses based on DSM-IV-TR nosology, and disposition at discharge. Records are extracted from over 95% of the mental health information systems used to provide services to adult, child and adolescent, geriatric and Aboriginal clients, and then linked into the central database; the remaining 5% of users engage in services in which complete data may not obtained from the client because of the nature of the service (e.g. in some crisis or outreach services the clients are not formally enrolled and ULI is not obtained). Based on the postal codes, the majority of mental health service users live within the Calgary Zone.

We defined cases of bipolar illness from the administrative dataset based on the following criteria: (1) the patient was formally registered in a mental health service in the Calgary Zone; these services included inpatient services, day hospitals, psychiatric emergency services, outpatient clinics, and community outreach programs; and (2) the most responsible diagnosis (MRD) recorded for the registration was bipolar I or II disorder; the MRD represents the main reason the patient was admitted to the program in question. This case definition excluded patients treated by other health care workers for medical care unrelated to their bipolar condition (e.g. dietary consultation) and one-time visits to other professionals for non-specific social issues (e.g. housing). Most patients registered in mental health have multiple diagnoses. The presence of other diagnostic codes in the health record did not exclude patients as long as bipolar I or II disorder was listed as the MRD. We were concerned about including secondary (i.e. not MRD) diagnoses as these may often have been recorded as a “rule out” diagnosis on certain visits. In all mental health services, diagnosis is made based on comprehensive clinical assessment, although the specific interview tools and other assessment instruments vary across programs.

The Calgary Zone does not have a long-term psychiatric institution although it does have long-term care facilities for geriatric patients. Data from these facilities are not linked to the central data repository for mental health services; as a result, elderly people with bipolar illness who live in nursing homes are not represented in our estimate of treated prevalence unless they had accessed one of the services covered.

We obtained aggregate estimates of the treated prevalence of bipolar disorders from the Information and Evaluation Unit in the Calgary Zone. These analyses were performed “in house” as part of the functioning of these units and did not require ethical approval. Results from the administrative database are expressed as a mean with 95% CI, and are not weighted since they are not samples.

All the data we present here are for individuals 18 years and older.

Results

The basic demographics of the study populations are shown in Table 1. In CCHS 1.2, 364 and 890 respondents scored positive in the 12-month and lifetime bipolar I algorithms, respectively. The higher proportion of women compared to men reflects the higher percentage of female respondents in CCHS 1.2; the prevalence of bipolar I disorder has been estimated to be equal in men and women in this survey.²¹ Data from the Calgary Zone are very similar to that for CCHS 1.2 as assessed by lifetime criteria. In the case of the Calgary Zone, however, treated prevalence was sex dependent: significantly more men than women were being treated for bipolar I disorder, while the opposite was true for bipolar II disorder, with almost two-thirds of treated patients women. These discrepancies suggest differential help-seeking between the two disorders by gender.

TABLE 1: Characteristics of bipolar patients in the general population of Canada, 2002, and the Calgary Zone, 2002–2008

	Canadaa (2002)		Calgary Zoneb (2002–2008)
	Bipolar I (12-month estimate)c (n = 364)	Bipolar I (lifetime estimate)d (n = 890)	Bipolar I (n = 3659)	Bipolar II (n = 1065)
Abbreviations: CCHS 1.2: 2002 Canadian Community Health Survey: Mental Health and Well-Being; CI, confidence interval; n, sample size. a Derived from CCHS 1.2. b Derived from 2002–2008 Calgary Zone administrative data repository. c One or more episodes in the preceding 12 months. d One or more lifetime episodes. e Percentages may not add up to 100% due to rounding. f Sample size is too small for release; Statistics Canada forbids the release of small cell sizes due to confidentiality concerns.
Mean percentagee (95% CI)
Men	42.2% (35.2–49.3)	46.1% (41.6–50.5)	53.7% (52.1–55.3)	38.5% (35.6–41.4)
Women	57.7% (50.7–64.8)	53.9% (49.5–58.4)	46.3% (44.6–47.9)	61.5% (58.6–64.4)
Mean age, years	34.8 (33.0–36.5)	38.7 (37.6–39.9)	40.0 (39.5–40.5)	39.5 (38.7–40.3)
Age distribution in yearse (95% CI)
18–24	26.5% (19.9–33.1)	17.2% (13.5–20.9)	17.0% (15.8–18.2)	14.4% (12.3–16.5)
25–44	49.9% (42.7–57.0)	48.0% (43.4–52.6)	48.4% (46.8–50.0)	52.0% (49.0–55.0)
45–64	23.7% (17.7–29.6)	33.1% (28.8–37.4)	27.8% (26.3–29.2)	29.8% (27.0–32.5)
65+	—f	1.7% (0.8–2.6)	6.8% (6.0–7.6)	3.8% (2.7–5.0)

 

Stratification by age group (Table 1) shows that the four study populations were similar in terms of age distribution. The only clear exception is the somewhat younger population that screened positive for 12-month bipolar I disorder in CCHS 1.2 when compared to the other 3 groups.

We used administrative data from the Calgary Zone to estimate the treated prevalence for both bipolar disorders as 0.41% and 0.12% for bipolar I and II disorders, respectively (Table 2).
Data from CCHS 1.2 enabled us to estimate the proportion of Canadians with bipolar I disorder who sought help for their condition. (Bipolar II disorder was not included in the survey.) We made both 12-month and lifetime estimates since these might be expected to bracket our 7-year administrative data estimate. These 12-month and lifetime estimates were 0.44% and 1.17% respectively (Table 3).

TABLE 2: Treatment by psychiatrists of bipolar I and II disorders in the population with mental health disorders, Calgary Zone, 2002–2008

	Number of adults with bipolar disorder, n	Percentage of mental health patients with bipolar disordera , % (95% CI)	Treated prevalence of bipolar disorder in the Calgary Zoneb , % (95% CI)
Abbreviations: CI, confidence interval; n, sample size. a Denominator is 63 016, i.e. the number of adults diagnosed with a mental disorder, 2002–2008. b Denominator is 894 905, i.e., the estimated population of the Calgary Health Region aged 18 years and older at the mid-point between 2002 and 2008.
Bipolar I	3659	5.81 (5.63–5.99)	0.41% (0.40–0.42)
Bipolar II	1065	1.70 (1.59–1.79)	0.12% (0.11–0.13)

 

TABLE 3: Treatment of bipolar I^a disorder by psychiatrists based on CCHS 1.2, 2002, Canada

	Prevalence estimates
	12 months	Lifetime
Abbreviations: CCHS 1.2: 2002 Canadian Community Health Survey: Mental Health and Well-Being; CI, confidence interval; n, sample size. a Bipolar II disorder was not included in CCHS 1.2. b Numbers less than the full number of CCHS 1.2 respondents (36 984) due to missing data. c Numbers lower than those shown in Table 1 due to missing data. d Weighted estimate.
Number of survey respondents, n:
Overall	34 946b	34 921b
Who screened positive for bipolar I	357c	880c
Who were being treated for bipolar I	171	430
Percentage of those who screened positive for bipolar I who receive psychiatric treatment	48.7%d (41.8–55.6)	49.8%d (45.1–54.4)
Percentage of CCHS respondents who receive psychiatric treatment	0.44%d (0.36–0.52)	1.17%d (1.02–1.33)

Discussion

To the best of our knowledge, we are the first to investigate the consistency of self-reported treatment rates with actual administrative records for a specific mental health disorder.
A key element of this study is the use of data repository rather than physician billing data. Our results indicate that the population survey estimate of the proportion of people with bipolar disorder who self-report receiving treatment from a psychiatrist approximates the treated prevalence estimate derived from actual administrative records of mental health service users. The congruence of these estimates is an important finding and has implications for future prevalence studies: using administrative data could be a cost-effective and accessible way of accurately estimating prevalence of a disorder in general population.

Since we were unable to account for patients who were receiving treatment by GPs and not psychiatrists, the question arises as to what proportion of patients in the Calgary Zone are being treated only by GPs. Using data from CCHS 1.2 on respondents that screen positive for bipolar I disorder, we estimated the prevalence of lifetime bipolar I disorder in respondents aged 18 years and over to be 2.39% (95% CI: 2.19–2.60%)‡ and the proportion treated by GPs alone to be 0.46% (95% CI: 0.35–0.57%). In actuality, a higher proportion of respondents (1.17%; Table 3) receive psychiatric care, and hence the proportion of patients receiving psychiatric care is 72%, i.e. [1.17/(1.17 + 0.46)] x 100. This suggests that the data repository has captured the majority (about 70%) of patients under medical care for bipolar I disorder in the Calgary Health Region.

What proportion of patients with bipolar disorders is not being treated by either a GP or a psychiatrist? From CCHS 1.2, we estimate that 0.73% (95% CI: 0.62–0.84%) of respondents with bipolar I disorder are not under medical care. Individuals with mild variants of bipolar disorder may not require treatment; others may have clinically significant disorders that could benefit from treatment, but issues such as fear of stigma or limited access to specialized care stop them from accessing treatment. These alternatives obviously have important implications; it is likely that the availability of a variety of sources of information will help to distinguish between these possibilities. Survey data can estimate the proportion of a population that has a diagnosable disorder, whereas a treated prevalence is restricted to the proportion actually receiving treatment. These results indicate that administrative data may provide a valuable perspective on the treated prevalence of bipolar disorder.

A limitation of health surveys is that they rely upon self-report. On the other hand, administrative data provide an objective assessment of actual treatment received. For mental disorders that are relatively infrequent in the population, administrative data can provide substantially more cases for analysis than survey samples.³ This was evident in the present study in which the sample of bipolar I cases obtained from administrative data sources was substantially larger than the sample from a national mental health survey (Table 1).

Researchers have questioned the quality of administrative data, particularly regarding the coding of diagnoses.^{22, 23} Local re-abstraction studies for inpatient encounters^{24, 25} suggest that the Calgary Zone’s coding practices are reliable. Although sensitivity rates vary considerably by medical condition, specificity rates in Calgary have been found to be 99% or better across all conditions examined (i.e. in nearly every case, the most responsible diagnosis on record for the inpatient encounter was verified by an independent medical expert). We acknowledge that there is limited research on the validity of mental health diagnoses in administrative data.

It should be noted that we may have overestimated actual treated prevalence since some individuals may contact a physician but not receive treatment. For this reason the term “contact prevalence” may be preferable when estimating the prevalence of an illness from administrative data sources.²⁶

Limitations

A limitation of our study is that we were unable to assess the proportion of bipolar patients being treated by those private psychiatrists (about 30%) who do not have an affiliation with the psychiatric services in the Calgary Zone. Taken together these considerations suggest that the actual treated prevalence of bipolar disorders by psychiatrists in the Calgary Zone (Table 2) is even closer to the national­­survey-based estimates (Table 3).

Second, CCHS 1.2 did not include Aboriginal peoples or those living in institutions. These individuals cannot be removed from the data repository so this limits the comparison of administrative data to that from CCHS 1.2.

Another limitation of CCHS 1.2 is that the criteria for bipolar I disorder do not fully conform with DSM-IV criteria. The latter requires manic symptoms to be present for 7 days unless hospitalization is required.¹⁵ Meanwhile, CCHS 1.2 requires manic symptoms to be present for 4 days, reducing the specificity compared with that obtainable by strict DSM-IV criteria. This consideration may in part explain the higher estimate of treatment of bipolar I disorder from CCHS 1.2 data relative to our local administrative data.

In summary, we found a significant degree of agreement between estimates of treated bipolar I disorder in local administrative data and national survey data. This observation reinforces the potential utility of administrative data repositories in the surveillance of chronic mental disorders.

‡ This differs slightly from the prevalence of 2.2% reported by Shaffer  et al.²¹ because their result was for all respondents aged 15 years and over.

Acknowledgements

This project was funded by a grant from the Hotchkiss Brain Institute, Faculty of Medicine, University of Calgary. We thank Jim Si (Population Surveillance Group, Calgary Zone) for providing the estimated population of the Calgary Zone in 2005. CCHS 1.2 data were collected by Statistics Canada. However, the analyses and interpretations presented here are those of the authors and not Statistics Canada.

References

1. 2002 Canadian Community Health Survey: Mental Health and Well-being [Internet]. Ottawa (ON): Health Canada; 2003 [cited 2010 Apr 6]. Available from:
   http://www.statcan.gc.ca/pub/82-617-x/index-eng.htm
2. Frohlich KL, Dunn JR, McLaren L, Shiell A, Potvin L, Hawe P, et al. Understanding place and health: a heuristic for using administrative data. Health Place. 2007;13:299-309.
3. Mortensen PB. The untapped potential of case registers and record-linkage studies in psychiatric epidemiology. Epidemiologic Rev. 1995;17:205-9.
4. Advisory Committee on Population Health and Health Security Surveillance Systems for Chronic Disease Risk Factors Task Group. Enhancing capacity for surveillance of chronic disease risk factors and determinants. Ottawa (ON): Public Health Agency of Canada; 2005. Catalogue No.: HP5-11/2005.
5. Greenberg GA, Rosenheck RA. Does system reform reduce geographic variation in mental health system performance. Psychiatric Q. 2005;76:231-42.
6. Speer DC, Newman FL. Mental health services outcome evaluation. Clin Psychol Sci Pr. 1996;3:105-29.
7. Addington D, McKenzie E, Addington J, Patten S, Smith H, Adair C. Performance measures for early psychosis treatment service. Psychiatric Serv. 2005;56:1570-82.
8. Karlin BE, Norris MP. Public mental health care utilization by older adults. Adm Policy Ment Health. 2006;33:730-6.
9. Andrews G, Issakidis C, Sanderson K, Corry J, Lapsley H. Utilising survey data to inform public policy: comparison of the cost-effectiveness of treatment of ten mental disorders. Br J Psychiatry. 2004;184:526-33.
10. Slomp M, Bland R, Patterson S, Whittaker L. Three-year physician treated prevalence rate of mental disorders in Alberta. Can J Psychiatry. 2009;54:199-203.
11. Andrews G. It would be cost-effective to treat more people with mental disorders. Aust N Z J Psychiatry. 2006;40:613-5.
12. Paradis M, Woogh C, Marcotte D, Chaput Y. Is psychiatric emergency service (PES) use increasing over time? Int J Ment Health Syst. 2009;3:3.
13. Woogh CM. An experience in psychiatric record linkage. Can J Psychiatry. 1988;33:134-9.
14. Oswald P, Souery D, Kasper, Lecrubier Y, Montgomery S, Wyckaert S, et al. Current issues in bipolar disorder: a critical review. Eur Neuropsychopharmacol. 2007;17:687-95.
15. American Psychiatric Association. Diagnostic and statistical manual of mental disorders, 4th ed. rev. Washington (DC): American Psychiatric Association; 2000.
16. Zimmerman M, Ruggero CJ, Chelminski I, Young D. Is bipolar disorder overdiagnosed? J Clin Psychiatry. 2008;69:935-40.
17. Patten SB, Paris J. The bipolar spectrum—a bridge too far? Can J Psychiatry. 2008;53:762-8.
18. Gravel R, Beland Y. The Canadian Community Health Survey: mental health and well-being. Can J Psychiatry. 2005;10:573-9.
19. Kessler RC, Ustun TB. The World Mental Health (WMH) Survey Initiative version of the World Health Organization (WHO) Composite International Diagnostic Interview (CIDI). Int J Methods Psychiatr Res. 2004;13:93-121.
20. Diaz-Granados N, Georgiades K, Boyle MH. Regional and individual influences on use of mental health services in Canada. Can J Psychiatry. 2010;55:9-20.
21. Schaffer A, Cairney J, Cheung A, Veldhuizen S, Levitt A. Community survey of bipolar disorder in Canada: lifetime prevalence and illness characteristics. Can J Psychiatry. 2006;51:9-16.
22. Roos LL, Soodeen R, Gupta S, Jebamani L. Canadian administrative data: evaluating the quality. Winnipeg (MB): University of Manitoba; 2002.
23. Roos LL, Gupta S, Soodeen R, Jebamani L. Data quality in an information-rich environment: Canada as an example. Can J Ageing. 2005;24, Suppl 1:153-70.
24. Quan H, Parsons GA, Ghali WA. Validity of procedure codes in International Classification of Diseases, 9th revision, clinical modification of administrative data. Med Care. 2004;42:801-9.
25. Quan H, Parsons GA, Ghali WA. Assessing accuracy of diagnosis-type indicators for flagging complications in administrative data. J Clin Epidemiol. 2004;57:366-74.
26. Goldner EM, Jones W, Waraich P. Using administrative data to analyze the prevalence and distribution of schizophrenic disorders. Psychiatr Serv. 2003;54:1017-21.