Agreement in Measuring Socio-economic Status: Area-based Versus Individual Measures

Kitaw Demissie, James A Hanley, Dick Menzies, Lawrence Joseph and Pierre Ernst

Abstract

Area-based socio-economic status (SES) measures are frequently used in epidemiology. Such an approach assumes socio-economic homogeneity within an area. To quantify the agreement between area-based SES measures and SES assessed at the individual level, we conducted a cross-sectional study of 943 children who resided in 155 small enumeration areas and 117 census tracts from 18 schools in Montreal, Quebec. We used street address information together with 1986 census data and parental occupation to establish area-based and individual level SES indicators, respectively. As compared with the SES score determined at the level of the individual, 13 different area-based SES indices classified the children within the same quintile 28.7% (± 2.8%) of the time. The discrepancy was within one quintile in 35.3% (± 2.3%) of cases, two quintiles in 20.6% (± 3.6%), three quintiles in 11.3% (± 4.2%) and four quintiles in 4.1% (± 0.2%). In conclusion, we observed a substantial discrepancy between area-based SES measures and SES assessed at the individual level. Caution should therefore be used in designing or interpreting the results of studies in which area-based SES measures are used to test hypotheses or control for confounding.

Key words: contextual variables; cross-level bias; ecological variables; etiologic research; small area; socio-economic status

Introduction

Individual health outcomes may have individual and aggregate (environmental) level determinants. Cross-level bias is a phenomenon whereby inference about one level of analysis is made on the basis of associations observed at a different level.¹ The most commonly discussed cross-level bias in epidemiology is "ecological fallacy," which is the result of improper interpretation and inference about individual level associations based on associations at the aggregate level.^2,3 Analyses that relate aggregate characteristics to individual outcomes do not necessarily result in ecological bias.¹

In surveys, socio-economic status (SES) may be a sensitive issue for an individual, and this may impair the practicality and validity of direct questions concerning various attributes of SES.⁴ An alternative approach is to use indirect information based on average values of social and economic conditions for geographic areas of residence. In addition to the advantage of limiting non-response, this area-based approach is relatively inexpensive, and information is readily accessible.

In etiologic research, the health outcome may be collected at the level of the individual, while the exposure characteristics, such as SES attributes, are collected at the level of geographic area to test the hypothesis that SES is associated with a particular health outcome or to control for confounding by SES.^5-13 Various area-based attributes of SES have been used-for example, average house value, median monthly rental value of dwellings and the proportion of university-educated subjects, single-parent families or unemployed people as well as composite scales formed by combining these variables.^7,8 This approach is based on an assumption that the characteristics are stable and homogeneous within the geographic area. However, in urban areas the population can be heterogeneous in terms of social and economic conditions,⁹ even in small areas (e.g. one city block), and can change over time.

We therefore studied the degree of agreement between area-based SES measures and SES assessed at the individual level, rating the various area-based SES variables as to their proximity to the individual SES measure.

Materials and Methods

Study Population

This study was part of a larger study designed to investigate the indoor risk factors for airway hyper-responsiveness in Montreal school children. In order to represent a broad range of SES, all schools belonging to five school boards in central Montreal, Quebec, were ranked according to neighbourhood average house value¹⁴ and, within each school board, schools were randomly selected from the upper, middle and lower ranks. One class was selected from each of the 18 schools from each of grades one (ages 6 and 7), three (ages 8 and 9) and five (ages 10 and 11).

Comparison of selected and non-selected schools showed no difference with respect to neighbourhood poverty, income or educational attainment level.¹⁴ For example, for schools selected and not selected in the upper stratum (>$95,000 neighbourhood average house value), the respective proportions of subjects living in poverty (as defined by Statistics Canada) were 6.1% (n = 6) versus 5.4% (n = 115), p = 0. 83; in the middle stratum ($70,000-$95,000 average house value) the respective values were 12.2% (n = 6) versus 12.2% (n = 124), p = 0.98; and in the lower stratum (<$70,000 average house value), they were 45.1% (n = 6) versus 35.9% (n = 52), p = 0.21.

Assessment of Socio-economic Status at the Individual Level

At the individual (family) level, one parent's most recent occupation was used to assess the SES of the child. A trained person interviewed both parents at home in order to obtain a detailed employment history: the name of the company, type of industry, department within a company, job title, short job description, and year job began and ended. This information was collected for current and all preceding jobs. Information on the most recent occupation of the parent was used to identify the corresponding codes of the Canadian Classification and Dictionary of Occupations.¹⁵ The codes were then converted into SES scores for the child, based on income and educational level for each occupation from the tables developed by Blishen and colleagues.¹⁶ The highest score from either parent was retained for analysis (individual level SES index).

The validity of self-reported employment history is well established.¹⁷ The correlations between the SES score derived from the most recent and from the previous three jobs respectively were r = 0.86, r = 0.84 and r = 0.62 for the mothers' jobs and r = 0.87, r = 0.84 and r = 0.81 for the fathers' jobs. The correlation between the fathers' and mothers' current jobs was r = 0.89.

Other proxy SES measures obtained at the individual level were number of people per room in the home (crowding index), single-parent family status and maximal level of education attained by either parent.

Assessment of Socio-economic Status at the Area Level

By this method, socio-economic status for the individual (family of the child in this case) is inferred from the place of residence. In Canada, information on demographic factors, housing, income, education, quality of housing and other household characteristics is summarized at the level of enumeration area by Statistics Canada. Enumeration areas (EAs) represent census geostatistical neighbourhoods that contain up to a maximum of 375 households with relatively homogeneous economic and social living conditions. Census tracts (CTs) represent the next level of geostatistical area, containing between 2,500 and 8,000 (average of 4,000) residents. The boundaries of these areas have been drawn along recognizable divisions between neighbourhoods to create units that are as homogeneous as possible in socio-economic terms.¹⁴

The street address and postal code of the child's usual place of residence together with the Statistics Canada Postal Code Conversion File¹⁸ were used to identify the enumeration area and census tract. The Canada Postal Code Directory¹⁹ in hard copy was used to verify postal codes when the correct spelling of a street was in doubt. The validity of the Statistics Canada Postal Code Conversion file has been indirectly evaluated. The SES rankings of small geographic areas obtained using this conversion file have been found to accurately predict numerous health outcomes.²⁰

The 1986 Census Dictionary²¹ and the code books of the 1986 census tape files prepared by Statistics Canada were used to select variables that were thought to reflect a neighbourhood's SES. Variables used in our analysis for both CTs and EAs were selected from the literature^22-28 on the basis of their frequency of use as social inequality indicators for small areas and were defined as follows.

• Net educational level: the proportion of people aged 15 and over without a high school certificate or diploma subtracted from the proportion of people aged 15 and over with a university degree or
  post-secondary diploma for each small area (This variable is believed to be the most sensitive indicator of educational attainment.¹⁶)
• Proportion university educated: the proportion of people aged 15 and over with a university degree or post-secondary diploma
• Median income: median income of census families
• Income adequacy: median income divided by family size of census families
• Average house value: average house value of occupied private dwellings
• Proportion unemployed: unemployed people aged 15 and over as a percentage of people aged 15 and over who were in the labour force
• Proportion of owned dwellings: total owner-occupied private dwellings as a percentage of total occupied private dwellings
• Proportion of male parents: the number of census families with a male parent as a percentage of total census families
• Proportion of female parents: the number of census families with a female parent as a percentage of total census families
• Proportion of single parents: the number of single-parent census families as a percentage of total census families

In addition to the above variables, the socio-economic status score for each occupation developed by Blishen and his colleagues¹⁶ was weighted according to the proportion of people in each occupation category and summed, in order to get one SES score for each small area, the neighbourhood SES index-I; the z scores of net educational level, median income and proportion unemployed were summed to create the neighbourhood SES index-II; and the z scores of net educational level, median income, average house value, proportion of owned dwellings and proportion of single parents were also summed to calculate the neighbourhood SES index-III.

These area-based SES variables were abstracted for each child in our sample at the EA level. To abstract the variables for the corresponding CTs, raw data were summed whenever possible before averages and proportions were calculated. If means and proportions of enumeration areas were used to derive variables at the CT level, appropriate weighting was used.

Statistical Methods

Agreement between the various area-based SES indices (at both CT and EA levels) and the individual level SES index was assessed in four ways. First, we used both the area-based and individual level SES scores to group the children into quintiles (quintile I having the most deprived and quintile V the least deprived), and we calculated the amount of agreement in classifying the child into the same or a close quintile by the two methods. Second, we ranked children according to the scores obtained from the two methods and used the ranks to plot the data and draw the median trace (Figure 1). Third, we calculated Spearman's rank correlations of the various area-based SES indices with the individual-based SES indices. Finally, we used the ranks to calculate the intra-class correlation coefficient.²⁹

Statistical analysis was carried out using SAS statistical software.³⁰

FIGURE 1
Neighbourhood SES index-I (at census tract level) versus the individual SES score ranks

Results

From the 18 Montreal schools selected, 1,274 eligible children were identified, of whom 989 (77.6%) participated in the study; parental occupation could be coded into an SES score for 934 (94.4%) of these children. The postal codes for the addresses of 952 (96.3%) children could be linked to 117 corresponding census tract identifiers and 155 smaller enumeration area identifiers to determine area-based SES.

The 13 different area-based SES indices classified children as being within the same quintile as that determined by the individual level SES score 28.7% (± 2.8%) of the time. There was a discrepancy of one quintile in 35.3% (± 2.3%) of cases, two quintiles in 20.6% (± 3.6%), three quintiles in 11.3% (± 4.2%) and four quintiles in 4.1% (± 0.2%) of cases (Table 1). The degree of discrepancy was similar for area-based SES measures obtained at both the CT and EA level. Similarly, Spearman's rank correlations and the intra-class correlation coefficients between the area-based and individual level SES measures were always less than 0.40 (Table 2), suggesting little agreement. Figure 1 shows a scatter diagram with a median trace of the relation between occupation-derived, area-based SES measure (neighbourhood SES index-I) and the individual level SES. A wide variability around the median trace is evident.

Agreement of the various area-based SES indices with parental education, single-parent family status and crowding index obtained at an individual level was also poor and did not differ from that obtained for the individual level SES index (data not shown).

Discussion

We found poor agreement between the area-based and the individual level SES measures even when we used them to classify children into broad SES categories. The lack of agreement appeared across a wide distribution of both individual and neighbourhood indicators of SES, and a large number of small areas contained in various neighbourhoods were included. Furthermore, there were no important differences in terms of neighbourhood indicators of poverty between selected and non-selected schools, thus limiting selection bias.

Various methods and information have been used in forming area-based SES indices. In a British study, Campbell et al.²² used unemployment rates in different areas. Wegner combined (with equal weighting) the z scores of mean years of education and mean income of subjects residing in a given census tract.²³ From examining determinants of mortality in Connecticut and Rhode Island, Stockwell²⁴ obtained the mean of the percentile scores for occupation, education and median income by census tract and subsequently modified this by adding the rental value of housing and the extent of overcrowding in dwelling units within each census tract. There is no standard way of constructing area-based indices, and the various methods used in the literature have not been compared with or rated on their relative proximity to information on SES that is specific to the individual.

In this study we compared and rated 13 different variables thought to be related to SES that could be derived from area-based information, and all correlated poorly with the individual SES measure. Furthermore, there were no substantial differences among them as to their proximity to SES assessed individually.

The strength of association between SES and disease outcome has been reported to vary according to the level of small area used in the analysis. For instance, in a study in which census tract was used as a small area of analysis, median rental values of residences were not found to be associated with endometrial cancer;⁷ however, the same variable was associated with endometrial cancer when the unit of analysis was a residential block.⁸

In the present study, there was no substantial improvement when the smaller and potentially more homogeneous enumeration areas were used as opposed to the larger census tracts. This may have been the result of a close correspondence between enumeration areas and census tracts in our study sample, given that many more than 155 enumeration areas would be expected from 117 census tracts. One possible explanation for the small number of enumeration areas is that the school catchment areas were fairly small and may have included only a few enumeration areas from each census tract.

Individual level SES was used as a standard of reference with which area-based SES measures were compared. We used Blishen's SES index to measure individual level SES.¹⁶ Parents were directly interviewed to obtain detailed employment histories, and this information was used to discriminate between the various occupations with similar job titles. The SES score assigned for each occupation has been developed after substantial research and is based on objective measures of educational level and income for each occupation. Blishen's SES index, developed for occupations in Canada, and its US counterpart (Duncan's SES index) are considered to be the most appropriate measures of ranking North American societies in terms of social and economic conditions. These composite measures are the most frequently used in social science research,³¹ although they may not represent a true gold standard, given the complexities involved in the assessment of socio-economic status.

The poor agreement between the various area-based and individual SES measures is unlikely to have been due to intrinsic problems of the individual level Blishen ratings, because the area-based Blishen index (neighbourhood SES index-I) was found to have a similarly low agreement with the individual measures as the other area-based indices.

Persons of low and high SES tend to reside in certain areas within cities. Area-based measures of low SES appear to reflect a similar construct to that of low SES measured at the individual level, as shown in studies linking both levels of SES measures to various health indicators.⁴ Area-based SES scores have been shown to be inversely related to all-cause mortality.²⁴ Among women, strong correlations have been found between standardized mortality ratios and area-based socio-economic variables.³² Area-based SES variables have been linked to low birth weight, proportion of teenage mothers, non-immunized children and height as well as other potential markers of poor health.^23,33

At issue in this type of analysis is the extent of measurement error that results when area-based SES measures are used. We expect the misclassification associated with area-based SES measures to be non-differential (i.e. the probabilities of exposure misclassification are the same in all groups being compared and unrelated to disease status). The effect of unreliable measurements, therefore, is to attenuate exposure-outcome correlation or regression coefficients.²⁹ One consequence of attenuation is that a sample estimate of the observable correlation may fail to reach statistical significance, whereas a sample estimate of the correlation using more precise scores might be significant. The use of area-based SES measures may therefore involve use of larger sample sizes and incur additional costs.

A recent report from the US used a national sample to examine the use of census-based aggregate variables as a proxy for SES, concluding that associations of health outcomes with aggregate SES measures were substantially weaker than with micro-level measures.¹³ From the intra-class correlation coefficients presented in Table 2, one can calculate a correction factor for the attenuated correlations.²⁹

We used the 1986 Canadian census data to construct the area-based SES measures, whereas we obtained parental occupation for the period from April 1990 to November 1992. This time difference may have contributed to the lack of correlation between the individual and area-based measures of SES. Census data are collected only every 5 or 10 years in most countries, so that similar time differences are a regular feature of using area-based measures of SES. Geronimus and Bound¹³ reported that the time lag between the primary data set being analyzed and collection of the census data makes little difference to area-based SES measures in predicting health outcomes.

In conclusion, we observed a substantial discrepancy between area-based SES measures and SES assessed on the basis of individually obtained information. Caution should therefore be used in designing or interpreting the results of studies in which area-based SES measures are used to test hypotheses or control for confounding.

Acknowledgements

The study was approved by the Ethics Committee of the Department of Epidemiology and Biostatistics, McGill University, and was supported financially by the Medical Research Council of Canada and the Respiratory Health Network of Centres of Excellence (Canada).

References

1. Korff MV, Koepsell T, Curry S, Diehr P. Multi-level analysis in epidemiologic research on health behaviours and outcomes. Am J Epidemiol 1992;135:1077-82.

2. Greenland S, Robins J. Invited commentary: ecological studies-biases, misconceptions, and counterexamples. Am J Epidemiol 1994;139:747-60.

3. Morgenstern H. Ecologic studies in epidemiology: concepts, principles, and methods [review]. Annu Rev Public Health 1995;16:61-81.

4. Liberatos P, Link BG, Kelsey JL. The measurement of social class in epidemiology. Epidemiol Rev 1988;10:87-121.

5. Hakama M, Hakulinen T, Pukkala E, Saxen E, Teppo L. Risk indicators of breast and cervical cancer on ecologic and individual levels. Am J Epidemiol 1982;116:990-1000.

6. Kelsey JL, LiVolsi VA, Holford TR, Fischer DB, Mostow ED, Schwartz PE, et al. A case-control study of cancer of the endometrium. Am J Epidemiol 1982;116:333-42.

7. Graham S, Levin M, Lilienfeld AM. The socioeconomic distribution of cancer of various sites in Buffalo, NY, 1948-1952. Cancer 1960;13:180-91.

8. Elwood JM, Cole P, Rothman KJ, Kaplan SD. Epidemiology of endometrial cancer. J Natl Cancer Inst 1977;59:1055-60.

9. Chirikos TN, Reiches NA, Moeschberger ML. Economic differentials in cancer survival: a multivariate analysis. J Chronic Dis 1984;37:183-93.

10. Geronimus AT, Bound J, Neidert LJ. On the validity of using census geocode characteristics to proxy individual socioeconomic characteristics. J Am Stat Assoc 1996;91:529-37.

11. Krieger N, Williams DR, Moss NE. Measuring social class in US public health research: concepts, methodologies, guidelines. Annu Rev Public Health 1997;18:341-78.

12. Krieger N. Overcoming the absence of socioeconomic data in medical records: validation and application of a census-based methodology. Am J Public Health 1992;82:703-10.

13. Geronimus AT, Bound J. Use of census-based aggregate variables to proxy for socioeconomic group: evidence from national samples. Am J Epidemiol 1998;148:475-86.

14. Statistics Canada. Census of Canada 1986. Profiles, census tracts. Montreal, part I and II [computer files].

15. Statistics Canada (Standards Division). Standard Occupational Classification (1980). Ottawa; Cat 12-565E.

16. Blishen BR, Carroll WK, Moore C. The 1981 socioeconomic index for occupations in Canada. Rev Soc Anth 1987;24:465-88.

17. Stewart WF, Tonascia JA, Matanoski GM. The validity of questionnaire-reported work history in live respondents. J Occup Med 1987;29:795-800.

18. Statistics Canada (Geography Division). Postal code conversion file, January 1989 version. Ottawa, 1989.

19. Canada Post Corporation. Canada postal code directory. Ottawa: Canada Post Corporation, 1990.

20. Wilkins R. Use of postal codes and addresses in the analysis of health data. Health Reports 1993;5:157-77.

21. Statistics Canada. 1986 Census dictionary. Ottawa, 1987; Cat 99-101E.

22. Campbell DA, Radford JMC, Burton P. Unemployment rates: an alternative to the Jarman index? BMJ 1991;303:750-5.

23. Wegner EL. The computation of socioeconomic status scores for census tracts in Hawaii and suggestions for their use in health research. Research and Statistics Office, Hawaii State Department of Health; 1980. (ISSN
0145-9643).

24. Stockwell EG. A critical examination of the relationship between socioeconomic status and mortality. Am J Public Health 1963;53:956-64.

25. Reijneveld SA, Gunning-Schepers LJ. Age, socioeconomic status, and mortality at the aggregate level. J Epidemiol Community Health 1994;48:146-50.

26. Brennan ME, Lancashire R. Association of childhood mortality with housing status and unemployment. J Epidemiol Community Health 1978;32:28-33.

27. Sargent JD, Bailey A, Simon P, Blake M, Dalton MA. Census tract analysis of lead exposure in Rhode Island children. Environmental Research 1997;74:159-68.

28. Crosse EA, Alder RJ, Ostbye T, Campbell MK. Small area variation in low birthweight: looking beyond socioeconomic predictors. Can J Public Health 1997;88:57-61.

29. Fleiss JL. The reliability of measurement. In: Fleiss JL. The design and analysis of clinical experiments. New York: John Wiley & Sons, 1986:1-32.

30. SAS Institute Inc. Statistical Analysis System, Version 6. Cary (NC): SAS Institute Inc., 1989.

31. Powers MG. Measures of socioeconomic status: an introduction. In: Powers MG, ed. Measures of socioeconomic status: current issues. Boulder (CO): Westview Press, 1982.

32. Alexander FE, O'Brien F, Hepburn W, Miller M. Association between mortality among women and socioeconomic factors in general practices in Edinburgh: an application of small area statistics. BMJ 1987;295:754-6.

33. Reading R, Jarvis S, Openshaw S. Measurement of social inequalities in health and use of health services among children in Northumberland. Arch Dis Child 1993;68:626-31.

Kitaw Demissie, Dick Menzies and Pierre Ernst, Respiratory Epidemiology Unit, Department of Epidemiology and Biostatistics, McGill University, Montreal, Quebec

James A Hanley and Lawrence Joseph, Department of Epidemiology and Biostatistics, McGill University, Montreal, Quebec

Correspondence: Kitaw Demissie, Department of Environmental and Community Medicine, University of Medicine and Dentistry of New Jersey, Robert Wood Johnson Medical School, 675 Hoes Lane, Piscataway, New Jersey, USA, 08854-5635; Fax: (732) 235-4569; E-mail: demisski@umdnj.edu

[Table of Contents [Next]