Comparing two different approaches to measuring drug use within the same survey

C Ineke Neutel and Wikke Walop

Abstract

Respondents to the National Population Health Survey in Canada (1996-97) were asked two types of questions about drug use that allowed a comparison of the responses. The first question was about self-reported drug use categories: "In the past month, did you take [e.g., antidepressants]?" The second asked about specific drugs: "What specific medications did you take over the last two days?" Responses to the latter were coded according to the main chemical entity and then grouped in specific drug product categories similar to the first question's self-reported categories. The two sets of drug use categories were cross-tabulated for the 62,588 respondents who were 20 years of age and older. The proportion of persons who reported taking specific drugs who had not previously answered "yes" to the question related to the corresponding self-reported drug use category ranged from a low of 4.8% for insulin/oral hypoglycemics to a high of 43.7% for narcotic analgesics. Various reasons for these discrepancies are discussed. A series of logistic regression models relating the discrepancies to respondent characteristics shows that there is no clear pattern of variables associated with the discrepancies. These results show that surveys should be carefully planned to reflect the type of information needed.

Key words: drug classification; drug utilization; National Population Health Survey; pharmacoepidemiology; validation

Introduction

There are different parameters and aspects to measuring drug use in a population, each with implications for survey results. For example, the source of data, whether it be pharmacy, physician or billing records, will affect the quality and content of the drug use data. Although pharmacy records provide detailed information on the drugs themselves, data on the consumer are very limited. The latter could be improved if a patient is allowed to purchase drugs at only one designated pharmacy, as is the case in some European countries. Even in this situation, information such as the indication for use or the extent to which patients actually consume the drugs will remain largely unknown. Data from general practitioner (GP) records could be more informative about the indication for drugs prescribed, and on diagnoses and other health-related data, but these records are not always consistently completed. Billing records are another potential source of data, with the major limitation that in all provinces but Saskatchewan these data are available only on individuals who receive welfare benefits or who are over 65. Health surveys, such as Statistics Canada's National Population Health Survey, provide information on drug use from consumers themselves, as well as being a source of data on many other health-related issues.

Home inventories are considered by some to be the best method of obtaining accurate and complete drug use data.^1,2,3 In this scenario, an interviewer visits the home of the respondent and lists all of the drugs in the medicine cabinet. Lau et al. compared home inventory data with pharmacy records (where patients have been assigned to a pharmacy) and found considerable agreement. However, drug use obtained from carefully constructed questionnaires can also be accurate. Klungel et al. reported that more than 90% of drugs reported after "directed recall" were recorded in pharmacy records.^4,5 Less accurate was the percentage of drugs that ought to have been used by the patient according to pharmacy records (71% concordance). Sjahid et al. found 80% concordant pairs on comparing pharmacy records with patient interviews.⁶ In a military setting, self-report questionnaires showed a very high agreement (95%) with medical records. In another setting, medical records did not fare as well. For example, Heerdink et al. showed that GPs had recorded only 40% of drugs used by their patients, as learned in a home interview, while pharmacies had a record of 80%.⁷ Few studies compared different ways of collecting information from the general population within the same study.

De Jong compared three types of questions in an interview on drug use in pregnancy with the drugs in pharmacy records and found that questions involving indication for drug use- and drug-specific questions were more accurate than open-ended questions.⁸ The way the question on drug use is phrased is very important.

Collecting precise data during a survey on drug use is time-consuming and difficult. It is important to consider the way in which the drug use is recorded for the study, so that the data can be analyzed efficiently and accurately. Drugs can be recorded as specific chemical entities or specific drug products, e.g., lorazepam, fluoxetine, or can be grouped by chemical substance, e.g., benzodiazepines (BZD) or selective serotonin re-uptake inhibitors (SSRI). They can also be grouped by indication, e.g., cardiovascular drugs, which includes a variety of different types of drugs. The choice between these two methods is not always easy when planning a survey. Survey planners may be open to collecting actual drug names, but when they get to the final editing of the questionnaire and need to shorten the interview time, the larger drug categories may win.

Questions remain on the meaning of these categories. Can one assume that collecting the data as a drug category will give the same information as the grouping of specific drugs, e.g., to what extent will self-reported antidepressant use provide information on the use of tricyclics/SSRI, which are known to be the major antidepressant drug categories? The objective of this study is to determine the relation between self-reported drug use categories and the actual specific drug products that the respondents state they are taking.

Method

The National Population Health Survey (NPHS) is a large biannual Canadian health-related survey. The 1996-97 survey was conducted by telephone among residents in all provinces and territories, other than persons living on Indian reserves and Canadian Forces bases. Data collection took place in each of the four seasons and was carried out through a computer-assisted interviewing approach. Although the population included persons of 12 years of age and over, the present analysis will be limited to the responses of those 20 years of age and over. The NPHS 1996-97 survey used a statistical sampling design, making the study population representative of the Canadian population. However, for the present study statistical weighting will not be used since the major interest of the study is to compare answers to two sets of drug use questions by the same person, rather than drawing conclusions with respect to drug use in the Canadian population.

The first of the two sets of questions related to drug use over the past month: "Now, I would like to ask you a few questions about your use of medications, both prescription and over-the-counter as well as other health products. In the past month, did you take any of the following medications?" This was followed by a list of questions on 21 drug use categories. For the present study, three pairs of categories were combined:

1. Insulin and oral diabetic drug use were combined into one drug use category because it was suspected that there would be an overlap in users. Some results will be provided on the insulin and oral diabetic drug use separately.

2. Since antihypertensive drugs would also include diuretics, it was decided to combine the two questions on medicine for blood pressure, and diuretics or water pills, into one, for a better defined group.

3. The questions on tranquillizers and sedatives were combined since BZD are the most frequently prescribed drugs in both categories and the same drug could be given for either indication.

Respondents who answered "yes" to either one or both of these combined questions would be included in the relevant drug use category. Of the remaining questions, eight were omitted altogether, mainly because some questions, such as those on hormone replacement therapy and birth control, applied to women only, and questions concerning diet pills, allergy medication, cold medication, steroids, stomach pills and laxatives were more difficult to translate into specific drug use categories. Table 1 lists the 10 resulting categories that will be further explored and that will be called "self-reported drug categories" to signify that the respondents decided in which categories the drugs belonged.

The second set of questions was asked only of those respondents who had indicated drug use in the first set of questions. The question was worded as follows: "Now, I am referring to the past two days. During those two days, how many different medications did you take? What is the exact name of the medication to which you were referring?" The person was asked to look at the bottle, tube or box. The respondents were expected to collect all containers of drugs and related products and read the name of the drug or product from the label. The specific drugs or products were combined by the authors as much as possible into the 10 corresponding drug categories listed in Table 1. These are called "specific drug product" categories to stress that they are based on the exact drug product as distinguished from the "self-reported drug use" categories based on the first set of drug use category questions asked directly of the respondent.

The grouping of the specific drug products was made possible by coding the drugs using a Patented Medicine Prices Review Board (PMPRB) adaptation of the Anatomic Therapeutic Chemical (ATC) classification^9,10 that was further adapted for survey purposes by one of the authors. The ATC system consists of a seven-digit alphanumeric code based on anatomical, therapeutic, and chemical substance subgroups. It is a hierarchical classification that divides the drugs into 14 main groups and four levels of subgroups. The PMPRB adaptation consists of changes in the last two digits of the original WHO version. Further changes made for the NPHS allowed for coding of all the main chemical substances (e.g., M01AB04 for diclofenac), as well as combination drugs, (e.g., M01AB64 for diclofenac and misoprostol), and more general categories, (e.g., M01XX99 for anti-inflammatory for arthritis). The coding itself was done largely by computer, e.g., Ativan (or lorazepam) was automatically assigned the code N28GC07. Frequently misspelled drug names, if there was no possibility of confusion with another drug, were also coded by computer. If the drug name was severely misspelled, or if there was any difficulty in interpreting which drug was meant, the decision on its identity was made by one of the authors.

The two sets of drug use questions had different time frames: while self-reported drug categories covered 30 days before the interview, the specific drug question covered two days before the interview. Theoretically, the drugs recorded as having been consumed in the last two days should also have been included in the last 30-day category. The assumption might be that if there is an answer in the specific drug use category, there should be an answer in the corresponding self-reported drug use category. Analysis of the data sought to determine to what extent the assumption is not true. The first part of the analysis consisted of cross-tabulating the two sets of drug use categories and determining what proportion did not overlap as one would have expected. Subsequently, logistic regression was used to examine whether other variables, such as age, sex, education, marital and health status, were predictive of which respondents were most likely to answer the drug use questions as expected. For each model, the population was restricted to those answering "yes" in the specific drug use category and the dependent variable was the self-reported drug use category. For example, in the case of antibiotics, the probability was modelled so that respondents did not answer affirmatively to self-reported antibiotic use in the last 30 days while reporting using specific antibiotics in the last two days.

Results

Table 2 shows the number of people who answered affirmatively to the drug questions that fit in the 10 categories provided. The categories were ranked according to the percentages in the last column, i.e., the percentage of the drug use in the specific drug use category that was not included in the corresponding self-reported drug use categories. Nine hundred and eighty-two respondents listed antibiotics among the individual drugs that they had used in the past two days, but 17.8% of them had not answered "yes" to the question asking whether they had taken an antibiotic in the past month. Table 2 shows that the percentages of specific drug product use only (i.e., without answering the corresponding self-reported drug use category) ranged from a low of 4.8% for combined insulin and oral diabetic drug use to a high of 43.7% for narcotic analgesics. The percentage of specific drug product use only was also given separately for insulin and oral diabetic drug use and shows that each one has a higher individual percentage than the two combined.Table 3 presents the distribution of a series of variables among the study population in preparation for the logistic regression analysis of Table 4. In total there were 62,588 respondents, of whom 46.1% were male and 53.9% female. Most respondents were in the youngest of the three 20-year age groups and the number decreased with age. Current marital status categories were defined as "having a partner," i.e., married or common-law, and "without a current partner," i.e., single, divorced or widowed. The large majority of the population "had a partner."

In terms of the table contents, the most consistent result for the various drug categories is for age, where five of the 10 drug use categories showed a statistically significant odds ratio (OR) above 1.0, indicating that older people were less likely to have answered "yes" to the relevant self-reported drug category. However, the other five ORs in the column are near 1.0. For other variables, e.g., marital status, there were statistically significant ORs in both directions. Thus, having a partner appeared to make one more likely to report having taken narcotic analgesics in the last 30 days, but less likely to report thyroid medication and antibiotics.

Discussion

The results showed that survey respondents did not always answer in the affirmative to the use of drugs in the appropriate drug category when one considers their answers to subsequent questions about specific drugs that they had indicated using. For example, 17.8% of those who indicated taking specific antibiotics did not answer "yes" to the question asking them whether they had used antibiotics in the previous month. Similarly, almost half of the people who reported taking a narcotic analgesic in the last two days had not reported taking drugs in the narcotic analgesic category, which included Demerol, morphine or drugs with codeine.

To evaluate potential reasons for these discrepancies we will initially examine the quality of the data collection. The self-reported category is based on a fairly general question. The answer would combine the ability to recall having used a drug with the ability to interpret the question with some insight into what drugs would be included in this general class of medication. Because of these factors, the opportunity to be able to interpret respondents' replies was an important one and one of the reasons why this study was undertaken.

The more precise request for the names of specific medications was designed to discover as close to a home inventory of medications as is possible in a telephone interview. There was still an element of recall, depending on whether respondents remembered having taken any drugs at all in the last two days (or wanted to be bothered) or whether they kept their drugs in one place, e.g., the medicine cabinet, or had to remember which were stored in various locations in the home. The same problems would arise in the case of an interviewer who visits the home to take an inventory. The additional problem with a telephone survey is the need to read the label of the bottle and to be able to spell the sometimes difficult names. In general, the method of data collection in which the respondent is asked to collect all the containers and read the drug names off the labels would be the best possible approach for a telephone interview and as close to a home inventory as possible under the circumstances. Requesting both types of data within the same interview is an important opportunity to learn more about the meaning of these drug use questions and in particular the self-reported drug categories.

In spite of the data being reasonably accurate, it is clear from Table 2 that there is a considerable discrepancy between the self-reported drug use categories and the specific drug product categories, with the percentage of drugs reported in the latter categories varying from less than 5% to almost 50% for the various drug groups. A variety of reasons can be advanced for these discrepancies:

• Forgetfulness. The respondents may have forgotten that a drug was taken until they were asked to get all the containers. This would agree with de Jong et al.'s findings, in which the interviewer asked three successive drug use questions and found that more drugs were reported with each successive question.⁸
• Different types of questions. De Jong et al. found that different types of questions have different results, e.g., indication-specific and drug-specific questions were more accurate than open-ended questions.⁸ The larger self-reported drug categories used in the present study tended to be somewhat more indication-specific than drug-specific, although the indication may be only implied or the question may be a mixture of indication- and drug-specific components. For example, when people were asked whether they took "penicillin or other antibiotics," the question provided a mixture of a product-oriented component, "penicillin," and an indication-oriented one, "antibiotics."
• Terminology. To what extent do people know that antibiotics are the drugs taken for infections? This may appear to be self-evident to researchers and health professionals but it is possible that the health care provider may have used a different terminology, e.g., "I will give you something for your bladder problem," or "I will give you something for your infection," without using the word "antibiotics."
• Communication with health care providers. There may have been other gaps in communication between the respondent and his or her health care provider. The physician prescribed a drug and the patient did not understand exactly why he or she was given the drug or may have misunderstood the reason for taking the drug.
• Less obvious uses of the drug. Related to this is the possibility that a drug may have been given for a purpose that is different from its primary indication: an antidepressant may have been prescribed for difficulty sleeping, for example, if the physician suspected that depression was at the root of the insomnia. The patient would have thought of this drug as a sleeping pill rather than an antidepressant. Another example is that people taking acetylsalicylic acid (ASA) for prevention of heart problems might not have considered ASA to be an analgesic under the circumstances.
• Wording/order of the questions. One wonders whether suggesting a particular drug as an example of a drug category was helpful or whether it was more likely to confuse people. Was it useful or counterproductive to ask people whether they were taking tranquillizers such as Valium? People may never have heard of Valium, or if they had, what they heard may not have been very positive. They may not have realized that Ativan is the same type of drug as Valium and, in any case, they may not have wanted to admit that they were taking the same type of drug. In terms of order of questions, one would think that asking whether respondents were taking "medicine for blood pressure" followed immediately by a separate question about taking "diuretics or water pills," which are also mainly used as antihypertensives, could easily have led to confusion.

The series of logistic regression models did not present a consistent pattern of respondent characteristics that were associated with the presence or absence of the discrepancy between the two methods of collecting drug use information. For five of the drug categories older age seems to have been associated with the discrepancy, however, for the other five the OR was near 1.0. The other variables did not show any statistically significant association or may have shown statistically significant association in both directions for the same variable. The discrepancies between the two types of drug use measures were most likely a type of bias, e.g. collection bias rather than a factor that could have been calculated and allowed the use of the percentage as a correction factor. Asking the question in a different way may well have altered the results considerably.

These findings have important implications, both in terms of interpreting the results of a survey of this type and for planning further surveys with drug use questions. First of all, asking about specific drug products only from people who answered "yes" to any of the self-reported drug categories very likely leads to under-reporting of the latter, especially given the percentages in the last column of Table 2 where specific drugs are reported that were not included in the self-reported categories. Secondly, planners need to carefully consider what information on drug use they want to obtain from the survey. Asking about drug use categories rather than specific drug names will make the survey much easier to complete and will cut down the amount of time needed to complete the questionnaire. However, if one really wants to know what proportion of the elderly take BZD, then a category question about tranquillizers and sedatives as used in this survey will be misleading, as seen in Table 2. On the other hand, if one wants to know what proportion of people are treating their difficulty in sleeping with medication, then the question is fine as long as we realize that this is not a well-defined category in terms of the names of actual drugs.

References

1. Johnson RE, Vollmer WM. Comparing sources of drug data about the elderly. J Am Geriatr Soc 1991;39:1079-84.

2. Lau HS, de Boer A, Beuning KS, Porsius A. Validation of pharmacy records in drug exposure assessment. J Clin Epidemiol 1997;50:619-25.

3. Psaty BM, Lee M, Savage PJ, Rutan GH, German PS, Lyles M. Assessing the use of medications in the elderly: methods and initial experience in the Cardiovascular Health Study. J Clin Epidemiol 1992;45:683-92.

4. Klungel OH, de Boer A, Paes AHP, Herrings RMC, Seidell JC, Bakker A. Influence of question structure on the recall of self-reported drug use. J Clin Epidemiol 2000;53:273-7.

5. Klungel OH, de Boer A, Paes AH, Herings RM, Seidell JC, Bakker A. Agreement between self-recorded antihypertensive drug use and pharmacy records in a population-based study in The Netherlands. Pharm World Sci 1999;21:217-20.

6. Sjahid SI, van der Linden PD, Stricker BH. Agreement between the pharmacy medication history and patient interview for cardiovascular drugs: the Rotterdam elderly study. Br J Clin Pharmacol 1998;45:591-5.

7. Heerdink ER, Leufkens HG, Koppedraaijer C, Bakker A. Information on drug use in the elderly: a comparison of pharmacy, general practitioner and patient data. Pharm World Sci 1995;17:20-4.

8. De Jong-van den Berg LT, Waardenberg CM, Haaijer-Ruskamp FM, Dukes MN, Wesseling H. Drug use in pregnancy: a comparative appraisal of data collecting methods. Eur J Clin Pharmacol 1993;45:9-14.

9. Patented Medicine Prices Review Board. ATC Classification System for Human Medicines. Canadian Edition, 1994.

10. WHO Collaborating Centre for Drug Statistics Methodology/ Nordic Council on Medicines. Guidelines for ATC Classification. Oslo:1990. ISBN 82-90312-12-1.

C Ineke Neutel, Research Department, SCO Health Service, University of Ottawa Institute on Care of the Elderly, 43 Bruyère Street, Ottawa, Ontario K1N 5C8; fax: (613) 562-6387; e-mail: ineutel@scohs.on.ca

Wikke Walop, Vaccine Safety, Centre for Infectious Disease Prevention and Control, Health Canada, Ottawa, Ontario K1A 0L2