Searching for Adverse Effects in MEDLINE and EMBASE Requires a Combined Approach for Efficient Retrieval

Abstract

A review of:

Golder, Su, Heather M. McIntosh, Steve Duffy, and Julie Glanville. “Developing Efficient Search Strategies to Identify Reports of Adverse Effects in MEDLINE and EMBASE.” Health Information & Libraries Journal 23.1 (Mar. 2006): 3-12.

Objective – To assess the sensitivity and precision of various search strategies for retrieving adverse effects studies from the MEDLINE and EMBASE databases.

Design – Analytical survey.

Subjects – A case study using a recently published systematic review of the effectiveness and adverse effects of seven new anti-epileptic drugs.

Setting – MEDLINE and EMBASE searches performed by researchers at the Centre for Reviews and Dissemination and the UK Cochrane Centre Search Filters Design Group at the University of York, UK.

Methods – Five key approaches to searching were defined. The first approach used either text words or controlled vocabulary to search for specific adverse effects. The second used subheadings or qualifiers either attached to drug names found in the controlled vocabulary (approach 2a) or ‘floating’ without drug names (approach 2b). The third approach used text words as synonyms for the phrase ‘adverse effects.’ The fourth used controlled indexing terms for adverse effects. The fifth and final approach used two published search strategies incorporating study design (Badgett et al., Loke et al.).

These five approaches were used to search for studies of the adverse effects of seven new anti-epileptic drugs. 5,011 unique papers were retrieved. Of these, 236 were judged potentially relevant and 225 full text articles were obtained. The inclusion criteria from a previously published systematic review (Wilby et al.) were applied to the papers, and 79 met the criteria. Five papers were added to the set after being identified from reference lists, clinical experts, and other sources. This new set of 84 studies was used as a quasi gold standard (QGS) against which more than 300 combinations of the five approaches could be tested. To create the set of possible approaches, the researchers combined search strategies one through four in all possible ways, and used all available subheading combinations from 2a and 2b. The Badgett and Loke searches were tested separately.

Main Results – Sensitivity and precision were determined for each combination. Formulas used to calculate sensitivity and precision were provided. In MEDLINE, search strategies using floating subheadings achieved the highest sensitivity. The most useful single subheading in both MEDLINE and EMBASE was “adverse effects,” with 79.1% and 79.5% sensitivity respectively. Of the more than 300 combinations tested, the most sensitive combination in MEDLINE included specified adverse effects in combination with the floating subheadings “adverse effects,” “complications,” and “drug effects,” together with text words for adverse effects. This strategy had 97.0% sensitivity, but low precision at 2.8%. The highest precision was achieved by using subheadings attached to drug indexing terms. In EMBASE, the strategy of Loke et al. provided the highest sensitivity at 86.3% and precision of 2.0%. Since researchers are not likely to know in advance all of the reported adverse effects of a particular drug therapy, the most sensitive strategies without specific adverse events were also identified. The search with the highest sensitivity in MEDLINE had 95.5% sensitivity, and 97.3% sensitivity in EMBASE.

Conclusion – Searching for adverse effects requires a combination of approaches in both MEDLINE and EMBASE. In MEDLINE, the most sensitive combination yielded 97.0% sensitivity. Regardless of the approach used, precision remains low. An effective generic search filter for adverse effects searches may not yet be feasible. More research is needed on search strategies, as well as more consistent methods of reporting and indexing adverse effects.