Evaluation of clinical practice guidelines

Table 1: A framework for the evaluation of clinical practice guidelines

Category/factor Example/comments

Type of evaluation

Inception evaluation Need for new or revised guidelines

Guidelines program evaluation Assessment of health care program within which guidelines play a central role

Scientific evaluation Basic questions to be answered regarding development and implementation of guidelines

Goals of guidelines

Guidelines development Incorporation of new evidence into current practice
Improvement of current practice
Improvement of acceptance of guidelines by practitioners or public
Improvement of feasibility of guideline use

Guidelines program Improvement of quality of health care or health outcomes
Improvement of access to services
Decreased cost or improvement of resource utilization or both

Goals of evaluation

Inception evaluation Exploration (hypothesis-generating, qualitative assessment)

Guidelines program evaluation Demonstration of expected gains in quality of process

Scientific evaluation Influence of evidence-based versus authoritative format for guidelines

Design

Structure Cross-sectional, before-after, time series, comparative uncontrolled, crossover or randomized controlled trial

Sampling Retrospective or prospective

Unit of analysis

Patient encounter/episode; practice; hospital; system Randomization of hospitals to receive one of several strategies to implement guidelines; physician profiling

Measurement

Process (formative) Knowledge of guidelines, compliance, appropriateness

Impact (intermediate measures) Cesarean section rates; volume of tests ordered; provider satisfaction; laboratory test volume

Outcome Patient health status (e.g. morbidity, mortality, readmission rate, complications, requirements for intensive care)

Data sources

Routine Administrative data, clinical records

Unique Surveys, clinical data collection specific to evaluation

Analysis

Target Assessment against prespecifried benchmark

Comparative Interpractitioner or interhospital comparisons, analysis of small-area variations

Cofactors

Factors likely to influence effectiveness of guidelines Voluntary adoption of guidelines; participation in development or evaluation of guidelines; credentialling; incentives

Return to text

Table 1: A framework for the evaluation of clinical practice guidelines
Category/factor	Example/comments
Type of evaluation
Inception evaluation	Need for new or revised guidelines
Guidelines program evaluation	Assessment of health care program within which guidelines play a central role
Scientific evaluation	Basic questions to be answered regarding development and implementation of guidelines
Goals of guidelines
Guidelines development	Incorporation of new evidence into current practice Improvement of current practice Improvement of acceptance of guidelines by practitioners or public Improvement of feasibility of guideline use
Guidelines program	Improvement of quality of health care or health outcomes Improvement of access to services Decreased cost or improvement of resource utilization or both
Goals of evaluation
Inception evaluation	Exploration (hypothesis-generating, qualitative assessment)
Guidelines program evaluation	Demonstration of expected gains in quality of process
Scientific evaluation	Influence of evidence-based versus authoritative format for guidelines
Design
Structure	Cross-sectional, before-after, time series, comparative uncontrolled, crossover or randomized controlled trial
Sampling	Retrospective or prospective
Unit of analysis
Patient encounter/episode; practice; hospital; system	Randomization of hospitals to receive one of several strategies to implement guidelines; physician profiling
Measurement
Process (formative)	Knowledge of guidelines, compliance, appropriateness
Impact (intermediate measures)	Cesarean section rates; volume of tests ordered; provider satisfaction; laboratory test volume
Outcome	Patient health status (e.g. morbidity, mortality, readmission rate, complications, requirements for intensive care)
Data sources
Routine	Administrative data, clinical records
Unique	Surveys, clinical data collection specific to evaluation
Analysis
Target	Assessment against prespecifried benchmark
Comparative	Interpractitioner or interhospital comparisons, analysis of small-area variations
Cofactors
Factors likely to influence effectiveness of guidelines	Voluntary adoption of guidelines; participation in development or evaluation of guidelines; credentialling; incentives