Significant? Maybe, but how valid?
Online posting: June 20, 1996
Published in print: Sept 1, 1996 (CMAJ 1996;155:513)
Re: Statistical significance [letter], by Murray W. Enkin and Bruce
P. Squires, CMAJ 1996;154:1621.
This recent letter focused on whether a finding is statistically
significant, but it also reflected the widely held perception
that statistical significance, however defined, indicates
validity. It does not. Significance testing tells us the
likelihood that an observation is due to chance; it is a measure
of a finding's precision. The internal validity of an observation
is a function of the scientific rigour of the study's design and
execution (e.g., avoidance of bias and confounding variables). A
poorly designed study may identify an invalid observation that,
because of a large sample size, is statistically significant.
"Internal validity is the sine qua non of etiologic research;
thus, it is more important that an estimate be valid than that it
be precise."[1] Significance testing is a necessary tool for
assessing the role that chance may play in explaining an
observation, but the mere presence of statistical significance
does not ensure that an observation is valid.
Allen A. Mitchell, MD
Research professor of public health
Slone Epidemiology Unit
Boston University School of Public Health
Brookline, Mass.
Reference
- Kleinbaum DG, Kupper LL, Morgenstern H. Epidemiologic
research: principles and quantitative methods. Belmont, Calif:
Lifetime Learning Publications, 1982:90.
 CMAJ email
CMAJ Readers' Forum
CMAJ home page
|
