Statistical significance
Online posting: April 12, 1996
Published in print: June 1, 1996 (CMAJ 1996;154:1621)
Re: Bucking the trend [letter], by Andrew Murray, response by Bruce P. Squires, CMAJ 154: 758
This letter and the response seem to miss the point of
statistical testing, which is simply to estimate the probability
that the results found in a sample are valid. That probability is
a continuum. When we classify probability as "significant" and
"nonsignificant," we lose a great deal of important information.
The arbitrary choice of a Type I error of 5% as the threshold for
significance simply means that there is a 95% chance that the
sample results truly represent the results that would be found in
the population. Does it make sense to accept a finding if there
is a 95% chance that it is valid, but reject it if there is only,
for example, a 94% chance that it is valid (p <0.06), which would be considered statistically nonsignificant by the arbitrary standard? Sometimes, perhaps, it would make sense to reject such a result; at other times, a 94% probability, or even a 90% probability, would be enough to guide clinical activity.
Information on the qualitative direction and strength of a trend
is surely worth while to the clinician. Much more useful are the
confidence limits (to any required degree of precision) of the
experimental results.
Murray W. Enkin
Professor emeritus
Department of Obstetrics and Gynaecology
Department of Clinical Epidemiology and Biostatistics
McMaster University
Hamilton, Ont.
enkin@fhs.mcmaster.ca
[ The editor-in-chief responds: ]
Dr. Enkin is, of course, correct in pointing out that probability represents a continuum and that the selection of a 95% threshold as an indication that a finding is valid is arbitrary. I also agree that confidence limits are much more useful. But I continue to object to expressing probabilities that approach, but do not meet, defined limits as "trends." By all means, report the p value, and let readers draw their own conclusions; calling such results "trends," however, gives them more validity than they deserve.
Bruce P. Squires, MD, PhD
Editor-in-chief
CMAJ email
CMAJ Readers' Forum