Potential pitfalls in the use of p-values and in interpretation of significance levels

Abstract

In multiparameter analysis of clinical data the likelihood of obtaining significant results, just by chance, increases considerably with the overall number of tests performed. This can be compensated for by adjusting the p-values. Two tables are given from which adjusted p-values may be read off, providing a simple procedure to test the reliability of the results from clinical studies. Multivariate analyses often impress with extremely low p-values, but frequently these results turn out to be non-significant when the influence of multiple testing is considered. As a consequence the actually relevant results might be overlooked, because of the large number of spurious results accumulating in the literature. The use of inappropriate statistics hampers progress in clinical research. It is concluded that more care in the use of p-values in analysis and interpretation of clinical data is required.