Permutation Tests for Classification: Towards Statistical Significance in Image-Based Studies
- 1 January 2003
- book chapter
- conference paper
- Published by Springer Science and Business Media LLC in Lecture Notes in Computer Science
- Vol. 18, 330-341
- https://doi.org/10.1007/978-3-540-45087-0_28
Abstract
Estimating statistical significance of detected differences between two groups of medical scans is a challenging problem due to the high dimensionality of the data and the relatively small number of training examples. In this paper, we demonstrate a non-parametric technique for estimation of statistical significance in the context of discriminative analysis (i.e., training a classifier function to label new examples into one of two groups). Our approach adopts permutation tests, first developed in classical statistics for hypothesis testing, to estimate how likely we are to obtain the observed classification performance, as measured by testing on a hold-out set or cross-validation, by chance. We demonstrate the method on examples of both structural and functional neuroimaging studies.Keywords
This publication has 17 references indexed in Scilit:
- Discriminative Analysis for Image-Based StudiesLecture Notes in Computer Science, 2002
- Shape versus Size: Improved Understanding of the Morphology of Brain StructuresLecture Notes in Computer Science, 2001
- Automated manifold surgery: constructing geometrically accurate and topologically correct models of the human cerebral cortexIEEE Transactions on Medical Imaging, 2001
- Measuring the thickness of the human cerebral cortex from magnetic resonance imagesProceedings of the National Academy of Sciences of the United States of America, 2000
- High-resolution intersubject averaging and a coordinate system for the cortical surfaceHuman Brain Mapping, 1999
- Cortical Surface-Based Analysis: II: Inflation, Flattening, and a Surface-Based Coordinate SystemNeuroImage, 1999
- Cortical Surface-Based Analysis: I. Segmentation and Surface ReconstructionNeuroImage, 1999
- Global, voxel, and cluster tests, by theory and permutation, for a difference between two groups of structural MR images of the brainIEEE Transactions on Medical Imaging, 1999
- Hippocampal morphometry in schizophrenia by high dimensional brain mappingProceedings of the National Academy of Sciences of the United States of America, 1998
- The Jackknife, the Bootstrap and Other Resampling PlansPublished by Society for Industrial & Applied Mathematics (SIAM) ,1982