Dimensionality reduction algorithms help reduce the classification time and sometimes the classification error of a classifier (Yang, et al., 1997). For time critical applications, in order to have reduction in the feature acquisition phase, feature selection methods are more preferable to dimensionality reduction methods, which require measurement of all inputs. Traditional feature selection methods, such as forward or backward feature selection, are costly to implement. In this study, we introduce a new feature selection method that decides on which features to retain, based on how PCA (principal component analysis) or ICA (independent component analysis) (Hyvarinen and Oja, 1999) values those features. We compare the accuracy of our method to backward and forward feature selection with the same number of features selected and PCA and ICA using the same number of principal and independent components. For our experiments, we use spectral measurement data taken from corn kernels infested and not infested by fungi.