Estimating Optimal Feature Subsets Using Efficient Estimation of High-Dimensional Mutual Information
- 31 January 2005
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 16 (1), 213-224
- https://doi.org/10.1109/tnn.2004.841414
Abstract
A novel feature selection method using the concept of mutual information (MI) is proposed in this paper. In all MI based feature selection methods, effective and efficient estimation of high-dimensional MI is crucial. In this paper, a pruned Parzen window estimator and the quadratic mutual information (QMI) are combined to address this problem. The results show that the proposed approach can estimate the MI in an effective and efficient way. With this contribution, a novel feature selection method is developed to identify the salient features one by one. Also, the appropriate feature subsets for classification can be reliably estimated. The proposed methodology is thoroughly tested in four different classification applications in which the number of features ranged from less than 10 to over 15000. The presented results are very promising and corroborate the contribution of the proposed feature selection methodology.Keywords
This publication has 18 references indexed in Scilit:
- Input feature selection by mutual information based on Parzen windowIeee Transactions On Pattern Analysis and Machine Intelligence, 2002
- Input feature selection for classification problemsIEEE Transactions on Neural Networks, 2002
- A Comparative Study of Feature-Salience Ranking TechniquesNeural Computation, 2001
- Wrappers for feature subset selectionArtificial Intelligence, 1997
- Neural-network feature selectorIEEE Transactions on Neural Networks, 1997
- Feature selection for classificationIntelligent Data Analysis, 1997
- ESTIMATION OF MUTUAL INFORMATION USING KERNEL DENSITY ESTIMATORSPhysical Review E, 1995
- Using mutual information for selecting features in supervised neural net learningIEEE Transactions on Neural Networks, 1994
- Mutual information functions versus correlation functionsJournal of Statistical Physics, 1990
- On Estimation of a Probability Density Function and ModeThe Annals of Mathematical Statistics, 1962