Parameter-Free Loss for Class-Imbalanced Deep Learning in Image Classification
- 15 September 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 34 (6), 3234-3240
- https://doi.org/10.1109/tnnls.2021.3110885
Abstract
Current state-of-the-art class-imbalanced loss functions for deep models require exhaustive tuning on hyperparameters for high model performance, resulting in low training efficiency and impracticality for nonexpert users. To tackle this issue, a parameter-free loss (PF-loss) function is proposed, which works for both binary and multiclass-imbalanced deep learning for image classification tasks. PF-loss provides three advantages: 1) training time is significantly reduced due to NO tuning on hyperparameter(s); 2) it dynamically pays more attention on minority classes (rather than outliers compared to the existing loss functions) with NO hyperparameters in the loss function; and 3) higher accuracy can be achieved since it adapts to the changes of data distribution in each mini-batch instead of the fixed hyperparameters in the existing methods during training, especially when the data are highly skewed. Experimental results on some classical image datasets with different imbalance ratios (IR, up to 200) show that PF-loss reduces the training time down to 1/148 of that spent by compared state-of-the-art losses and simultaneously achieves comparable or even higher accuracy in terms of both G-mean and area under receiver operating characteristic (ROC) curve (AUC) metrics, especially when the data are highly skewed.Keywords
Funding Information
- National Natural Science Foundation of China (62006160)
- Educational Commission of Guangdong Province (2020KQNCX062)
- Shenzhen Fundamental Research Program (20200813102946001)
- Science and Technology Development Fund, Macau (0112/2020/A, 004/2019/AFJ)
- National Natural Science Foundation of China (81771922, 62071309, 61801305, 81971585, 61871274)
This publication has 28 references indexed in Scilit:
- A regularized ensemble framework of deep learning for cancer detection from multi-class, imbalanced training dataPattern Recognition, 2018
- Focal Loss for Dense Object DetectionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2017
- Cost-Sensitive Learning of Deep Feature Representations From Imbalanced DataIEEE Transactions on Neural Networks and Learning Systems, 2017
- Dynamic Weighted Majority for Incremental Learning of Imbalanced Data Streams with Concept DriftPublished by International Joint Conferences on Artificial Intelligence ,2017
- Deep Residual Learning for Image RecognitionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2016
- Learning Deep Representation for Imbalanced ClassificationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2016
- Maximum Margin of Twin Spheres Support Vector Machine for Imbalanced Data ClassificationIEEE Transactions on Cybernetics, 2016
- Bilayer Sparse Topic Model for Scene Analysis in Imbalanced Surveillance VideosIEEE Transactions on Image Processing, 2014
- Speech recognition with deep recurrent neural networksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2013
- Learning from Imbalanced DataIEEE Transactions on Knowledge and Data Engineering, 2009