Choosing the optimal hybrid covariance estimators in adaptive elastic net regression models using information complexity

Abstract

Elastic net (EN) is a regularization technique which is used for modelling and variable selection with high-dimensional data at the same time. In the literature, it is claimed that EN modelling can be used for undersized samples with high dimensions (i.e. $n<<p$). But, both the model matrix and the Gram matrix are not of full rank $p$ and the inverse of the Gram matrix cannot be calculated. It degenerates and becomes singular. To overcome this problem in EN modelling, Mohebbi et. al. [A new data adaptive elastic net predictive model using hybridized smoothed covariance estimators with information complexity. J Stat Comput Simul. 2019;89(6):1060–1089] purposed a new adaptive elastic net (AEN) modelling using hybrid covariance estimators (HCEs) and information complexity (ICOMP) criteria. There are several forms of HCEs which can be used in AEN regression modelling. Thus, how to decide which HCEs is appropriate is an important problem to be solved. In this paper, we study the performance of the AEN models under several different HCEs using the ICOMP criterion for both the implementation of experimental data and Monte Carlo simulation study with different scenarios of the protocol.