A Method to Estimate Regression Model Confidence Interval and Risk of Artificial Neural Network Model

Abstract

Artificial neural networks (ANNs) have found increasing usage in regression problems because of their ability to map complex nonlinear relationships. In recent years, ANN regression model applica-tions have rapidly increased in the engine calibration and controls area. The data used to build ANN models in engine calibration and controls area generally consists of noise due to instrument error, sensor precision, human error, stochastic process, etc. Filtering the data helps in reducing noise due to instrument error, but noise due to other sources still exist in data. Furthermore, many researchers have found that ANNs are susceptible to learning from noise. Also ANNs cannot quantify the uncer-tainty of their output in critical applications. Hence, a methodology is developed in the present manuscript which computes the noise-based confidence interval using engine test data. Moreover, a method to assess the risk of ANN learning from noise is also developed. The noise-based confi-dence prediction methodology does not make any unreasonable assumptions about the data and explores the parameter space. The developed method is based on the Bayesian neural network (BNN), and a key input parameter to the BNN is the likelihood standard deviation. A novel method is developed to predict the noise standard deviation, and when this noise standard deviation is inputted as likelihood standard deviation, the noise-based confidence interval is computed by the BNN. The risk assessment methodology of ANN learning from noise is based on the BNN distribution of the regression metric used for the ANN. The developed methodology is illustrated on two engine datasets: an engine friction dataset and an engine torque dataset. The datasets were used to compare the confidence interval of the developed method with confidence interval predictions of the stochastic Kriging regression model. From the study, it can be seen that the present methodology neither makes stationary noise nor minimum noise at training points assumptions which are present in the Kriging model. Moreover, from the engine torque dataset, it has been shown that even though two models can have similar regression metrics, their risk of learning from noise can be vastly different.