A Novel Comparison of Shrinkage Methods Based on Multi Criteria Decision Making in Case of Multicollinearity

Abstract

Streszczenie. Data analysis is very important in many fields of science. The most preferred methods in data analysis is linear regression due to its simplicity to interpret and ease of application. One of the assumptions accepted while obtaining linear regression is that there is no correlation between the independent variables in the model which refers to absence of multicollinearity. As a result of multicollinearity, the variance of the parameter estimates will be high and this reduces the accuracy and reliability of the linear models. Shrinkage methods aim to handle the multicollinearity problem by minimizing the variance of the estimators in linear model. Ridge Regression, Lasso, and Elastic-Net methods are applied to different simulated data sets with different characteristics and also real world data sets. Based on performance results, the methods are compared according to multi-criteria decision making method named TOPSIS, and the order of preference is determined for each data set. © (2024), (American Institute of Mathematical Sciences). All rights reserved.