Enhancing Classification Modeling Through Feature Selection and Smoothness: a Conic-Fused Lasso Approach Integrated With Mean Shift Outlier Modelling

Abstract

Outlier detection and variable selection are among main objectives of statistical analysis. In our study, we address the outlier problem for classification by using the Mean Shift Outlier Model (CLMSOM). Since the MSOM has more coefficients than the linear regression model, the complexity of the model MSOM is high. Therefore, we consider feature selection for MSOM by using fused Lasso (FLasso), which is beneficial and helpful in the cases where the number of explanatory variables or features is greater than the sample size. FLasso is penalizing both the coefficients and their successive differences by the L-1-norm, and it allows sparsity for both of them, while Lasso only allows the coefficients by considering a nonsmooth optimization problem. In this study, we take into account Iterated Ridge approximation which enables us to use a smooth optimization for FLasso problem. Generated smooth optimization problem is solved by using one of continuous optimization techniques called Conic Quadratic Programming (CQP), which is enabling the utilization of interior point methods. The newly developed method is called Conic FLasso for classification by MSOM (C-FLasso-CLMSOM) and is applied to real world data set to show its performance.