A lazy support vector regression model for prediction problems with small sample size

Prediction problems with small sample size are problems which widely exist in engineering application. Because lazy prediction algorithms can utilize the information of predicted individual, it is often possible for them to achieve better predictive effect. Traditional lazy prediction algorithms generally use sample information directly, and therefore the predictive effect still has room for improvement. In this paper, we combine support vector regression (SVR) with lazy prediction algorithm, and propose a lazy support vector regression (LSVR) model. The insensitive loss function in LSVR depends on the distance between the individual in training sample set and the predicted individual. The smaller the distance, the smaller the lossless interval of the individual in training sample set, which means that the individual in training sample set has a great impact on the predicted individual. To solve the LSVR model, a generalized Lagrangian function is introduced to obtain the dual problem of the primal problem, and the solution to the primal problem is obtained by solving the dual problem. Finally, three numerical experiments are conducted to validate the predictive effect of LSVR. The experimental results show that the predictive effect of LSVR is better than those of e-SVR, neural network (NN) and random forest (RF), and it is also better than that of k-nearest neighbor (k-NN) algorithm when the sample size is not too small and the distance between the predicted individual and the individual in training sample set is not too large. Therefore, LSVR not only has the advantage of good generalization ability of traditional SVR, but also has the advantage of good local accuracy of lazy prediction algorithm.