Ultra-fast preselection in lasso-type spatio-temporal solar forecasting problems

Solar forecasting using data collected by satellites or sensor networks is often framed as a many-predictor spatio-temporal regression problem. Whereas the regressand is the irradiance at the focal location, the regressors could be the irradiance at any neighboring location with any time lag. Lasso—the least absolute shrinkage and selection operator—is commonly used for regressor selection and regularization in order to enhance the forecast accuracy and interpretability of the regression model. However, when the number of regressors is much larger than the number of samples, lasso-type regressions are limited by the curse of dimensionality and the insufficient degree of freedom. The ultra-fast preselection algorithm herein proposed provides a remedy to the above-mentioned problem. The algorithm uses (probably) the world's fastest similarity search routine, and preselects a user-defined number of most-relevant regressors based on the z-normalized Euclidean distance. The preselected regressors are then fed to lasso for forecasting. The algorithm is completely free from the curse of dimensionality, and does not require any meteorological prior, such as wind information or cloud vector field. It is customizable for other forecasting methods, such as vector autoregression or kriging, and can be implemented in just 10 lines of code.