Can we gauge forecasts using satellite-derived solar irradiance?

Satellite-derived irradiance data, as an alternative to ground-based measurements, offer a unique opportunity to verify gridded solar forecasts generated by a numerical weather prediction model. Previously, it has been shown that the mean square errors (MSE) evaluated against ground-based measurements and satellite-derived solar irradiance are comparable, which might warrant the use of satellite-based products for regional forecast verification. In this paper, the 24-h-ahead hourly forecasts issued by the North American Mesoscale forecast system are verified against both ground-based (Surface Radiation Budget Network, or SURFRAD) and satellite-based (National Solar Radiation Data Base, or NSRDB) measurements, at all 7 SURFRAD stations over 2015-2016. Three different MSE decomposition methods are used to characterize - e.g., through association, calibration, refinement, resolution, or likelihood - how well the two types of measurements can gauge the forecasts. However, despite their comparable MSEs, NSRDB is found suboptimal in its ability to verify forecasts as compared to SURFRAD. Nonetheless, if a new forecasting model produces significantly better forecasts than the benchmarking model, satellite-derived data are able to detect such improvements and make conclusions. This article comes with supplementary material (data and code) for reproducibility.