MM-RNN: A Multimodal RNN for Precipitation Nowcasting

Precipitation nowcasting, the high-resolution forecasting of precipitation in a short term, is essential in various applications in the real world. Previous deep learning methods use huge samples to learn potential laws, and the learning process lacks regularity, making it difficult to model the complex nonlinear precipitation phenomenon. Inspired by traditional numerical weather prediction models, we propose the multimodal recurrent neural network (MM-RNN), which introduces knowledge of elements to guide precipitation prediction. This constraint forces the movement of precipitation to follow the underlying atmospheric motion laws. MM-RNN not only can provide accurate precipitation nowcasting but other meteorological elements' predictions. Besides, it has high flexibility and is compatible with multiple RNN models, such as ConvLSTM, PredRNN, MIM, and MotionRNN. We conduct experiments on two multimodal datasets (MeteoNet and RAIN-F), and the results indicate that MM-RNN is superior to common RNN [multiscale RNN (MS-RNN)] using a single radar modality. For the MeteoNet, compared to MS-MotionRNN, the critical success index (CSI) ( $R\geqslant 10$ ) of MM-MotionRNN increases by 23.4%, and the mean square error (mse) of MM-MotionRNN decreases by 6.7%. For the RAIN-F, compared to MS-MIM, the HSS ( $R\geqslant 5$ ) of MM-MIM increases by 209.4%, and the balanced mse (B-MSE) of MM-MIM decreases by 4.6%.