TRCDNet: A Transformer Network for Video Cloud Detection

In remote-sensing image (RSI) preprocessing steps, detecting and removing cloudy areas is a critical task. Recently, cloud detection methods based on deep neural networks achieve outstanding performance over traditional methods. Current approaches mostly focus on cloud detection on a single image captured by polar-orbiting satellites. However, there is another type of meteorological satellite - geostationary satellite, which can capture temporal consecutive frames of a particular location. Therefore, the cloud detection task targeting a geostationary satellite can be treated as a video cloud detection task. And in addition to extracting features on a single image, extracting and making full use of the relations between sequential frames is also important. To tackle this problem, we design a deep-learning video cloud detection model: transformer network for video cloud detection (TRCDNet). The proposed network is based on the encoder-decoder structure. In the encoder, the module ContextGhostLayer is proposed to encode more semantic information to tackle challenging problems like thin clouds in RSIs. Besides, we design a transformer-based video sequence transformer (VSTR) block. Based on the attention mechanism, VSTR can fully extract the across-frame relations. In the proposed decoder, the cloud masks are recovered gradually to the same scale as the input image. To evaluate the methods, we create a Video Cloud Detection dataset based on the captured videos from Fengyun 4 (FY-4) satellite: Fengyun4aCloud. Extensive experiments of current cloud detection methods, semantic segmentation methods, and video semantic segmentation (VSS) methods indicate that the designed TRCDNet achieves state-of-art performance in video cloud detection.