Cross-Scene Deep Transfer Learning with Spectral Feature Adaptation for Hyperspectral Image Classification

The small size of labeled samples has always been one of the great challenges in hyperspectral image (HSI) classification. Recently, cross-scene transfer learning has been developed to solve this problem by utilizing auxiliary samples of a relevant scene. However, the disparity between hyperspectral datasets acquired by different sensors is a tricky problem which is hard to overcome. In this article, we put forward a cross-scene deep transfer learning method with spectral feature adaptation for HSI classification, which transfers the effective contents from source scene to target scene. The proposed framework contains two parts. First, the distribution differences of spectral dimension between source domain and target domain are reduced through a joint probability distribution adaptation approach. Then, a multiscale spectral-spatial unified network with two-branch architecture and a multiscale bank is designed to extract discriminating features of HSI adequately. Finally, classification of the target image is achieved by applying a model-based deep transfer learning strategy. Experiments conducted on several real hyperspectral datasets demonstrate that the proposed approach can explicitly narrow the disparity between HSIs captured by different sensors and yield ideal classification results of the target HSI.