Dictionary-Induced Manifold Learning for Incomplete Multi-modal Fusion

Data missing is a common problem in multi-modal fusion, and existing incomplete multi-modal methods usually only consider the case of two modalities and ignore the semantic information of samples during data recovery. In this paper, we propose dictionary-induced manifold incomplete multi-modal latent space representation, which reconstructs missing views with dictionary to assist consensus representation and captures the local manifold structure with reverse graph regularization. Specifically, we adopt dictionary learning to recover missing data with linear combinations of available samples for latent space alignment, and Laplacian matrix is utilized to embed the structural information of the high-dimensional space into the low-dimensional manifold latent space for optimizing the common representation. The proposed method can not only deal with multi-modal data fusion task, but also recovering missing data by effectively mining the structural information among different modalities. Experimental results demonstrate that our method performs better than other incomplete multi-modal fusion methods.