Structured detail enhancement for cross-modality face synthesis

Transformation, matching, and retrieval of face images in different modalities are often encountered in many applications. Cross-modality face synthesis, which aims to transform face images between modalities, can serve as an intermediate representation for these tasks, and remains an active yet challenging topic in low level vision. Most existing cross-modality face synthesis methods are effective in recovering global features for a given photo, but fail to capture fine-scale details or introduce some extra noise in the synthesis results. In this paper, we propose a two-step algorithm to tackle this problem by enhancing structured details after initial synthesis. In the first step, a combination of quasi-smooth area synthesis and facial component synthesis is proposed to generate more plausible facial components. In the second step, an MAP-based model is constructed to introduce more facial details into the initial synthesis, meanwhile eliminate noise. Experiments on face sketch synthesis and near-infrared face synthesis demonstrate the effectiveness of the proposed method in enhancing fine-scale facial details while reducing noise.