Multi-scale feature fusion for facial expression recognition

As an important computer vision task that can be used in many areas, facial expression recognition (FER) has been widely studied which much progress has been obtained especially when deep learning (DL) approaches have been introduced in FER. However, existing methods are restricted to extracting localized information or utilizing attention mechanism to focus on the region most relevant to the emotion, possibly resulting in errors in facial expression interpretation. Moreover, the inconsistency of image sizes in the wild FER datasets further increases the difficulty of the FER task. To address these issues, we propose a multi-scale feature fusion (MSFF) for FER. Specifically, our method first extracts features from facial images using multi-scale feature extraction to obtain rich multi-dimensional information. Meanwhile, extensive feature extraction at different scales is achieved by expanding the receptive field of the model, which provides the basis for more detailed analysis of facial expressions. Eventually, features at different scales are fused to enhance their synergistic effect. Subsequently, the adaptive extraction of comprehensive and valuable information from the fused features is performed, optimizing the performance of expression recognition. Extensive experimental results demonstrate that the proposed method outperforms the state-of-the-art methods on in-the-wild datasets. It is promising for FER in realistic and complex scenarios.