JADFER: Exploring Spatial-Contextual Interaction With Joint Attention Dropping for Facial Expression Recognition

Facial Expression Recognition (FER) aims to cate-gorize emotional expressions depicted on a human face, and isa challenging task under unconstrained conditions, such as faceocclusions and pose variations. Recent methods usually adopt selfattention or cross attention to explore global or local relationshipsamong different level features. However, these methods are inclinedto focus on the redundant facial regions, causing model overfitting.To address this problem, we propose a new FER model namedJADFER, which drops the joint attention in the weight matrix toadaptively enhance facial expression representations. Specifically,our JADFER model consists of three components: Spatial Branch(SB), Contextual Branch (CB), and Spatial-Contextual Interaction(SCI). First, SB runs N paths in parallel, where a Variety loss is de-signed to guide the paths of SB to focus on different discriminativeregions. Meanwhile, CB abstracts the contextual facial represen-tations using self attention with Joint Attention Dropping (JAD).Then, the SCI adopts the spatial features from SB to query thecontextual representations from CB through cross attention withJAD, which regulates the attention weights by dropping the similaractivations to further enhance the facial embeddings. Experimen-tal results demonstrate that the proposed model outperforms thestate-of-the-art methods on several FER benchmarks.