Article Preview
TopIntroduction
Facial expressions are one of human beings’ most natural, powerful, and pervasive signals in expressing emotional states and intentions (Darwin & Prodger, 1998; Tian et al., 2001). As a result, facial expression recognition (FER) is widely used in social robotics, medical care, and driver fatigue checking. With the development of deep learning, the research hotspot on FER has shifted from shallow features to deep features, and researchers have made significant progress in FER by improving algorithms and applying large-scale datasets.
Despite the powerful feature learning capabilities of deep learning, problems still exist when applied to FER. Since deep neural networks require a large amount of training data to avoid overfitting, the facial expression data collected in the laboratory can no longer meet the demand; hence the dataset collection site has gradually changed from the laboratory to the wild. This provides a guarantee for the large amount of data required for deep learning, such as AffectNet (Mollahosseini et al., 2017), RAF-DB (S. Li et al., 2017), ExpW (Zhang et al., 2018). However, during the construction of wild datasets of traditional facial expression categories or scene-specific facial expression categories, low-quality facial images, ambiguous facial expressions, and subjectivity of annotators can lead to uncertainty in expression labels, forming the key challenge of FER in the deep learning era.
In general, uncertain samples can cause problems during training. First, they may lead to overfitting of the model to uncertain samples. Second, they affect the model’s learning of facial features from reliable samples. Third, a high proportion of uncertain samples will affect the convergence of the model early in training. Moreover, with the application of FER technology in various fields, traditional facial expression classification and datasets can no longer meet the needs, and more people are building facial expression datasets on specific occasions and reclassifying expressions. As a result, these datasets may have problems, such as uneven distribution of expressions, poor image quality, and ambiguous definitions of expressions, leading to the more prominent problem of uncertainty in sample labels.
To solve the problem of sample uncertainty, Wang et al. (2020) proposed a Self-Cure Network (SCN) to suppress the uncertainty of large-scale facial expression recognition. SCN consists of three modules: self-attention importance weighting, rank regularization, and noise relabeling. SCN first extracts the facial features through the backbone convolutional neural network (CNN), and the self-attention importance weighting module then calculates the corresponding weights according to the facial features of the samples. In addition, the rank regularization module sorts the sample weights in descending order and divides them into high- and low-importance groups proportionally. It then calculates the rank regularization loss (RR_Loss). Finally, the noise relabeling module changes the labels of the low-importance group samples by calculating whether the difference between the maximum predicted probability and the given label probability is bigger than the threshold.
SCN effectively suppresses the uncertainty of the sample labels but is too cautious in setting the threshold. The fixed threshold reduces the number of triggering the noise relabeling module, making the network less robust to noise. This paper proposes a simple and effective optimization algorithm (i.e., correction strategy) based on SCN to improve the robustness of the network to label noise. First, the correction weights of the samples are calculated based on the sample weights, and the correction weights are combined with the prediction results of the sample to obtain the correction strategy loss. Second, the correction weights are combined with the threshold of the noise relabeling module to generate new dynamic thresholds, so each sample has a corresponding threshold.
The rest of the paper is structured as follows. First, we summarize the current research status at home and abroad on FER, student classroom facial expression recognition, and uncertainties in FER. We then introduce the correction strategy and its components. We then present the experimental verification and analysis and summarize these results.