Abstract:Over the past several decades, micro-expression recognition (MER) has become a growing concern for scientific community. As the filming conditions vary from database to database, previous single-domain MER methods generally exhibit severe performance drop when applied to another database. To deal with this pressing problem, in this paper, a sample-aware and feature refinement network (SFR-Net) is proposed, which combines domain adap-tation with deep metric learning to extract intrinsic features of micro-expressions for accurate recognition. With the help of decoders, siamese networks increasingly refine shared features relevant to emotions while exclusive features irrelevant to emotions are gradually obtained by private networks. In order to achieve promising performance, we further design sample-aware loss to constrain the feature distribution in the high-dimensional feature space. Ex-perimental results show the proposed algorithm can effectively mitigate the diversity among different mi-cro-expression databases, and achieve better generalization performance compared with state-of-the-art methods.