广东工业大学学报 ›› 2023, Vol. 40 ›› Issue (03): 10-16.doi: 10.12052/gdutxb.210179

• • 上一篇    下一篇

基于多特征融合的表情识别算法

赖东升1, 冯开平2, 罗立宏2   

  1. 1. 广东工业大学 计算机学院, 广东 广州 510006;
    2. 广东工业大学 艺术与设计学院, 广东 广州 510090
  • 收稿日期:2021-11-17 出版日期:2023-05-25 发布日期:2023-06-08
  • 通信作者: 冯开平(1963-),男,教授,硕士生导师,主要研究方向为计算机视觉、虚拟现实,E-mail:fengkp@gdut.edu.cn
  • 作者简介:赖东升(1998-),男,硕士研究生,主要研究方向为图像处理
  • 基金资助:
    教育部人文社科资助项目(20YJAZH073)

Facial Expression Recognition Based on Multi-feature Fusion

Lai Dong-sheng1, Feng Kai-ping2, Luo Li-hong2   

  1. 1. School of Computer Science and Technology, Guangdong University of Technology, Guangzhou 510006, China;
    2. School of Art and Design, Guangdong University of Technology, Guangzhou 510090, China
  • Received:2021-11-17 Online:2023-05-25 Published:2023-06-08

HTML

PDF (PC)

1162

摘要： 针对轻量级面部表情识别算法泛化能力的不足，提出了一种结合多特征融合和注意力机制的表情识别方法。使用局部二值模式(Local Binary Pattern, LBP)算子减少面部图像中无关信息的干扰，双分支神经网络提取原始人脸图像和LBP图像的特征，融合两个网络提取的中高层特征，并通过注意力机制加强重要特征，在保持较少参数量的同时生成大量的有效特征信息提高算法的识别效果。实验结果表明，该方法在Fer2013和CK+数据集上的识别率分别为70.21%和95.59%，有效地提高了轻量级表情识别算法的性能。

关键词: 人脸表情识别, 轻量级网络, 局部二值模式, 多特征融合, 注意力机制

Abstract: For the insufficient generalization ability of lightweight facial expression recognition algorithms, a facial expression recognition method based on multi-feature fusion and attention mechanism is proposed. LBP is used to reduce the interference of meaningless information of original face image, the dual-branch neural network extracts the features of original face image and LBP image respectively, merges the middle-level and high-level feature information extracted by these two networks and strengthens important features through attention mechanism for facial expression recognition, which can generate a large amount of discriminative features by multi-feature fusion to improve the recognition performance with a small number of parameters. The experimental results show that the recognition rate of this method has reached 70.21% on Fer2013 dataset and 95.59% on CK+ dataset respectively, which effectively improves the performance of lightweight expression recognition algorithm.

Key words: facial expression recognition, lightweight network, local binary pattern, multi-feature fusion, attention mechanism

中图分类号: 

  • TP391
[1] SHI D, TANG H. Face recognition algorithm based on self-adaptive blocking local binary pattern [J]. Multimedia Tools and Applications, 2021, 80: 23899-23921.
[2] YAO A, CAI D, HU P, et al. HoloNet: towards robust emotion recognition in the wild[C]// Proceedings of the 18th ACM International Conference on Multimodal Interaction. Tokyo: Association for Computing Machinary, 2016: 472-478.
[3] DING H, ZHOU P, CHELLAPPA R. Occlusion-adaptive deep network for robust facial expression recognition[C]// IEEE International Joint Conference on Biometrics. Houston: IEEE, 2020: 1-9.
[4] FAN Y, LAM J, LI V. Multi-ensemble convolutional neural network for facial expression recognition[C]// International Conference on Artificial Neural Networks. Rhodes: Springer International Publishing, 2018: 84-94.
[5] WANG K, PENG X, YANG J, et al. Region attention networks for pose and occlusion robust facial expression recognition [J]. IEEE Transactions on Image Processing, 2020, 29: 4057-4069.
[6] WANG K, CHEN J, ZHANG X, et al. Facial expression recognition based on deep convolutional neural network[C]// IEEE 8th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems. Tianjin: IEEE, 2018: 629-634.
[7] MA Y, WANG X, WEI L. Multi-level spatial and semantic enhancement network for expression recognition [J]. Applied Intelligence, 2021, 51: 1-14.
[8] LI J, JIN K, ZHOU D, et al. Attention mechanism-based CNN for facial expression recognition [J]. Neurocomputing, 2020, 411: 340-350.
[9] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141.
[10] CHEN S, LIU Y, GAO X, et al. Mobile Face Nets: efficient cnns for accurate real-time face verification on mobile devices[C]// Biometric Recognition. Urumqi: Springer International Publishing, 2018: 428-438.
[11] 徐琳琳, 张树美, 赵俊莉. 构建并行卷积神经网络的表情识别算法[J]. 中国图像图形学报, 2019, 24(2): 227-236.
XU L L, ZHANG S M, ZHAO J L. Expression recognition algorithm for parallel convolutional neural networks [J]. Journal of Image and Graphics, 2019, 24(2): 227-236.
[12] MOLLAHOSSEINI A, CHAN D, MAHOOR M H. Going deeper in facial expression recognition using deep neural networks[C]// 2016 IEEE Winter Conference on Applications of Computer Vision. Lake Placid: IEEE, 2016: 1-10.
[13] JEON J, PARK J C, JO Y J, et al. A real-time facial expression recognizer using deep neural network[C]// Proceedings of the 10th International Conference on Ubiquitous Information Management and Communication. Danang: Association for Computing Machinery, 2016: 1-4.
[14] SANG D V, DAT N V, THUAN D P. Facial expression recognition using deep convolutional neural networks[C]// 9th International Conference on Knowledge and Systems Engineering. Hue: IEEE, 2017: 130-135.
[15] MA N N, ZHANG X Y, ZHENG H T, et al. ShuffleNet V2: practical guidelines for efficient cnn architecture design[C]// Proceedings of the European Conference on Computer Vision. Munich: Springer International Publishing, 2018: 116-131.
[16] 何志超, 赵龙章, 陈闯. 用于人脸表情识别的多分辨率特征融合卷积神经网络[J]. 激光与光电子学进展, 2018, 55(7): 370-375.
HE Z C, ZHAO L Z, CHEN C. Convolution neural network with multi-resolution feature fusion for facial expression recognition [J]. Laser & Optoelectronics Progress, 2018, 55(7): 370-375.
[17] QIAN Z, MU J, ZHANG J, et al. Facial expression recognition based on sas-net attention mechanism[C]// International Conference on Computer Network, Electronic and Automation. Xian: IEEE, 2021: 159-163.
[1] 赖志茂, 章云, 李东. 基于Transformer的人脸深度伪造检测技术综述[J]. 广东工业大学学报, 2023, 40(06): 155-167.
[2] 曾安, 陈旭宙, 姬玉柱, 潘丹, 徐小维. 基于自注意力和三维卷积的心脏多类分割方法[J]. 广东工业大学学报, 2023, 40(06): 168-175.
[3] 吴亚迪, 陈平华. 基于用户长短期偏好和音乐情感注意力的音乐推荐模型[J]. 广东工业大学学报, 2023, 40(04): 37-44.
[4] 曹智雄, 吴晓鸰, 骆晓伟, 凌捷. 融合迁移学习与YOLOv5的安全帽佩戴检测算法[J]. 广东工业大学学报, 2023, 40(04): 67-76.
[5] 吴俊贤, 何元烈. 基于通道注意力的自监督深度估计方法[J]. 广东工业大学学报, 2023, 40(02): 22-29.
[6] 刘洪伟, 林伟振, 温展明, 陈燕君, 易闽琦. 基于MABM的消费者情感倾向识别模型——以电影评论为例[J]. 广东工业大学学报, 2022, 39(06): 1-9.
[7] 滕少华, 董谱, 张巍. 融合语义结构的注意力文本摘要模型[J]. 广东工业大学学报, 2021, 38(03): 1-8.
[8] 梁观术, 曹江中, 戴青云, 黄云飞. 一种基于注意力机制的无监督商标检索方法[J]. 广东工业大学学报, 2020, 37(06): 41-49.
[9] 曾碧卿, 韩旭丽, 王盛玉, 徐如阳, 周武. 基于双注意力卷积神经网络模型的情感分析研究[J]. 广东工业大学学报, 2019, 36(04): 10-17.
[10] 高俊艳, 刘文印, 杨振国. 结合注意力与特征融合的目标跟踪[J]. 广东工业大学学报, 2019, 36(04): 18-23.
[11] 孙伟, 钟映春, 谭志, 连伟烯. 多特征融合的室内场景分类研究[J]. 广东工业大学学报, 2015, 32(1): 75-79.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 2017 《广东工业大学学报》编辑部
地址：广州市东风东路729号广东工业大学 邮编：510090
电话：020-37626653,020-37626139,020-37626396
邮箱：xbzrb@gdut.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发