基于红外可见光融合的复杂环境下人脸识别方法

doi:10.12052/gdutxb.230052

摘要/Abstract

摘要： 随着深度学习方法的发展，理想环境下基于可见光的人脸识别精度和速度已经达到优秀的水平。但是在弱光等复杂环境下，由于缺少光源，可见光图像无法体现人脸细节，导致人脸识别效果下降甚至失效。为了解决这一问题，提出一种基于红外可见光融合的复杂环境下人脸识别方法。首先，针对低照度环境提出联合CNN(Convolutional Neural Network) 和Transformer的红外与可见光融合识别网络，并联CNN和视觉Transformer组成单模态特征融合模块，充分利用源图像的局部细节信息和全局上下文信息。同时，提出一种基于模态平均差异度的多模态特征融合策略，强化对源图像不同区域特征的差异化表达。其次，针对实际应用中融合识别网络模型大、速度慢的问题提出轻量化人脸识别网络MobileFaceNet-Coo和基于边云协同的自适应识别策略，通过图像质量选择识别模型，有效利用硬件资源。实验结果表明，弱光条件下，融合红外光与仅使用可见光图像相比，识别率提升了13.96个百分点。同时，将本方法应用实际项目中，结果表明：本方法在复杂环境下，能提高人脸识别的实时性和准确率。

关键词: 人脸识别, 图像融合, 低照度, Transformer

Abstract: With the development of deep learning methods, the accuracy and speed of face recognition based on visible light in ideal environments have reached an excellent level. However, in complex environments such as low light, the lack of a light source keeps visible images from reflecting face details, resulting in reduced or even invalid face recognition. Aiming at the problems in this issue, a face recognition method in complex environments based on infrared-visible light fusion is proposed. Firstly, an infrared and visible fusion recognition network combining CNN and Transformer is introduced, specifically designed for low illumination environments. This network combines CNN and visual Transformer in parallel to form a single-mode feature fusion module, which is utilized to effectively utilize local details and global context information from the source image. Additionally, a multimodal feature fusion strategy based on the average difference of modes is proposed to enhance the distinctive expression of different regional features in the source image. Secondly, a lightweight face recognition network MobileFaceNet-Coo and an adaptive recognition strategy based on edge-cloud collaboration are proposed in order to solve the problem of large and slow fusion recognition network models in practical applications. This strategy selects the recognition model through image quality and effectively utilizes hardware resources. Experimental results demonstrate that the recognition rate of fused infrared light is 13.96 percentage point higher than that of visible light alone. Real-world project result shows that this method significantly improves real-time and accuracy of face recognition in complex environments.

Key words: face recognition, image fusion, low illumination, Transformer

中图分类号:

TP183

冯广, 鲍龙. 基于红外可见光融合的复杂环境下人脸识别方法[J]. 广东工业大学学报, 2024, 41(03): 62-70,109.doi: 10.12052/gdutxb.230052

Feng Guang, Bao Long. Face Recognition Method in Complex Environment Based on Infrared Visible Fusion[J]. Journal of Guangdong University of Technology, 2024, 41(03): 62-70,109.doi: 10.12052/gdutxb.230052

参考文献

[1] SINGH R, AHMED T, SINGH R, et al. Identifying tiny faces in thermal images using transfer learning [J]. Journal of Ambient Intelligence and Humanized Computing, 2020, 11(5): 1957-1966.
[2] 马娜. 基于高光谱图像的人脸识别算法和实验研究[D]. 济南: 山东师范大学, 2017.
[3] 郭婷, 张天序, 郭诗嘉. 一种红外图像和宽光谱融合的人脸识别算法[J]. 武汉工程大学学报, 2022, 44(3): 320-324.
GUO T, ZHAG T X, GUO S J. A face recognition algorithm based on infrared image and wide spectrum fusion [J]. Journal of Wuhan Engineering University, 2022, 44(3): 320-324.
[4] 谢志华, 李毅, 牛杰一. 联合分块谱带优选和深度特征的高光谱人脸识别[J]. 中国图象图形学报, 2021, 26(12): 2870-2878.
XIE Z H, LI Y, NIU J Y. Hyperspectral face recognition combining block spectral band optimization and depth feature [J]. Chinese Journal of Image Graphics, 2021, 26(12): 2870-2878.
[5] LI H, WU X J. DenseFuse: a fusion approach to infrared and visible images [J]. IEEE Transactions on Image Processing, 2018, 28(5): 2614-2623.
[6] MA J, YU W, LIANG P, et al. FusionGAN: a generative adversarial network for infrared and visible image fusion [J]. Information Fusion, 2019, 48: 11-26.
[7] PRABHAKAR K R, SRIKAR V S, BABU R V. Deepfuse: a deep unsupervised approach for exposure fusion with extreme exposure image pairs[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2017: 4714-4722.
[8] KIM M, JAIN A K, LIU X. AdaFace: quality adaptive margin for face recognition[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2022: 18750-18759.
[9] ZHANG K, ZHANG Z, LI Z, et al. Joint face detection and alignment using multitask cascaded convolutional networks [J]. IEEE Signal Processing Letters, 2016, 23(10): 1499-1503.
[10] HOU Q, ZHOU D, FENG J. Coordinate attention for efficient mobile network design[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2021: 13713-13722.
[11] CHEN S, LIU Y, GAO X, et al. MobileFaceNets: efficient CNNs for accurate real-time face verification on mobile devices [EB/OL]. arXiv: 1804.07573(2019-03-21) [2023-03-01].https://arxiv.org/ftp/arxiv/papers/1804/1804.07573.pdf.
[12] MA N, ZHANG X, ZHENG H T, et al. ShuffleNet V2: practical guidelines for efficient CNN architecture design[EB/OL]. arXiv: 1807.11164(2018-07-30) [2023-03-10].https://arxiv.org/abs/1807.11164.
[13] HUANG G, LIU Z, VAN Der M L , et al. 2017. Densely connected convolutional networks[J]. IEEE Computer Society, 2017: 2261-2269.
[14] ZAMIR S W, ARORA A, KHAN S , et al. Restormer: efficient transformer for high-resolution image restoration[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway: IEEE Press, 2022: 5718-5729.
[15] DAI C . Attentional feature fusion[C]// Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Piscataway: IEEE Press, 2021: 3560-3569.
[16] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]// European Conference on Computer Vision(ECCV) . Munich: Springer, 2018: 3-19.
[17] YI D, LEI Z, LIAO S, et al. Learning face representation from scratch [J]. Computer Science, 2014, 29(1): 51-59.
[18] STAN L, DONG Y, ZHEN L, et al. The casia nir-vis 2.0 face database[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway: IEEE Press, 2013: 348-353.
[19] HUANG G B, RAMESH M, BERG T, et al. Labeled Faces in the Wild: a database for studying face recognition in unconstrained environments[EB/OL]. (2008-10-01) [2023-03-10].https://people.cs.umass.edu/~elm/papers/Huang_eccv2008-lfw.pdf
[20] ZHENG T, DENG W, HU J. Cross-pose lfw: a database for studying corsspose face recognition in unconstrained environments[EB/OL]. arXiv: 1708.08197(2017-08-28) [2023-03-10]. https://arxiv.org/abs/1708.08197v1.
[21] MOSCHOGLOU S, PAPAIOANNOU A, SAGONAS C, et al. Agedb: the first manually collected, in-the-wild age database[C]//Proceedings of the IEEE Conference on Compute Vision and Pattern Recognition Workshops. [S.l.]: IEEE, 2017: 51-59.
[22] TANG L, YUAN J, MA J. Image fusion in the loop of high-level vision tasks: a semantic-aware real-time infrared and visible image fusion network [J]. Information Fusion, 2022, 82: 28-42.
[23] MA J, ZHANG H, SHAO Z, et al. GANMcC: a generative adversarial network with multiclassification constraints for infrared and visible image fusion[J]. IEEE Transactions on Instrumentation and Measurement, 2020, 99: 1.
[24] PANETTA, KAREN, QIANWEN W, et al. A comprehensive database for benchmarking imaging systems[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(3): 509-520.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed