Journal of Guangdong University of Technology ›› 2024, Vol. 41 ›› Issue (03): 62-70,109.doi: 10.12052/gdutxb.230052

• Computer Science and Technology • Previous Articles     Next Articles

Face Recognition Method in Complex Environment Based on Infrared Visible Fusion

Feng Guang1, Bao Long2   

  1. 1. School of Automation, Guangdong University of Technology, Guangzhou 510006, China;
    2. School of Computer Science and Technology, Guangdong University of Technology, Guangzhou 510006, China
  • Received:2023-03-20 Online:2024-05-25 Published:2024-05-25

HTML

PDF (PC)

76

Abstract: With the development of deep learning methods, the accuracy and speed of face recognition based on visible light in ideal environments have reached an excellent level. However, in complex environments such as low light, the lack of a light source keeps visible images from reflecting face details, resulting in reduced or even invalid face recognition. Aiming at the problems in this issue, a face recognition method in complex environments based on infrared-visible light fusion is proposed. Firstly, an infrared and visible fusion recognition network combining CNN and Transformer is introduced, specifically designed for low illumination environments. This network combines CNN and visual Transformer in parallel to form a single-mode feature fusion module, which is utilized to effectively utilize local details and global context information from the source image. Additionally, a multimodal feature fusion strategy based on the average difference of modes is proposed to enhance the distinctive expression of different regional features in the source image. Secondly, a lightweight face recognition network MobileFaceNet-Coo and an adaptive recognition strategy based on edge-cloud collaboration are proposed in order to solve the problem of large and slow fusion recognition network models in practical applications. This strategy selects the recognition model through image quality and effectively utilizes hardware resources. Experimental results demonstrate that the recognition rate of fused infrared light is 13.96 percentage point higher than that of visible light alone. Real-world project result shows that this method significantly improves real-time and accuracy of face recognition in complex environments.

Key words: face recognition, image fusion, low illumination, Transformer

CLC Number: 

  • TP183
[1] SINGH R, AHMED T, SINGH R, et al. Identifying tiny faces in thermal images using transfer learning [J]. Journal of Ambient Intelligence and Humanized Computing, 2020, 11(5): 1957-1966.
[2] 马娜. 基于高光谱图像的人脸识别算法和实验研究[D]. 济南: 山东师范大学, 2017.
[3] 郭婷, 张天序, 郭诗嘉. 一种红外图像和宽光谱融合的人脸识别算法[J]. 武汉工程大学学报, 2022, 44(3): 320-324.
GUO T, ZHAG T X, GUO S J. A face recognition algorithm based on infrared image and wide spectrum fusion [J]. Journal of Wuhan Engineering University, 2022, 44(3): 320-324.
[4] 谢志华, 李毅, 牛杰一. 联合分块谱带优选和深度特征的高光谱人脸识别[J]. 中国图象图形学报, 2021, 26(12): 2870-2878.
XIE Z H, LI Y, NIU J Y. Hyperspectral face recognition combining block spectral band optimization and depth feature [J]. Chinese Journal of Image Graphics, 2021, 26(12): 2870-2878.
[5] LI H, WU X J. DenseFuse: a fusion approach to infrared and visible images [J]. IEEE Transactions on Image Processing, 2018, 28(5): 2614-2623.
[6] MA J, YU W, LIANG P, et al. FusionGAN: a generative adversarial network for infrared and visible image fusion [J]. Information Fusion, 2019, 48: 11-26.
[7] PRABHAKAR K R, SRIKAR V S, BABU R V. Deepfuse: a deep unsupervised approach for exposure fusion with extreme exposure image pairs[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2017: 4714-4722.
[8] KIM M, JAIN A K, LIU X. AdaFace: quality adaptive margin for face recognition[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2022: 18750-18759.
[9] ZHANG K, ZHANG Z, LI Z, et al. Joint face detection and alignment using multitask cascaded convolutional networks [J]. IEEE Signal Processing Letters, 2016, 23(10): 1499-1503.
[10] HOU Q, ZHOU D, FENG J. Coordinate attention for efficient mobile network design[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2021: 13713-13722.
[11] CHEN S, LIU Y, GAO X, et al. MobileFaceNets: efficient CNNs for accurate real-time face verification on mobile devices [EB/OL]. arXiv: 1804.07573(2019-03-21) [2023-03-01].https://arxiv.org/ftp/arxiv/papers/1804/1804.07573.pdf.
[12] MA N, ZHANG X, ZHENG H T, et al. ShuffleNet V2: practical guidelines for efficient CNN architecture design[EB/OL]. arXiv: 1807.11164(2018-07-30) [2023-03-10].https://arxiv.org/abs/1807.11164.
[13] HUANG G, LIU Z, VAN Der M L , et al. 2017. Densely connected convolutional networks[J]. IEEE Computer Society, 2017: 2261-2269.
[14] ZAMIR S W, ARORA A, KHAN S , et al. Restormer: efficient transformer for high-resolution image restoration[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway: IEEE Press, 2022: 5718-5729.
[15] DAI C . Attentional feature fusion[C]// Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Piscataway: IEEE Press, 2021: 3560-3569.
[16] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]// European Conference on Computer Vision(ECCV) . Munich: Springer, 2018: 3-19.
[17] YI D, LEI Z, LIAO S, et al. Learning face representation from scratch [J]. Computer Science, 2014, 29(1): 51-59.
[18] STAN L, DONG Y, ZHEN L, et al. The casia nir-vis 2.0 face database[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway: IEEE Press, 2013: 348-353.
[19] HUANG G B, RAMESH M, BERG T, et al. Labeled Faces in the Wild: a database for studying face recognition in unconstrained environments[EB/OL]. (2008-10-01) [2023-03-10].https://people.cs.umass.edu/~elm/papers/Huang_eccv2008-lfw.pdf
[20] ZHENG T, DENG W, HU J. Cross-pose lfw: a database for studying corsspose face recognition in unconstrained environments[EB/OL]. arXiv: 1708.08197(2017-08-28) [2023-03-10]. https://arxiv.org/abs/1708.08197v1.
[21] MOSCHOGLOU S, PAPAIOANNOU A, SAGONAS C, et al. Agedb: the first manually collected, in-the-wild age database[C]//Proceedings of the IEEE Conference on Compute Vision and Pattern Recognition Workshops. [S.l.]: IEEE, 2017: 51-59.
[22] TANG L, YUAN J, MA J. Image fusion in the loop of high-level vision tasks: a semantic-aware real-time infrared and visible image fusion network [J]. Information Fusion, 2022, 82: 28-42.
[23] MA J, ZHANG H, SHAO Z, et al. GANMcC: a generative adversarial network with multiclassification constraints for infrared and visible image fusion[J]. IEEE Transactions on Instrumentation and Measurement, 2020, 99: 1.
[24] PANETTA, KAREN, QIANWEN W, et al. A comprehensive database for benchmarking imaging systems[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(3): 509-520.
[1] Guo Ao, Xu Bo-yan, Cai Rui-chu, Hao Zhi-feng. Temporal Alignment Style Control in Text-to-Speech Synthesis Algorithm [J]. Journal of Guangdong University of Technology, 2024, 41(02): 84-92.
[2] Lai Zhi-mao, Zhang Yun, Li Dong. A Survey of Deepfake Detection Techniques Based on Transformer [J]. Journal of Guangdong University of Technology, 2023, 40(06): 155-167.
[3] Zhang Miao, Pang Zhuo-biao, Hao Xue-dong, Xie Si-wei, Zhang Xing-wang. A Research on a Transformerless Parallel Hybrid Active Power Filter [J]. Journal of Guangdong University of Technology, 2019, 36(05): 33-37.
[4] Dong Wen-hua, Li Chun-lai, Lan Xiong. Design and Experimental Analysis of an Open-close Micro Current Transformer [J]. Journal of Guangdong University of Technology, 2019, 36(04): 65-69.
[5] Zeng Bi, Ren Wan-ling, Chen Yun-hua. An Unpaired Face Illumination Normalization Method Based on CycleGAN [J]. Journal of Guangdong University of Technology, 2018, 35(05): 11-19.
[6] He Rui-wen， Xie Qiong-xiang， Cai Ze-xiang. Influence of Digital Acquisition of the Electrical Information on the Reliability of Relay Protection [J]. Journal of Guangdong University of Technology, 2013, 30(2): 68-73.
[7] Chen He-en, , Feng Kai-ping, Pan Li-pei, Wu Yue-ming, . Study of Architecture Transformation [J]. Journal of Guangdong University of Technology, 2012, 29(2): 94-96.
[8] WU Jian-Hong, LI Wei-Tong, LIAO Jie. An Improved Image Fusion Algorithm Based on Wavelet-Based Contourlet Transform [J]. Journal of Guangdong University of Technology, 2011, 28(1): 12-15.
[9] WANG Ting-Ting, SU Cheng-Yue, WANG Xian, ZHOU Zhen. High Precision M easurement Based on e-PSF of Multi-Image [J]. Journal of Guangdong University of Technology, 2010, 27(3): 51-55.
[10] CAI Zheng, WANG Fu-Long, XU Ai-Hui. A New Image Distance Applied in Face Recognition [J]. Journal of Guangdong University of Technology, 2010, 27(3): 64-67.
[11] Huang Cheng,Wang Fu-long,Liu Xiao-liang . Face Recognition Based on Kernel Fisher Discriminant Analysis under Exponential Decay [J]. Journal of Guangdong University of Technology, 2009, 26(4): 62-64.
[12] Liu Xiao-liang,Wang Fu-long,Huang Cheng,Zeng Ai-hua. The Application of a Weighted Kernel Fisher Discriminant Analysis Applied in Face Recognition [J]. Journal of Guangdong University of Technology, 2009, 26(4): 65-69.
[13] ZHANG Chen,PAN Bao-chang,ZHENG Sheng-lin,LU Yi-tao. Neural Network Face Recognition Based on Length Extension Technique [J]. Journal of Guangdong University of Technology, 2006, 23(3): 108-112.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © 2017 Journal of Guangdong University of Technology
Add：729 East Dongfeng RD., Guangzhou PRC. Postcode: 510090
Tel：020-37626653,020-37626139,020-37626396
Email：xbzrb@gdut.edu.cn
Supported:Beijing Magtech