广东工业大学学报

• •    

基于掩蔽卷积及外部注意力的视频异常行为检测算法

邝永年1, 王丰1, 丁克2   

  1. 1. 广东工业大学 信息工程学院, 广东 广州 510006;
    2. 佛山显扬科技有限公司, 广东 佛山 528200
  • 收稿日期:2023-12-14 出版日期:2024-09-27 发布日期:2024-09-27
  • 通信作者: 王丰(1987–)，男，副教授，博士，主要研究方向为通信信号处理、移动边缘计算与机器视觉，E-mail：fengwang13@gdut.edu.cn
  • 作者简介:邝永年(1995–)，男，硕士研究生，主要研究方向为视频异常检测，E-mail:365840026@qq.com
  • 基金资助:
    国家自然科学基金资助项目(61901124) ；广东省自然科学基金资助项目(2021A1515012305) ；广东省研究生创新教育创新计划项目(2023JGXM_048)

Video Frame Anomaly Behavior Detection Method Based on Masked Convolution and External Attention

Kuang Yong-nian1, Wang Feng1, Ding Ke2   

  1. 1. School of Information Engineering, Guangdong University of Technology, Guangzhou 510006, China;
    2. Foshan Hinyeung Limited, Foshan 528200, China
  • Received:2023-12-14 Online:2024-09-27 Published:2024-09-27

HTML

PDF (PC)

81

摘要： 针对视频异常行为检测难题，本文提出一种基于掩蔽卷积和外部注意力机制的卷积神经网络检测算法模型。一方面，通过掩蔽卷积限制有效卷积区域使神经网络高效学习，以此生成对正常行为特征有效的正态化建模。另一方面，通过结合轻量化的外部注意力机制，提高对感兴趣区域的建模质量。针对重构或预测架构的卷积神经网络，添加特征重构损失函数，提高视频异常行为检测的检测正确率。实验结果表明，在Avenue数据集、UCSD-Ped1数据集以及UCSD-Ped2数据集上，本文的检测方法分别提高了2.86%、2.54%以及0.56%的视频异常行为检测正确率。

关键词: 视频异常行为检测, 掩蔽卷积, 外部注意力, 自监督学习

Abstract: In order to solve the challenges of video anomaly behavior detection, a new detection algorithm model is proposed based on masked convolution and external attention mechanism convolutional neural network. On the one hand, by masked convolution, it can restrict the effective regions of convolution to make the neural network efficiently learn, so as to model the positive moderate that is effective in the feathers of normal behavior. On the other hand, by combining with lightweight external attention mechanisms, the modeling quality of interested regions is improved. Convolutional neural network in reconstruction or predictive architectures is added in feature reconstruction loss to improve the detection accuracy of video abnormal behavior detection. The experimental results show that the proposed method can effectively improve the detection performance of video anomaly behaviors by 2.86%, 2.54% and 0.56% on the Avenue dataset, UCSD-Ped1 dataset and UCSD-Ped2 dataset, respectively.

Key words: video anomaly behavior detection, masked convolution, external attention, self-supervised learning

中图分类号: 

  • TP391
[1] LAVEE G, RIVLIN E, RUDZSKY M. Understanding video events: a survey of methods for automatic interpretation of semantic occurrences in video [J]. IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews) , 2009, 39(5): 489-504.
[2] LIU W, LUO W, LIAN D, et al. Future frame prediction for anomaly detection—a new baseline[C]//2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . New York: IEEE Press, 2018: 6536-6545.
[3] MAHADEVAN V, LI W X, BHALODIA V, et al. Anomaly detection in crowded scenes[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) . New York: IEEE Press, 2010: 1975-1981.
[4] KIM J, GRAUMAN K. Observe locally, infer globally: a space-time MRF for detecting abnormal activities with incremental updates[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . New York: IEEE Press, 2009: 2921-2928.
[5] HASAN M, CHOI J, NEUMANN J, et al. Learning temporal regularity in video sequences[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . New York: IEEE Press, 2016: 733-742.
[6] GONG D, LIU L, LE V, et al. Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV) . New York: IEEE Press, 2019: 1705-1714.
[7] LUO W, LIU W, GAO S. Remembering history with convolutional LSTM for anomaly detection[C]//2017 IEEE International Conference on Multimedia and Expo (ICME) . New York: IEEE Press, 2017: 439-444.
[8] NGUYEN T N, MEUNIER J. Anomaly detection in video sequence with appearance-motion correspondence[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV) . New York: IEEE Press, 2019: 1273-1283.
[9] ZHAO Y, DENG B, SHEN C, et al. Spatio-temporal autoencoder for video anomaly detection[C]//Proceedings of the 25th ACM International Conference on Multimedia. New York: Association for Computing Machinery, 2017: 1933-1941.
[10] DEEPAK K, CHANDRAKALA S, MOHAN C K. Residual spatiotemporal autoencoder for unsupervised video anomaly detection [J]. Signal, Image and Video Processing, 2021, 15(1): 215-222.
[11] 李自强, 王正勇, 陈洪刚, 等. 基于外观和动作特征双预测模型的视频异常行为检测[J]. 计算机应用, 2021, 41(10): 2997-3003.
LI Z Q, WANG Z Y, CHEN H G, et al. Video abnormal behavior detection based on dual prediction model of appearance and motion features [J]. Journal of Computer Applications, 2021, 41(10): 2997-3003.
[12] WU C, SHAO S, TUNC C, et al. An explainable and efficient deep learning framework for video anomaly detection [J]. Cluster Computing, 2022, 25(4): 2715-2737.
[13] LIU C, FU R, LI Y, et al. A self-attention augmented graph convolutional clustering networks for skeleton-based video anomaly behavior detection [J]. Applied Sciences, 2021, 12(1): 4.
[14] LU Y, KUMAR K M, SHAHABEDDIN NABAVI S, et al. Future frame prediction using convolutional VRNN for anomaly detection[C]//2019 16Th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) . New York: IEEE Press, 2019: 1-8.
[15] ZHANG Y, NIE X, HE R, et al. Normality learning in multispace for video anomaly detection [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 31(9): 3694-3706.
[16] 邝永年, 王丰. 基于前景区域生成对抗网络的视频异常行为检测研究[J]. 广东工业大学学报, 2024, 41(1): 63-68,92.
KUANG Y N, WANG F. Video frame anomaly behavior detection based on foreground area generative adversarial networks [J]. Journal of Guangdong University of Technology, 2024, 41(1): 63-68,92.
[17] GAO P, MA T, LI H, et al. Convmae: masked convolution meets masked autoencoders[EB/OL]. arXiv: 2205.03892(2022-05-19) [2024-02-06]. https://doi.org/10.48550/arxiv. 2205.03892v2.
[18] HE K, CHEN X, XIE S, et al. Masked autoencoders are scalable vision learners[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 16000-16009.
[19] GUO M H, LIU Z N, MU T J, et al. Beyond self-attention: External attention using two linear layers for visual tasks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(5): 5436-5447.
[20] LU C W, SHI J P, JIA J Y. Abnormal event detection at 150 FPS in MATLAB[C]//2013 IEEE International Conference on Computer Vision. New York: IEEE Press, 2013: 2720-2727.
[21] LIU Z, NIE Y, LONG C, et al. A hybrid video anomaly detection framework via memory-augmented flow reconstruction and flow-guided frame prediction[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 13588-13597.
[22] LUO W, LIU W, LIAN D, et al. Video anomaly detection with sparse coding inspired deep neural networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(3): 1070-1084.
[23] LI N, CHANG F, LIU C. Spatial-temporal cascade autoencoder for video anomaly detection in crowded scenes [J]. IEEE Transactions on Multimedia, 2021, 23: 203-215.
[24] CHANG Y, TU Z, XIE W, et al. Clustering driven deep autoencoder for video anomaly detection[C]//Computer Vision–ECCV 2020: 16th European Conference. Glasgow: Springer International Publishing, 2020: 329-345.
[25] CHANG Y, TU Z, XIE W, et al. Video anomaly detection with spatio-temporal dissociation [J]. Pattern Recognition, 2022, 122: 108213.
[26] DOSHI K, YILMAZ Y. Online anomaly detection in surveillance videos with asymptotic bound on false alarm rate [J]. Pattern Recognition, 2021, 114: 107865.
[27] HAO Y, LI J, WANG N, et al. Spatiotemporal consistency-enhanced network for video anomaly detection [J]. Pattern Recognition, 2022, 121: 108232.
[1] 邝永年, 王丰. 基于前景区域生成对抗网络的视频异常行为检测研究[J]. 广东工业大学学报, 2024, 41(01): 63-68,92.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 2017 《广东工业大学学报》编辑部
地址：广州市东风东路729号广东工业大学 邮编：510090
电话：020-37626653,020-37626139,020-37626396
邮箱：xbzrb@gdut.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发