基于蓝图可分离残差蒸馏网络的图像超分辨率重建

doi:10.12052/gdutxb.230022

摘要/Abstract

摘要： 标准卷积的单图像超分辨率重建性能受限于堆叠网络层的冗余性，算法难以实施，特征提取层单一的残差结构也无法高效地利用卷积得到的特征信息。为改善上述问题，本文改进残差蒸馏结构，提出残差蒸馏复用模块，以减少残差蒸馏过程中图像高频信息的损失；此外，将基础残差块替换为蓝图可分离卷积，解耦特征图的空间相关性，以降低高相关性特征的权重，提高卷积的效率，降低参数量。为验证算法的性能，在Set5等标准数据集中对算法进行验证。实验结果表明，该算法模型的峰值信噪比(Peak Signal-To-Noise Ratio, PSNR)和结构相似度(Structural Similarity, SSIM)相比于基于残差蒸馏网络的轻量级图像超分辨率重建网络分别有0.06~0.25 dB与0.004~0.012的提升。

关键词: 图像超分辨率重建, 残差蒸馏, 蓝图可分离卷积, 特征融合

Abstract: The performance of single image super-resolution reconstruction based on standard convolution is limited by the redundancy of the stacked network layers, making it difficult to implement the algorithm on the ground. Moreover, the single residual structure of the feature extraction layer cannot efficiently utilize the feature information obtained from convolution. To address these, this paper proposes a residual distillation reuse module based on the existing residual distillation-based structure to reduce the high-frequency information of the image lost in the residual distillation process. In addition, the base residual block is replaced by a blueprint separable convolution to decouple the spatial correlation of the feature map, such that the weight of highly correlated features can be reduced. As a result, the efficiency of convolution can be improved and the number of parameters can be reduced. We conduct comparative experiments on standard datasets such as Set5 to evaluate the proposed algorithm. The experimental results show that the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) of the proposed algorithm can be improved by approximately 0.06~0.25 dB and 0.004~0.012, respectively, over the lightweightresidual distillation image super-resolution networks.

Key words: image super-resolution reconstruction, residual distillation, blueprint separable convolution, feature fusion

中图分类号:

TP391

熊荣盛, 王帮海, 杨夏宁. 基于蓝图可分离残差蒸馏网络的图像超分辨率重建[J]. 广东工业大学学报, 2024, 41(02): 65-72.

Xiong Rong-sheng, Wang Bang-hai, Yang Xia-ning. Super-resolution Reconstruction of Images Based on Blueprint Separable Residual Distillation Network[J]. Journal of Guangdong University of Technology, 2024, 41(02): 65-72.

参考文献

[1] 李佳星, 赵勇先, 王京华. 基于深度学习的单幅图像超分辨率重建算法综述[J]. 自动化学报, 2021, 47(10) : 2341-2363.
LI J Q, ZHAO Y X, WANG J H. A review of single image super-resolution reconstruction algorithms based on deep learning[J]. Journal of Automation, 2021, 47(10) : 341-2363.
[2] 李现国, 冯欣欣, 李建雄. 多尺度残差网络的单幅图像超分辨率重建[J]. 计算机工程与应用, 2021, 57(7): 215-221.
LI X G, FENG X X, LI J X. Multi-scale residual networks for single image super-resolution reconstruction [J]. Computer Engineering and Applications, 2021, 57(7): 215-221.
[3] 卢峰, 周琳, 蔡小辉. 面向安防监控场景的低分辨率人脸识别算法研究[J]. 计算机应用研究, 2021, 38(4): 1230-1234.
LU F, ZHOU L, CAI X H. Research on low-resolution face recognition algorithm for security surveillance scenarios [J]. Computer Application Research, 2021, 38(4): 1230-1234.
[4] 谢海平, 谢凯利, 杨海涛. 图像超分辨率方法研究进展[J]. 计算机工程与应用, 2020, 56(19): 34-41.
XIE H P, XIE K L, YANG H T. Research progress of image super-resolution methods [J]. Computer Engineering and Applications, 2020, 56(19): 34-41.
[5] 李强, 汪西原, 何佳玮. 基于生成对抗网络的遥感图像超分辨率重建改进算法[J]. 激光与光电子学进展, 2023, 37(5): 1-14.
LI Q, WANG X Y, HE J W. An improved algorithm for super-resolution reconstruction of remote sensing images based on generative adversarial networks [J]. Advances in Laser and Optoelectronics, 2023, 37(5): 1-14.
[6] 肖春蕾. 基于直方图均衡插值的盆底超声图像细节增强方法[J]. 自动化与仪器仪表, 2022(6): 261-264.
XIAO C L. A histogram balanced interpolation-based detail enhancement method for pelvic floor ultrasound images [J]. Automation and Instrumentation, 2022(6): 261-264.
[7] 刘嘉敏, 郑超, 张丽梅, 等. 基于图像重构特征融合的高光谱图像分类方法[J]. 中国激光, 2021, 48(9): 203-212.
LLU J M, ZHENG C, ZHANG L M, et al. Hyperspectral image classification method based on image reconstruction feature fusion [J]. China Laser, 2021, 48(9): 203-212.
[8] WANG Z, CHEN J, HOU S. Deep learning for image super-resolution: a survey [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(10): 3365-3387.
[9] 柳聪, 屈丹, 司念文, 等. 基于深度可分离卷积的轻量级图像超分辨率重建[J]. 计算机工程, 2022, 48(6): 228-234.
LIU C, QU D, SI N W, et al. Lightweight image super-resolution reconstruction based on depth-separable convolution [J]. Computer Engineering, 2022, 48(6): 228-234.
[10] 胡长胜, 詹曙, 吴从中. 基于深度特征学习的图像超分辨率重建[J]. 自动化学报, 2017, 43(5): 814-821.
HU C S, ZHAN S, WU C Z. Deep feature learning based image super-resolution reconstruction [J]. Journal of Automation, 2017, 43(5): 814-821.
[11] 吴靖, 叶晓晶, 黄峰, 等. 基于深度学习的单帧图像超分辨率重建综述[J]. 电子学报, 2022, 50(9): 2265-2294.
WU J, YE X J, HUANG F, et al. A review of single frame image super-resolution reconstruction based on deep learning [J]. Journal of Electronics, 2022, 50(9): 2265-2294.
[12] 李轩, 刘立柱. 多分支残差特征蒸馏网络的图像超分辨重建[J]. 小型微型计算机系统, 2023, 48(3): 1-9.
LI X, LIU L Z. Multi-branch residual feature distillation network for image super-resolution reconstruction [J]. Small Microcomputer Systems, 2023, 48(3): 1-9.
[13] CHAO D, CHEN C L, HE K, et al. Learning a deep convolutional network for image super-resolution[C]//ECCV Springer International Publishing. Istanbul: Springer Cham, 2014: 184-199.
[14] KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//IEEE Conference on Computer Vision & Pattern Recognition(CVPR) . Las Vegas: IEEE, 2016: 1646-1654.
[15] LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super- resolution[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW) . Honolulu: IEEE, 2017: 1132-1140.
[16] KIM J, LEE J K, LEE K M. Deeply-recursive convolutional network for image super- resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR) . Las Vegas: IEEE, 2016: 1637-1645.
[17] YING T, JIAN Y, LIU X. Image super-resolution via deep recursive residual network[C]//IEEE Conference on Computer Vision & Pattern Recognition(CVPR) . Honolulu: IEEE, 2017: 2790-2798.
[18] TAI Y, YANG J, LIU X, et al. MemNet: a persistent memory network for image restoration[C]//IEEE International Conference on Computer Vision. Venice: IEEE Computer Society, 2017: 4549-4557.
[19] AHN N, KANG B, SOHN K A. Fast, accurate, and lightweight super-resolution with cascading residual network[C]//Computer Vision ECCV 2018 15th European Conference. Martial: ECCV, 2018: 256-272.
[20] ZHENG H, WANG X, GAO X . Fast and accurate single image super-resolution via information distillation network[C]//Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 723-731.
[21] HUI Z, GAO X , YANG Y , et al. Lightweight image super-resolution with information multi-distillation network[C]//Proceedings of the 27th ACM International Conference on Multimedia. France: ACM, 2019: 2024-2032.
[22] LIU J, TANG J, WU G. Residual feature distillation network for lightweight image super-resolution[C]//2020 ECCV Springer International Publishing. UK: Springer Cham, 2020: 41-55.
[23] HOWARD A G, ZHU M, CHEN B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[C]// Conference on Computer Vision and Pattern Recognition(CVPR) . Honolulu: IEEE, 2017: 112-121.
[24] HAASE D, AMTHOR M. Rethinking depthwise separable convolutions: how intra-kernel correlations lead to Improved mobileNets[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Seattle: IEEE, 2020: 14588-14597 .
[25] GUO J, LI Y, LIN W , et al. Network decoupling: fom regular to depthwise separable convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR) . Salt Lake City: IEEE, 2018: 1879-1901.
[26] ZHANG Y, LI K, LI K , et al. Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the European Conference on Computer Visionand Pattern Recognition(CVPR) . Munich: Springer, Cham, 2018: 286-301.
[27] WANG Q , WU B , ZHU P , et al. ECA-net: efficient channel attention for deep convolutional neural networks[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Seattle: IEEE, 2020: 11531-11539.
[28] SUN L, PAN J, TANG J. ShuffleMixer: an efficient convNet for image super-resolution[C]//IEEE International Conference on Computer Vision. Hawaii: IEEE Computer Society, 2022: 321-329.
[29] KONG F , LI M , LIU S , et al. Residual local feature network for efficient super-resolution[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . New Orleans: IEEE, 2022: 765-775.
[30] LI Z, LIU Y, CHEN X, et al. Blueprint separable residual network for efficient image super-resolution[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . New Orleans: IEEE, 2022: 832-842.
[31] ZHANG Y, TIAN Y, KONG Y, et al. Residual dense network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Munich: IEEE, 2018: 2472-2481.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed