To solve the problem of low running speed of image rain removal models, a multi scale attention map guided image rain removal network is proposed. Firstly, an original resolution main network with residual structure is constructed for image rain removal. Secondly, in order to enhance the rain removal ability of the main network, especially the recognition ability of multi-scale features, a multi-scale attention map generation network is designed as an auxiliary to the main network. The generated multi-scale attention maps are used to enhance the features of rain stripes in the image, guiding the main network to identify and remove them in turn. In addition, this paper also designed the dual attention guided convolution block, which adds the ability to pay attention to spatial features on the basis of traditional channel attention mechanisms to achieve joint attention to spatial and channel features. Experimental results show that the proposed method maintains the comparability and stability of the restored image quality while significantly reducing the amount of network parameters and effectively improving the speed.

Gu Kunyuan, Pang Xiaoyan, Zhu Xiaoli, Zhang Peng. Multi-scale Attention Map Guided Image Rain Removal Network. Academic Journal of Computing & Information Science (2023), Vol. 6, Issue 4: 14-25. https://doi.org/10.25236/AJCIS.2023.060403.

[1] Sha M, Zeng K, Tao Z, et al. Lightweight Pedestrian Detection Based on Feature Multiplexed Residual Network [J]. Electronics, 2023, 12(4): 918.

[2] Le V T, Kim Y G. Attention-based residual autoencoder for video anomaly detection [J]. Applied Intelligence, 2023, 53(3): 3240-3254.

[3] Ashiq F, Asif M, Ahmad M B, et al. CNN-based object recognition and tracking system to assist visually impaired people [J]. IEEE Access, 2022, 10: 14819-14834.

[4] Santhaseelan V, Asari V K . Utilizing local phase information to remove rain from video[J]. International Journal of Computer Vision, 2015, 112(1):71-89.

[5] Zheng X, Liao Y, Guo W, et al. Single-image-based rain and snow removal using multi-guided filter[C]//Neural Information Processing: 20th International Conference, ICONIP 2013, Daegu, Korea, November 3-7, 2013. Proceedings, Part III 20. Springer Berlin Heidelberg, 2013: 258-265.

[6] Luo Y, Xu Y, Ji H. Removing rain from a single image via discriminative sparse coding [C] // Proceedings of the IEEE international conference on computer vision. 2015: 3397-3405.

[7] Zhang H, Patel V M. Convolutional sparse and low-rank coding-based rain streak removal[C]//2017 IEEE Winter conference on applications of computer vision (WACV). IEEE, 2017: 1259-1267.

[8] Li Y., Tan R. T., Guo X., Lu J., & Brown M. S. Rain streak removal using layer priors. In IEEE conference on computer vision and pattern recognition (pp. 2736–2744).

[9] Li Y., Tan R. T., Guo X., Lu J., & Brown M. S. Single image rain streak decomposition using layer priors. IEEE Transactions on Image Processing, 26(8), 3874–3885.

[10] Fu X, Huang J, Ding X, et al. Clearing the skies: A deep network architecture for single-image rain removal [J]. IEEE Transactions on Image Processing, 2017, 26(6): 2944-2956.

[11] Fu X, Huang J, Zeng D, et al. Removing rain from single images via a deep detail network [C] // Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 3855-3863.

[12] Yang W, Tan R T, Feng J, et al. Deep joint rain detection and removal from a single image[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 1357-1366.

[13] Li X, Wu J, Lin Z, et al. Recurrent squeeze-and-excitation context aggregation net for single image deraining [C]//Proceedings of the European conference on computer vision (ECCV). 2018: 254-269.

[14] Wang T, Yang X, Xu K, et al. Spatial attentive single-image deraining with a high quality real rain dataset [C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 12270-12279.

[15] He D, Shang X, Luo J. Adherent mist and raindrop removal from a single image using attentive convolutional network[J]. Neurocomputing, 2022, 505: 178-187.

[16] Su Z, Zhang Y, Zhang X P, et al. Non-local channel aggregation network for single image rain removal [J]. Neurocomputing, 2022, 469: 261-272.

[17] Liang Y, Anwar S, Liu Y. Drt: A lightweight single image deraining recursive transformer [C] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 589-598.

[18] Ren D, Zuo W, Hu Q, et al. Progressive image deraining networks: A better and simpler baseline [C] //Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 3937-3946.

[19] Jiang K, Wang Z, Yi P, et al. Multi-scale progressive fusion network for single image deraining [C] // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020: 8346-8355.

[20] Jiang R, Li Y, Chen C, et al. Two‐stage learning framework for single image deraining[J]. IET Image Processing, 2022.

[21] Li F, Shen L, Mi Y, et al. DRCNet: Dynamic Image Restoration Contrastive Network[C]//Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XIX. Cham: Springer Nature Switzerland, 2022: 514-532.

[22] Zamir S W, Arora A, Khan S, et al. Multi-stage progressive image restoration[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 14821-14831.

[23] Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions [J]. arXiv preprint arXiv:1511.07122, 2015.

[24] Woo S, Park J, Lee J Y, et al. Cbam: Convolutional block attention module[C]//Proceedings of the European conference on computer vision (ECCV). 2018: 3-19.

[25] Zhang Y, Li K, Li K, et al. Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the European conference on computer vision (ECCV). 2018: 286-301.

[26] Cho S J, Ji S W, Hong J P, et al. Rethinking coarse-to-fine approach in single image deblurring [C] // Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 4641-4650.

[27] Jiang L, Dai B, Wu W, et al. Focal frequency loss for image reconstruction and synthesis [C] // Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 13919-13929.

[28] Tu Z, Talebi H, Zhang H, et al. Maxim: Multi-axis mlp for image processing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 5769-5780.

[29] Kingma D P, Ba J. Adam: A method for stochastic optimization [J]. arXiv preprint arXiv:1412.6980, 2014.

[30] Loshchilov I, Hutter F. Sgdr: Stochastic gradient descent with warm restarts [J]. arXiv preprint arXiv:1608.03983, 2016.

[31] Hore A, Ziou D. Image quality metrics: PSNR vs. SSIM[C]//the 20th International Conference on Pattern Recognition. IEEE, 2010: 2366-2369.

[32] Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.

[33] Yang W, Tan R T, Feng J, et al. Deep joint rain detection and removal from a single image [C] // Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 1357-1366.

[34] Martin D, Fowlkes C, Tal D, et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001. IEEE, 2001, 2: 416-423.

[35] Zhang H, Sindagi V, Patel V M. Image de-raining using a conditional generative adversarial network [J]. IEEE transactions on circuits and systems for video technology, 2019, 30(11): 3943-3956.

[36] Schaefer G, Stich M. UCID: An uncompressed color image database[C]//Storage and Retrieval Methods and Applications for Multimedia 2004. SPIE, 2003, 5307: 472-480.

[37] Arbelaez P, Maire M, Fowlkes C, et al. Contour detection and hierarchical image segmentation [J]. IEEE transactions on pattern analysis and machine intelligence, 2010, 33(5): 898-916.