Academic Journal of Computing & Information Science, 2023, 6(11); doi: 10.25236/AJCIS.2023.061109.
Yanlong Xu, Limin Sun, Qian Guo
School of Information and Intelligent Engineering, University of Sanya, Sanya, 572022, China
In recent years, Generative Adversarial Network (GAN) has been widely used in the field of computer vision due to its superior performance. Influenced by AlphaGAN, we propose U2Net-GAN for image matting algorithm, which is the basis of image synthesis technology. The purpose of matting algorithm is to obtain alpha matte of the foreground in a natural image. In practical applications, natural images may have similar foreground and background and more complex textures, and matting algorithms need to accurately extract a large number of detailed features of images. In order to solve the above problems, we use U2Net as generator of GAN. The generator U2Net cascades multi-layer depth features to accurately extract high-level semantic features of image. Self-Calibrated convolutions (SC) replaces standard convolution of the Residual Ublocks(RSU), which improves performance of the network to extract image detail features without significantly increasing the amount of calculation. And we use PatchGAN discriminator to train with the expanded public datasets and evaluate it on the benchmark dataset. Experiments show that our algorithm achieves the best results in both quantitative and qualitative aspects compared with other algorithms.
Image Matting, GAN, PatchGAN, RSU, U2Net, Self-Calibrated Convolutions
Yanlong Xu, Limin Sun, Qian Guo. Image Matting Algorithm Using Deep Generative Adversarial Networks. Academic Journal of Computing & Information Science (2023), Vol. 6, Issue 11: 60-71. https://doi.org/10.25236/AJCIS.2023.061109.
[1] Porter T, Duff T. Compositing digital images[C]//Proceedings of the 11th annual conference on Com-puter graphics and interactive techniques. 1984:253-259.
[2] Chuang Y Y, Curless B, Salesin D H, et al. A bayesian approach to digital matting[C]//Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001. IEEE, 2001, 2: II-II.
[3] Wang J, Cohen M F. An iterative optimization approach for unified image segmentation and matting [C] //Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1. IEEE, 2005, 2: 936-943.
[4] Ruzon M A, Tomasi C. Alpha estimation in natural images[C]//Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No. PR00662). IEEE, 2000, 1: 18-25.
[5] Sun J, Jia J, Tang C K, et al. Poisson matting [M]//ACM SIGGRAPH 2004 Papers. 2004: 315-321.
[6] Levin A, Lischinski D, Weiss Y. A closed-form solution to natural image matting [J]. IEEE transact- tions on pattern analysis and machine intelligence, 2007, 30(2): 228-242.
[7] Liu J J, Hou Q, Cheng M M, et al. Improving convolutional networks with self-calibrated convolutions[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020: 10096-10105.
[8] Chen Q, Li D, Tang C K. KNN matting[J]. IEEE transactions on pattern analysis and machine intel- ligence, 2013, 35(9): 2175-2188.
[9] Rhemann C, Rother C, Rav-Acha A, et al. High resolution matting via interactive trimap segmen- tation [C]//2008 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2008:1-8.
[10] Mishima Y. Soft edge chroma-key generation based upon hexoctahedral color space: U.S. Patent 5,355,174[P]. 1994-10-11.
[11] Guan Y, Chen W, Liang X, et al. Easy Matting-A Stroke Based Approach for Continuous Image Matting[C]//Computer Graphics Forum. Oxford, UK and Boston, USA: Blackwell Publishing, Inc, 2006, 25(3): 567-576.
[12] Shahrian E, Rajan D, Price B, et al. Improving image matting using comprehensive sampling sets [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013: 636-643.
[13] Johnson J,Varnousfaderani E S, Cholakkal H, et al. Sparse coding for alpha matting[J]. IEEE Tran- sactions on Image Processing, 2016, 25(7): 3032-3043.
[14] Karacan L, Erdem A, Erdem E. Alpha matting with kl-divergence-based sparse sampling[J]. IEEE Transactions on Image Processing, 2017, 26(9):4523-4536.
[15] Xu N, Price B, Cohen S,et al. Deep image matting[C]//Proceedings of the IEEE conference on com-puter vision and pattern recognition. 2017: 2970-2979.
[16] Hou Q, Liu F. Context-aware image matting for simultaneous foreground and alpha estimation [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 4130-4139.
[17] Lutz S, Amplianitis K, Smolic A. Alphagan: Generative adversarial networks for natural image mat-ting [J]. arXiv preprint arXiv:1807.10088,2018.
[18] Cai S, Zhang X, Fan H, et al.Disentangled image matting[C]//Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision. 2019:8819-8828.
[19] Sengupta S, Jayaram V, Curless B, et al. Background matting: The world is your green screen [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 2291-2300.
[20] Yu Q, Zhang J, Zhang H, et al. Mask guided matting via progressive refinement network[C]//Proce-edings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 1154-1163.
[21] Qiao Y, Liu Y, Yang X, et al. Attention-guided hierarchical structure aggregation for image matting[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 13676-13685.
[22] Ke Z, Li K, Zhou Y, et al. Is a green screen really necessary for real-time portrait matting?[J]. arXiv preprint arXiv:2011.11961, 2020.
[23] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778.
[24] Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks[C] //Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 4700-4708.
[25] Simonyan K, Zisserman A. Very deep convolutional networks for largescale image recognition[J]. arXiv preprint arXiv:1409.1556, 2014.
[26] Everingham M, Van Gool L,Williams C K I, et al. The pascal visual object classes (voc) challenge[J]. International journal of computer vision, 2010, 88(2): 303-338.
[27] Rhemann C, Rother C, Wang J, et al. A perceptually motivated online benchmark for image mat- ting[C]//2009 IEEE conference on computer vision and pattern recognition. IEEE, 2009: 1826-1833.
[28] Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE transactions on image processing, 2004, 13(4): 600-612.
[29] Chen X , Zou D , Zhou S Z , et al. Image Matting with Local and Nonlocal Smooth Priors[C]//Com- puter Vision & Pattern Recognition. IEEE, 2013.