As a specific application of image inpainting, face inpainting is a critical content in the computer vision. It plays an important role in object removal, photo editing and other fields. Deep learning has become the mainstream approach of image inpainting. In specific applications, the corrupted area of face images is usually irregular. For the classical irregular face inpainting approaches based on deep learning, this paper divides it into convolution operator optimization methods and structural information constraint methods, the former includes PConv and GConv and the latter includes EC, PRVS, MED, CTSDG. We fist describe the basic principle of each algorithm and detail about the strengths and limitations. Then we experiment on CelebA-HQ dataset, evaluate and compare the performance quantitatively and qualitatively.

Xinyue Zuo, You Yang, Zixian Hao. Performance Comparison of Irregular Face Inpainting via Deep Learning. Academic Journal of Computing & Information Science (2022), Vol. 5, Issue 3: 70-77. https://doi.org/10.25236/AJCIS.2022.050310.

[1] Esedoglu S, Shen J. Digital inpainting based on the Mumford–Shah–Euler image model[J]. European Journal of Applied Mathematics, 2002, 13(4): 353-370. 

[2] Liu D, Sun X, Wu F, et al. Image compression with edge-based inpainting[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2007, 17(10): 1273-1287.

[3] Darabi S, Shechtman E, Barnes C, et al. Image melding: Combining inconsistent images using patch-based synthesis[J]. ACM Transactions on graphics (TOG), 2012, 31(4): 1-10.

[4] Huang J B, Kang S B, Ahuja N, et al. Image completion using planar structure guidance[J]. ACM Transactions on graphics (TOG), 2014, 33(4): 1-10.

[5] Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets[J]. Advances in neural information processing systems, 2014, 27. 

[6] Pathak D, Krahenbuhl P, Donahue J, et al. Context encoders: Feature learning by inpainting[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 2536-2544.

[7] Iizuka S, Simo-Serra E, Ishikawa H. Globally and locally consistent image completion[J]. ACM Transactions on Graphics (ToG), 2017, 36(4): 1-14.

[8] Yu J, Lin Z, Yang J, et al. Generative image inpainting with contextual attention[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 5505-5514.

[9] Liu G, Reda F A, Shih K J, et al. Image inpainting for irregular holes using partial convolutions[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 85-100.

[10] Yu J, Lin Z, Yang J, et al. Free-form image inpainting with gated convolution[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 4471-4480.

[11] Nazeri K, Ng E, Joseph T, et al. Edgeconnect: Generative image inpainting with adversarial edge learning[J]. arXiv preprint arXiv:1901.00212, 2019.

[12] Li J, He F, Zhang L, et al. Progressive reconstruction of visual structure for image inpainting[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 5962-5971.

[13] Liu H, Jiang B, Song Y, et al. Rethinking image inpainting via a mutual encoder-decoder with feature equalizations[C]//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16. Springer International Publishing, 2020: 725-741.

[14] Guo X, Yang H, Huang D. Image Inpainting via Conditional Texture and Structure Dual Generation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 14134-14143.

[15] Zhang R, Isola P, Efros A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 586-595.

[16] You Yang, Kesen Li, Sixun Liu, Ling Luo, Bin Xing. Perceptual Face Inpainting with Multi-column Gated Convolutional Network[J]. Journal of Electronic Imaging, 31(1), 013022 (2022). https://doi.org/10.1117/1.JEI.31.1.013022