Welcome to Francis Academic Press

International Journal of Frontiers in Engineering Technology, 2023, 5(3); doi: 10.25236/IJFET.2023.050307.

FPGA-Based Design for Accelerating 3D Convolutional Neural Networks

Author(s)

Yuesong Yang1,2, Wenjing He1, Jian Hu1, Sai Cheng1, Wanqiu Xu1

Corresponding Author:
Wenjing He
Affiliation(s)

1Key Laboratory of Quantitative Remote Sensing Information Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, 100094, China

2School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing, 100049, China

Abstract

Three-dimensional convolutional neural networks (3D CNNs) have been shown to be excellent for classification in human action recognition, hyperspectral image classification and many other aspects. However, 3D CNNs often contain convolution kernels of different sizes and a large number of operations, and the numerous operations make 3D CNNs consume a lot of time when executing inference on resource-constrained terminals. Aiming at these two points, this paper proposes a 3D CNN accelerator design based on loop optimization and weight reuse. The design adopts loop tiling, loop unrolling and loop fusion to optimize the direct convolution form of the 3D convolution operation; and guided by the optimized form of the 3D convolution operation, a module that can accelerate different 3D convolution operations is designed; and a memory access method based on weight reuse is also designed to reduce the memory access time of features and weights. Experimental results show that the proposed accelerator not only supports convolution kernels of different sizes, but also has a performance density of 1.59, which outperforms most existing accelerators.

Keywords

3D CNN, Loop Optimization, Parallel Computing, FPGA

Cite This Paper

Yuesong Yang, Wenjing He, Jian Hu, Sai Cheng, Wanqiu Xu. FPGA-Based Design for Accelerating 3D Convolutional Neural Networks. International Journal of Frontiers in Engineering Technology (2023), Vol. 5, Issue 3: 40-48. https://doi.org/10.25236/IJFET.2023.050307.

References

[1] Ji S, Xu W, Yang M, et al. 3D convolutional neural networks for human action recognition[J]. IEEE transactions on pattern analysis and machine intelligence, 2012,35(1):221-231.

[2] Roy S K, Krishna G, Dubey S R, et al. HybridSN: Exploring 3-D–2-D CNN feature hierarchy for hyperspectral image classification [J]. IEEE Geoscience and Remote Sensing Letters, 2019,17(2):277-281.

[3] Fan H, Niu X, Liu Q, et al. F-C3D: FPGA-based 3-dimensional convolutional neural network[C]. 2017 27th International Conference on Field Programmable Logic and Applications (FPL), 2017. IEEE.

[4] Shen J, Huang Y, Wang Z, et al. Towards a uniform template-based architecture for accelerating 2D and 3D CNNs on FPGA[C]. Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, 2018.

[5] Liu Z, Chow P, Xu J, et al. A uniform architecture design for accelerating 2D and 3D CNNs on FPGAs [J]. Electronics, 2019, 8(1):65.

[6] Bishop C M. Neural networks and their applications[J]. Review of scientific instruments, 1994, 65(6):1803-1832.

[7] Tran D, Bourdev L, Fergus R, et al. Learning spatiotemporal features with 3d convolutional networks[C]. Proceedings of the IEEE international conference on computer vision, 2015.

[8] Hou R, Zhang L, Huang M C, et al. Efficient data streaming with on-chip accelerators: Opportunities and challenges[C]. 2011 IEEE 17th International Symposium on High Performance Computer Architecture, 2011. IEEE.