Academic Journal of Computing & Information Science, 2023, 6(8); doi: 10.25236/AJCIS.2023.060803.
Xiyuan Miao, Shi Zhang
Central University of Finance and Economics, 100081, Beijing, China
This paper explores two main issues. First, this paper explores the optimal hyperparameters of the LeNet algorithm under the Fashion-MNIST dataset based on the grid method: where when the learning rate is 0.032, the regularization coefficient is 0.03, the momentum is 0.9, the weight decay parameter is 0.001, and the number of iterative rounds is 50, the model has the best results under the Fashion-MNIST dataset of 10% uniformly sampled samples has the relatively best results, i.e., the test accuracy converges to 85.8%. In addition, this paper improves the LeNet algorithm by constructing a LeNet deformation algorithm based on multi-conditional hyperparameter adjustment, specifically, the learning rate, momentum, and regularization coefficients change with the increase of the number of iteration rounds; in addition, in the construction of the model, the model introduces two blocks containing a convolutional layer, a batch normalization layer (BatchNorm), and a maximum pooling layer, and three linear neuron layers . After tuning, the tested accuracy of the algorithm is 91.5% under the full sample based on the Fashion-MNIST dataset.
LeNet algorithm; hyperparameter; Fashion-MNIST; pooling layer; convolutional layer
Xiyuan Miao, Shi Zhang. Revision of the LeNet algorithm——Construction of LeNet deformation algorithm based on multi-conditional hyperparameter adjustment. Academic Journal of Computing & Information Science (2023), Vol. 6, Issue 8: 22-36. https://doi.org/10.25236/AJCIS.2023.060803.
[1] E. R. Davies Machine Vision: Theory, Algorithms, Practicalities [M]. 2005.
[2] A. R. Pathak, M. Pandey and S. Rautaray(2018) Application of Deep Learning for Object Detection. Procedia Computer Science, 132, 1706-1717.
[3] Y. Zhang , E. Bieging , H. Tsui , and J. Jiang (2010) Efficient and Effective Extraction of Vocal Fold Vibratory Patterns from High-Speed Digital Imaging. Journal of Voice Official Journal of the Voice Foundation,24(1),21-29.
[4] Y. Niu, L. Ying, J. Yang, M. Bao and C. B. Sivaparthipan(2021). Organizational business intelligence and decision making using big data analytics. Information Processing & Management,58(6), 102725.
[5] J. Liu, H. Chang, Y.L. Forrest and B. Yang(2020) Influence of artificial intelligence on technological innovation: Evidence from the panel data of china's manufacturing sectors." Technological Forecasting and Social Change 158, 120142.
[6] Y. Lecun and L. Bottou(1998) "Gradient-based learning applied to document recognition." Proceedings of the IEEE, 86(11),2278-2324.
[7] S. Ghosh, R. Shet, P. Amon, A. Hutter and A. Kaup (2018), Robustness of Deep Convolutional Neural Networks for Image Degradations, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018, 2916-2920.
[8] X. Dai, H. Yin, and N. K. Jha. (2020) Incremental Learning Using a Grow-and-Prune Paradigm with Efficient Neural Networks. IEEE Transactions on Emerging Topics in Computing.
[9] Y. Wen, K. Zhang, Z. Li and Y. Qiao. (2016). A Discriminative Feature Learning Approach for Deep Face Recognition. European Conference on Computer Vision Springer, Cham.