Academic Journal of Computing & Information Science, 2026, 9(7); doi: 10.25236/AJCIS.2026.090702.
Tang Zequn
School of Physics and Electronic Science, Hunan University of Science and Technology, Xiangtan, Hunan, 411201, China
Deep learning inference on edge devices must balance speed, energy, memory, temperature, and accuracy. This paper evaluates PyTorch FP32, TensorRT FP32, and TensorRT FP16 inference on an NVIDIA Jetson Orin Nano 8GB using ResNet-18, MobileNetV2, and YOLOv5s. We measure throughput, power, energy, FPS/W, RAM usage, and temperature rise with a 1000 images x 5 repeated workload, while accuracy is evaluated on ImageNet and COCO2017. Results show that TensorRT greatly reduces energy and improves efficiency. ResNet-18 energy decreases from 557.9 J to 113.9 J under TensorRT FP16 with nearly unchanged Top-1 accuracy. MobileNetV2 achieves strong energy reduction but shows a clear FP16 accuracy drop. YOLOv5s TensorRT results are reported as engine-level measurements because the preprocessing and post-processing scope differs from the PyTorch pipeline. The results show that edge deployment should be evaluated as a system-level trade-off rather than by FPS alone.
Edge AI, Jetson Orin Nano, TensorRT, PyTorch, FP16 inference, power consumption, energy efficiency, memory usage, thermal behavior
Tang Zequn. Comprehensive Power and Performance Characterization of Deep Neural Networks on Jetson Orin Nano with PyTorch and TensorRT. Academic Journal of Computing & Information Science (2026), Vol. 9, Issue 7: 9-19. https://doi.org/10.25236/AJCIS.2026.090702.
[1] X. Wang et al., “A Comprehensive Survey on On-Device AI Models,” ACM Computing Surveys, 2025.
[2] D. K. Alqahtani, A. Cheema, and A. N. Toosi, “Benchmarking Deep Learning Models for Object Detection on Edge Computing Devices,” arXiv preprint, arXiv:2409.16808, 2024.
[3] S. P. Baller, A. Jindal, M. Chadha, and M. Gerndt, “DeepEdgeBench: Benchmarking Deep Neural Networks on Edge Devices,” arXiv preprint, arXiv:2108.09457, 2021.
[4] NVIDIA, “Jetson Orin Nano Super Developer Kit,” NVIDIA Embedded Systems, 2024. [Online]. Available: https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/nano-super-developer-kit/. Accessed: Jul. 3, 2026.
[5] NVIDIA, “NVIDIA TensorRT Developer Guide,” NVIDIA Developer Documentation, 2023. [Online]. Available: https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/. Accessed: Jul. 3, 2026.
[6] A. Paszke et al., “PyTorch: An Imperative Style, High-Performance Deep Learning Library,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2019.
[7] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2016, pp. 770–778.
[8] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “MobileNetV2: Inverted Residuals and Linear Bottlenecks,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2018, pp. 4510–4520.
[9] G. Jocher et al., “YOLOv5 by Ultralytics,” Zenodo, 2020, doi: 10.5281/zenodo.3908559.
[10] H. Ahn, T. Chen, N. Alnaasan, A. Shafi, M. Abduljabbar, H. Subramoni, and D. K. Panda, “Performance Characterization of Using Quantization for DNN Inference on Edge Devices,” arXiv preprint, arXiv:2303.05016, 2023.