In response to the problem of low efficiency and weak obstacle avoidance ability in finding the optimal or suboptimal path for 3D path planning of drones in complex dynamic environments, this paper uses Q-learning algorithm to complete the 3D path planning of drones, aiming to improve their path planning and obstacle avoidance capabilities. Firstly, by constructing a three-dimensional gridded environment model, the system calculates the reward for each state under the influence of natural environment and obstacles, and then guides the drone to avoid obstacles and find the optimal path. The system uses the ε - greedy strategy for exploration and learning, optimizing decisions by continuously updating the Q-table value table. The experimental results show that the drone has a success rate of 93.3% in obstacle avoidance in complex and multi obstacle scenes. Moreover, in terms of average path length, the Q-learning algorithm has shortened it by approximately 20.00%, 11.45%, and 40.39% compared to ant colony algorithm, A* algorithm, and RRT algorithm, respectively. In dynamic wind speed environments, the Q-learning algorithm reduces the path length by about 4% to 11% compared to other algorithms, further demonstrating its effectiveness and advantages in complex environments.

Zhengxiang Huang, Yong Tian. Three Dimensional Path Planning System for Unmanned Aerial Vehicles Based on Reinforcement Learning Algorithm. Academic Journal of Engineering and Technology Science (2024) Vol. 7, Issue 6: 155-161. https://doi.org/10.25236/AJETS.2024.070622.

[1] Ramasamy M, Ghose D. Learning-based preferential surveillance algorithm for persistent surveillance by unmanned aerial vehicles[J].IEEE, 2016.DOI:10.1109/ICUAS.2016.7502678.

[2] Mandloi D, Arya R, Verma A K. Unmanned aerial vehicle path planning based on A* algorithm and its variants in 3d environment [J]. International Journal of System Assurance Engineering and Management, 2021, 12(5): 990-1000.

[3] Kiani F, Seyyedabbasi A, Aliyev R, et al. 3D path planning method for multi-UAVs inspired by grey wolf algorithms[J]. Journal of Internet Technology, 2021, 22(4): 743-755.

[4] Lv J X, Yan L J, Chu S C, et al. A new hybrid algorithm based on golden eagle optimizer and grey wolf optimizer for 3D path planning of multiple UAVs in power inspection[J]. Neural Computing and Applications, 2022, 34(14): 11911-11936.

[5] Maboudi M, Homaei M R, Song S, et al. A Review on Viewpoints and Path Planning for UAV-Based 3-D Reconstruction [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2023, 16: 5026-5048.

[6] Zammit C, van Kampen E J. Real-time 3D UAV path planning in dynamic environments with uncertainty [J]. Unmanned Systems, 2023, 11(03): 203-219.

[7] Wang C, Yang X, Li H. Improved Q-learning applied to dynamic obstacle avoidance and path planning [J]. IEEE Access, 2022, 10: 92879-92888.

[8] Shang Y, Liu F, Qin P, et al. Research on path planning of autonomous vehicle based on RRT algorithm of Q-learning and obstacle distribution[J]. Engineering Computations, 2023, 40(5): 1266-1286.

[9] Bonny T, Kashkash M. Highly optimized Q‐learning‐based bees approach for mobile robot path planning in static and dynamic environments[J]. Journal of Field Robotics, 2022, 39(4): 317-334.

[10] Lyu J, Ma X, Li X, et al. Mildly conservative q-learning for offline reinforcement learning[J]. Advances in Neural Information Processing Systems, 2022, 35(1): 1711-1724.

[11] Tran C H, Bui T K, Pham T V. Virtual machine migration policy for multi-tier application in cloud computing based on Q-learning algorithm[J]. Computing, 2022, 104(6): 1285-1306.

[12] Chintala P, Dornberger R, Hanne T. Robotic path planning by Q learning and a performance comparison with classical path finding algorithms[J]. International Journal of Mechanical Engineering and Robotics Research, 2022, 11(6): 373-378. 

[13] Hu J. A novel deep learning driven robot path planning strategy: Q-learning approach[J]. International Journal of Computer Applications in Technology, 2023, 71(3): 237-243.