USING REINFORCEMENT LEARNING ALGORITHMS FOR UAV FLIGHT OPTIMIZATION

O. Dutsiak, V. Yuzevych

Abstract


Unmanned aerial vehicles (UAVs) have become a compelling research topic, offering potential for both civilian and military applications such as inspection, delivery, reconnaissance, and surveillance. Areas of interest include autonomous navigation, real-time path planning and object recognition. However, autonomous flight poses a number of challenges for UAVs, including the development of control strategies, parameter tuning, adaptive control, real-time trajectory planning, and object recognition in uncertain environments. In recent years, the implementation of machine learning has gained popularity as a promising approach to solving these problems in autonomous UAV flight. Machine learning gives UAVs the ability to discern patterns and make predictions based on data, thus bypassing the need for pre-programmed instructions during autonomous flight.

In this work, the main attention was focused on the study of a sub-method of machine learning, which is called learning with emphasis, with a view to optimizing the flight of a UAV by optimizing the process of constructing its path. A UAV trajectory planning method based on the gradient RL algorithm was proposed. It systematically presents the principles and components of the teaching method with emphasis. The path planning task is implemented using deep reinforcement learning, and the 3D environment is described using raster methods to construct a raster map. The study of the results of the functionality of the proposed algorithm was carried out in the environment of three-dimensional modeling. The algorithm is used to determine the optimal policy by treating each raster as a state, the 26 possible directions as action sets, and the action value function as the objective function.

When studying the results of the conducted experiment, it was found that after 160 experiments, thanks to the use of the RL gradient method, it was possible to determine the path from the starting point to the final point and successfully overcome all obstacles within a 84 seconds time period. Prospects for further improvement of the proposed method may include the study of methods to ensure more flexible movement within the map and optimization of sampling efficiency for improved real-world applicability.

Keywords: autonomous systems, machine learning, flight navigation, path planning, reinforcement learning, control algorithms.


References


  1. Ruseno, Neno & Lin, Chung-Yan. (2024). Real-Time UAV Trajectory Prediction for UTM Surveillance Using Machine Learning. Unmanned Systems. 10.1142/S230138502550030X
  2. Afifi, Ghada & Gadallah, Yasser. (2022). Cellular Network-Supported Machine Learning Techniques for Autonomous UAV Trajectory Planning. IEEE Access. PP. 1-1. 10.1109/ACCESS.2022.3229171.
  3. Ma, Chao & Qu, Zhi & Li, Xianbin & Zongmin, Liu & Zhou, Chao. (2023). Machine Learning and UAV Path Following Identification Algorithm Based on Navigation Spoofing. Measurement Science and Technology. 34. 10.1088/1361-6501/acf3da
  4. Khan, Anamta & Campos, Joao & Ivaki, Naghmeh & Madeira, Henrique. (2023). A Machine Learning driven Fault Tolerance Mechanism for UAVs' Flight Controller. 10.1109/PRDC59308.2023.00034.
  5. Fagundes-Junior, Leonardo & de Carvalho, Kevin & Ferreira, Ricardo & Brandão, Alexandre Santos. (2024). Machine Learning for Unmanned Aerial Vehicles Navigation: An Overview. SN Computer Science. 5. 10.1007/s42979-023-02592-5
  6. Journal, IJSREM. (2023). Autonomous Drone Navigation. INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT. 07. 1-11. 10.55041/ijsrem27050
  7. Fadi AlMahamid, Katarina Grolinger, Autonomous Unmanned Aerial Vehicle navigation using Reinforcement Learning: A systematic review, Engineering Applications of Artificial Intelligence, Volume 115, 2022, 105321, ISSN 0952-1976. 10.1016/j.engappai.2022.105321.
  8. Ullah, Zaib & Al-Turjman, Fadi & Moatasim, Uzair & Mostarda, Leonardo & Gagliardi, Roberto. (2020). UAVs joint optimization problems and machine learning to improve the 5G and Beyond communication. Computer Networks. 182. 10.1016/j.comnet.2020.107478
  9. Choi, Su Yeon & Cha, Dowan. (2019). Unmanned aerial vehicles using machine learning for autonomous flight; state-of-the-art. Advanced Robotics. 33. 1-13. 10.1080/01691864.2019.1586760
  10. Quan, Yunhao & Cheng, Nan & Wang, Xiucheng & Shen, Jinglong & Ma, Longfei & Yin, Zhisheng. (2023). Interpretable and Secure Trajectory Optimization for UAV-Assisted Communication. 10.48550/arXiv.2307.02002
  11. Teixeira da Silva, Karolayne & Miguel, Geovane & Silva, Hugerles & Madeiro, Francisco. (2023). A Survey on Applications of Unmanned Aerial Vehicles Using Machine Learning. IEEE Access. PP. 1-1. 10.1109/ACCESS.2023.3326101
  12. Al-Haddad, Luttfi & Jaber, Alaa. (2023). Applications of Machine Learning Techniques for Fault Diagnosis of UAVs. ceur-ws.org/Vol-3360/p03.pdf
  13. Jacob, Billy & Kaushik, Abhishek & Velavan, Pankaj. (2022). Autonomous Navigation of Drones Using Reinforcement Learning. 10.1007/978-981-16-7220-0_10
  14. Yuzevych, V. M., Lozovan, V. P. Influence of Mechanical Stresses on the Propagation of Corrosion Cracks in Pipeline Walls // Materials Science, 2022. Volume 57, No. 4. P. 539-548. 10.1007/s11003-022-00576-z
  15. Сопрунюк П. М., Юзевич В. М. Діагностика матеріалів і середовищ. Енергетичні характеристики поверхневих шарів. Львів: Сполом, 2005. 292 с. nvd-nanu.org.ua/d91f5656-e1ea-1bc9-3bfb-8d57729fbdad
  16. Дослідження взаємозв’язку між енергією поверхні волокнистих наповнювачів та міцністю полімерних композицій на їх основі / О. В. Миронюк, В. А. Дудко, Д. В. Баклан, К. О. Смольнич // Вісник НТУ “ХПІ”. № 50 (1222). 2016. C. 3-8. mtsc.khpi.edu.ua/article/view/99759/95070




DOI: http://dx.doi.org/10.30970/eli.28.10

Refbacks

  • There are currently no refbacks.