AirSim-Based Evaluation of Deep Q-Networks for Autonomous UAV Navigation in Dynamic Environments
Keywords:
AI drones, Air Sim, Autonomous UAV, Deep Q-network, Dynamic navigation, Obstacle avoidance, Reinforcement learning, Simulation framework, Smart navigationAbstract
Unmanned Aerial Vehicles (UAVs) are increasingly being utilized in dynamic and complex environments for tasks ranging from surveillance to disaster response. However, real-time decision-making in unpredictable terrains remains a significant challenge. This paper presents a simulation-based evaluation of a Deep Q-Network (DQN) reinforcement learning algorithm for autonomous UAV navigation using Microsoft's AirSim platform. The framework integrates sensory inputs such as GPS, IMU, and front-facing RGB camera streams to train an agent capable of obstacle avoidance and target reaching in real time. The UAV learns optimal policies through reward-based interactions within a photorealistic environment featuring dynamic obstacles and shifting target zones. Performance metrics, including episode success rate, average cumulative reward, and navigation time, are analyzed and compared against a baseline random policy. Experimental results demonstrate that the DQN-based agent achieves significantly improved navigation accuracy and robustness, validating its potential for real-world deployment in intelligent drone systems.
References
H. Lee and S. Park, “Sensing-Aware Deep Reinforcement Learning With HCI-Based Human-in-the-Loop Feedback for Autonomous Nonlinear Drone Mobility Control,” IEEE Access, vol. 12, pp. 1727–1736, 2024, doi: https://doi.org/10.1109/access.2023.3346917.
C.-Y. Lee, A. Khanum, N.-C. Wang, Bala, and C.-S. Yang, “An Efficient Lane Following Navigation Strategy With Fusion Attention for Autonomous Drones in Urban Areas,” IEEE Transactions on Vehicular Technology, vol. 73, no. 3, pp. 3094–3105, Oct. 2023, doi: https://doi.org/10.1109/tvt.2023.3322808.
L. Zeng, H. Chen, D. Feng, X. Zhang, and X. Chen, “A3D: Adaptive, Accurate, and Autonomous Navigation for Edge-Assisted Drones,” IEEE/ACM transactions on networking, pp. 1–16, Jan. 2023, doi: https://doi.org/10.1109/tnet.2023.3297876.
L. Lamberti, E. Cereda,, G. Abbate, L. Bellone, “A Sim-to-Real Deep Learning-Based Framework for Autonomous Nano-Drone Racing,” IEEE Robotics and Automation Letters, vol. 9, no. 2, pp. 1899–1906, Feb. 2024, doi: https://doi.org/10.1109/lra.2024.3349814.
M. Bosello , D. Aguiari ,Y. Keuter, “Race Against the Machine: A Fully-Annotated, Open-Design Dataset of Autonomous and Piloted High-Speed Flight,” IEEE Robotics and Automation Letters, vol. 9, no. 4, pp. 3799–3806, Feb. 2024, doi: https://doi.org/10.1109/lra.2024.3371288.
A. Famili, A. Stavrou, H. Wang, and J. Park, “PILOT: High-Precision Indoor Localization for Autonomous Drones,” IEEE Transactions on Vehicular Technology, vol. 72, no. 5, pp. 6445–6459, May 2023, doi: https://doi.org/10.1109/tvt.2022.3229628.
R. Alyassi, M. Khonji, A. Karapetyan, S. C.-K. Chau, K. Elbassioni, and C.-M. Tseng, “Autonomous Recharging and Flight Mission Planning for Battery-Operated Autonomous Drones,” IEEE Transactions on Automation Science and Engineering, pp. 1–13, 2022, doi: https://doi.org/10.1109/tase.2022.3175565.
S. H. Alsamhi, “Blockchain-Empowered Security and Energy Efficiency of Drone Swarm Consensus for Environment Exploration,” IEEE Transactions on Green Communications and Networking, pp. 1–1, 2022, doi: https://doi.org/10.1109/tgcn.2022.3195479.
M. Zager and A. Fay, “Design Principles for Distributed Context Modeling of Autonomous Systems,” IEEE Open Journal of Systems Engineering, vol. 1, pp. 179–189, Jan. 2023, doi: https://doi.org/10.1109/ojse.2023.3342572.
T. Hiraguri , H. Shimizu, T. Kimura, “Autonomous Drone-Based Pollination System Using AI Classifier to Replace Bees for Greenhouse Tomato Cultivation,” IEEE Access, vol. 11, pp. 99352–99364, Jan. 2023, doi: https://doi.org/10.1109/access.2023.3312151.
H. Du, W. Wang, X. Wang, and Y. Wang, “Autonomous landing scene recognition based on transfer learning for drones,” Journal of Systems Engineering and Electronics, vol. 34, no. 1, pp. 28–35, Feb. 2023, doi: https://doi.org/10.23919/jsee.2023.000031.
X. Li, H. Huang, and A. V. Savkin, “Autonomous Navigation of an Aerial Drone to Observe a Group of Wild Animals With Reduced Visual Disturbance,” IEEE Systems Journal, vol. 16, no. 2, pp. 3339–3348, Jun. 2022, doi: https://doi.org/10.1109/jsyst.2021.3135982.
A. V. Savkin and H. Huang, “Range-Based Reactive Deployment of Autonomous Drones for Optimal Coverage in Disaster Areas,” IEEE Transactions on Systems, Man and Cybernetics Systems, vol. 51, no. 7, pp. 4606–4610, Oct. 2019, doi: https://doi.org/10.1109/tsmc.2019.2944010.
K. Li, W. Ni, and F. Dressler, “Continuous Maneuver Control and Data Capture Scheduling of Autonomous Drone in Wireless Sensor Networks,” IEEE Transactions on Mobile Computing, pp. 1–1, 2021, doi: https://doi.org/10.1109/tmc.2021.3049178.
M. A. Khan, H. Menouar, A. Eldeeb, A. Abu-Dayya, and F. D. Salim, “On the Detection of Unauthorized Drones—Techniques and Future Perspectives: A Review,” IEEE Sensors Journal, vol. 22, no. 12, pp. 11439–11455, Jun. 2022, doi: https://doi.org/10.1109/JSEN.2022.3171293.
K.-W. Chen, M.-R. Xie, Y.-M. Chen, T.-T. Chu, and Y.-B. Lin, “DroneTalk: An Internet-of-Things-Based Drone System for Last-Mile Drone Delivery,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 9, pp. 1–14, 2022, doi: https://doi.org/10.1109/TITS.2021.3138432.
M. A. Arshad et al., “Drone Navigation Using Region and Edge Exploitation-Based Deep CNN,” IEEE Access, vol. 10, pp. 95441–95450, 2022, doi: https://doi.org/10.1109/access.2022.3204876.
A. Devo, J. Mao, G. Costante, and G. Loianno, “Autonomous Single-Image Drone Exploration With Deep Reinforcement Learning and Mixed Reality,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 5031–5038, Apr. 2022, doi: https://doi.org/10.1109/lra.2022.3154019.
T. Huynh-The, Q.-V. Pham, T.-V. Nguyen, D. Benevides, and D.-S. Kim, “RF-UAVNet: High-Performance Convolutional Network for RF-Based Drone Surveillance Systems,” IEEE Access, vol. 10, pp. 49696–49707, Jan. 2022, doi: https://doi.org/10.1109/access.2022.3172787.
D. Hernández, J.-C. Cano, F. Silla, C. T. Calafate, and J. M. Cecilia, “AI-Enabled Autonomous Drones for Fast Climate Change Crisis Assessment,” IEEE Internet of Things Journal, vol. 9, no. 10, pp. 7286–7297, May 2022, doi: https://doi.org/10.1109/JIOT.2021.3098379.
M. Kazim, M. Zaidi, S. Ali, “Perception Action Aware-Based Autonomous Drone Race in a Photorealistic Environment,” IEEE Access, vol. 10, pp. 42566–42576, Jan. 2022, doi: https://doi.org/10.1109/access.2022.3168710.
V. R. Parihar, S. A. Chaturvedi, “An Adaptive Machine Learning based Framework for Autonomous Drone Navigation”, International Conference on Recent Trends and Research in Engineering & Science (ICRTRES 2K25), May 2025, Available: https://zenodo.org/records/16314432
