Simulations are conducted to verify the effectiveness of the proposed method, and the kinematic model of the missile is used in simulations instead of the missile engagement zone to test whether the maneuver decision-making method is effective or not.

Then, repeat the above process to gradually improve the maneuver decision-making ability. These samples are used to train the neural network, and the neural network with a greater winning rate is selected by simulations. It starts from random behaviors and generates samples consisting of states, actions, and results of air combat through self-play without using human knowledge.

To this end, Monte Carlo tree search in continuous action space is proposed and neural networks-guided Monte Carlo tree search with self-play is utilized to improve the ability of air combat agents. Therefore, a maneuver decision-making method based on deep reinforcement learning and Monte Carlo tree search is proposed to investigate whether it is feasible for maneuver decision-making without human knowledge or advantage function. Aeronautics Engineering College, Air Force Engineering University, Xi'an, ChinaĪutonomous maneuver decision-making methods for air combat often rely on human knowledge, such as advantage functions, objective functions, or dense rewards in reinforcement learning, which limits the decision-making ability of unmanned combat aerial vehicle to the scope of human experience and result in slow progress in maneuver decision-making.

Hongpeng Zhang * Huan Zhou Yujie Wei Changqiang Huang

Air combat maneuver