Maneuver Decision-Making Through Proximal Policy Optimization And Monte Carlo Tree Search. (arXiv:2309.08611v1 [cs.AI])

Maneuver decision-making can be regarded as a Markov decision process and can
be address by reinforcement learning. However, original reinforcement learning
algorithms can hardly solve the maneuvering decision-making problem. One reason
is that agents use random actions in the early stages of training, which makes
it difficult to get rewards and learn how to make effective decisions. To
address this issue, a method based on proximal policy optimization and Monte
Carlo tree search is proposed. The method uses proximal policy optimization to
train the agent, and regards the results of air combat as targets to train the
value network. Then, based on the value network and the visit count of each
node, Monte Carlo tree search is used to find the actions with more expected
returns than random actions, which can improve the training performance. The
ablation studies and simulation experiments indicate that agents trained by the
proposed method can make different decisions according to different states,
which demonstrates that the method can solve the maneuvering decision problem
that the original reinforcement learning algorithm cannot solve.