[author_cn_name].[cn_title][J].空军工程大学学报:自然科学版,[year_id],[volume]([issue]):[start_page]-[end_page] 基于Q-network强化学习的超视距空战机动决策-BVR Air Combat Maneuvering Decision by Using Q-network Reinforcement Learning
文章摘要
张强,杨任农,俞利新,张涛,左家亮.基于Q-network强化学习的超视距空战机动决策[J].空军工程大学学报:自然科学版,2018,19(6):8-14
基于Q-network强化学习的超视距空战机动决策
BVR Air Combat Maneuvering Decision by Using Q-network Reinforcement Learning
  
DOI:
中文关键词: 超视距空战  机动决策  强化学习  纳什均衡
英文关键词: beyond visual range air combat  maneuvering decision  reinforcement learning  Nash equilibrium
基金项目:
作者单位
张强,杨任农,俞利新,张涛,左家亮 空军工程大学空管领航学院,西安,710051 
摘要点击次数: 234
全文下载次数: 212
中文摘要:
      考虑到空空导弹对空战胜负的重要影响,针对空战态势状态特征连续、多维的情况以及传统方法缺乏对空战对抗中敌方策略的考虑,将强化学习应用到1vs1超视距空战机动决策。首先,建立了同时为对抗双方进行机动决策的强化学习框架,提出ε-纳什均衡策略来选取机动动作,并通过导弹攻击区优势函数来修正奖赏函数;其次,基于记忆库和目标网络训练Q-network,形成超视距空战机动决策的“价值网络”;最后,设计了Q-network强化学习决策模型,并将机动决策过程分为了学习阶段与实战阶段。仿真结果表明:智能体可以感知空战的态势并作出合理的超视距空战机动决策。
英文摘要:
      In consideration of the great Impact of missiles on air combat, the continuous and multidimensional state space and the weakness of traditional approaches in ignoring opponent’s strategy in the air combat, reinforcement learning is applied to 1vs1 beyond visual range (BVR) air combat maneuvering decisions. Firstly, a new reinforcement learning framework is built to decide both sides’ maneuvers. In this framework,ε-Nash equilibrium strategy is proposed to choose action, and reward function is revised by missile attack zone scoring function. Then, by using a memory base and a target network, Q-network can be trained, forming a “value network” for BVR air combat maneuvering decisions. Finally,Q-network reinforcement learning model is designed, and the whole maneuvering decision is divided into learning part and strategy forming part. In the simulation, considering that the enemy in the air combat confrontation adopts a fixed maneuver and the two sides are both agents, the former agent wins, and the latter has the advantage of the situation to win, verifying that the agent can perceive the situation of air combat and make a reasonable BVR air combat maneuver.
查看全文   查看/发表评论  下载PDF阅读器
关闭