Optimal Tracking Control of Nonlinear Multiagent Systems Using Internal Reinforce Q-Learning

Zhinan Peng, Rui Luo, Jiangping Hu, Kaibo Shi, Sing Kiong Nguang, Bijoy Kumar Ghosh

Research output: Contribution to journalArticlepeer-review

4 Scopus citations

Abstract

In this article, a novel reinforcement learning (RL) method is developed to solve the optimal tracking control problem of unknown nonlinear multiagent systems (MASs). Different from the representative RL-based optimal control algorithms, an internal reinforce Q-learning (IrQ-L) method is proposed, in which an internal reinforce reward (IRR) function is introduced for each agent to improve its capability of receiving more long-term information from the local environment. In the IrQL designs, a Q-function is defined on the basis of IRR function and an iterative IrQL algorithm is developed to learn optimally distributed control scheme, followed by the rigorous convergence and stability analysis. Furthermore, a distributed online learning framework, namely, reinforce-critic-actor neural networks, is established in the implementation of the proposed approach, which is aimed at estimating the IRR function, the Q-function, and the optimal control scheme, respectively. The implemented procedure is designed in a data-driven way without needing knowledge of the system dynamics. Finally, simulations and comparison results with the classical method are given to demonstrate the effectiveness of the proposed tracking control method.

Keywords

  • Artificial neural networks
  • Heuristic algorithms
  • Iterative algorithms
  • Mathematical model
  • Multi-agent systems
  • Neural networks (NNs)
  • Nonlinear dynamical systems
  • Optimal control
  • nonlinear multiagent systems (MASs)
  • optimal tracking control
  • reinforcement learning (RL).

Fingerprint

Dive into the research topics of 'Optimal Tracking Control of Nonlinear Multiagent Systems Using Internal Reinforce Q-Learning'. Together they form a unique fingerprint.

Cite this