Sampled-data Control through Model-Free Reinforcement Learning with Effective Experience Replay

Bo Xiao, Hak-Keung Lam, Xiaojie Su, Ziwei Wang, Frank P. W. Lo , Shihong Chen, Eric Yeatman

Research output: Contribution to journalArticlepeer-review

17 Downloads (Pure)

Abstract

Reinforcement Learning (RL) based control algorithms can learn the control strategies for nonlinear and uncertain environment during interacting with it. Guided by the rewards generated by environment, a RL agent can learn the control strategy directly in a model-free way instead of investigating the dynamic model of the environment. In the paper, we propose the sampled-data RL control strategy to reduce the computational demand. In the sampled-data control strategy, the whole control system is of a hybrid structure, in which the plant is of continuous structure while the controller (RL agent) adopts a discrete structure. Given that the continuous states of the plant will be the input of the agent, the state–action value function is approximated by the fully connected feed-forward neural networks (FCFFNN). Instead of learning the controller at every step during the interaction with the environment, the learning and acting stages are decoupled to learn the control strategy more effectively through experience replay. In the acting stage, the most effective experience obtained during the interaction with the environment will be stored and during the learning stage, the stored experience will be replayed to customized times, which helps enhance the experience replay process.
Original languageEnglish
JournalJournal of Automation and Intelligence
DOIs
Publication statusPublished - 25 Jan 2023

Fingerprint

Dive into the research topics of 'Sampled-data Control through Model-Free Reinforcement Learning with Effective Experience Replay'. Together they form a unique fingerprint.

Cite this