Continuous Interval Type-2 Fuzzy Q-learning Algorithm for Trajectory Tracking Tasks for Vehicles

TY - JOUR

T1 - Continuous Interval Type-2 Fuzzy Q-learning Algorithm for Trajectory Tracking Tasks for Vehicles

AU - Xuan, Chengbin

AU - Lam, Hak-Keung

AU - Shi, Qian

AU - Chen, Ming

N1 - Funding Information: This work was supported by King's College London. Publisher Copyright: © 2022 The Authors. International Journal of Robust and Nonlinear Control published by John Wiley & Sons Ltd.

PY - 2022/5/25

Y1 - 2022/5/25

N2 - Trajectory tracking is a fundamental but challenging task for vehicle automation. In addition to the system nonlinearity, the main difficulties in the trajectory tracking task are due to the environmental noise and the model uncertainties under different driving scenarios. Considering the uncertainties in the environment, the reinforcement learning method with continuous action and noise-resistance capability could be a promising way to overcome these issues. In this article, a novel continuous interval type-2 fuzzy Q-learning (CIT2FQL) algorithm is proposed to deal with the trajectory tracking task. By introducing the n-dimensional interval type-2 fuzzy inference system (n-D IT2FIS) in fuzzy Q-learning, our proposed method achieves the continuous Q-learning by combining the action interpolation with IT2FIS for the first time. We also proposed a simplified type-reduction method for n-D IT2FIS to improve the computing efficiency of the proposed method. Moreover, a radial basis function (RBF) layer is chosen as the basis function to achieve the q-value interpolation. Finally, a trajectory tracking task in a simulation environment is conducted to verify the effectiveness and robustness of the proposed method under different scenarios. The results demonstrate that the proposed method has better robustness and noise-resistance capability while maintaining good tracking performance compared with the state-of-the-art baseline algorithms including double deep Q network (DDQN), proximal policy optimization (PPO), and interval type-2 dynamic fuzzy Q-learning (IT2DFQL).

AB - Trajectory tracking is a fundamental but challenging task for vehicle automation. In addition to the system nonlinearity, the main difficulties in the trajectory tracking task are due to the environmental noise and the model uncertainties under different driving scenarios. Considering the uncertainties in the environment, the reinforcement learning method with continuous action and noise-resistance capability could be a promising way to overcome these issues. In this article, a novel continuous interval type-2 fuzzy Q-learning (CIT2FQL) algorithm is proposed to deal with the trajectory tracking task. By introducing the n-dimensional interval type-2 fuzzy inference system (n-D IT2FIS) in fuzzy Q-learning, our proposed method achieves the continuous Q-learning by combining the action interpolation with IT2FIS for the first time. We also proposed a simplified type-reduction method for n-D IT2FIS to improve the computing efficiency of the proposed method. Moreover, a radial basis function (RBF) layer is chosen as the basis function to achieve the q-value interpolation. Finally, a trajectory tracking task in a simulation environment is conducted to verify the effectiveness and robustness of the proposed method under different scenarios. The results demonstrate that the proposed method has better robustness and noise-resistance capability while maintaining good tracking performance compared with the state-of-the-art baseline algorithms including double deep Q network (DDQN), proximal policy optimization (PPO), and interval type-2 dynamic fuzzy Q-learning (IT2DFQL).

UR - http://www.scopus.com/inward/record.url?scp=85124614942&partnerID=8YFLogxK

U2 - /doi.org/10.1002/rnc.6056

DO - /doi.org/10.1002/rnc.6056

M3 - Article

SN - 1049-8923

VL - 32

SP - 4788

EP - 4815

JO - INTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL

JF - INTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL

IS - 8

ER -