Q-Learning & Economic NL-MPC for Continuous Biomass Fermentation

Tom Vinestock; Hak-Keung Lam; Mark Taylor; Miao Guo

doi:10.1016/B978-0-443-28824-1.50302-1

Q-Learning & Economic NL-MPC for Continuous Biomass Fermentation

Tom Vinestock, Hak-Keung Lam, Mark Taylor, Miao Guo^*

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Chapter › peer-review

Abstract

An economic optimal control problem for biomass fermentation is presented. Building on this, this work contributes a performance comparison between Q-Learning and non-linear model predictive control (MPC) as applied to a simulated biomass fermentation system. Two different scenarios are considered. In the first scenario, it is assumed an accurate model is available. In the second scenario, it is assumed there is parametric mismatch between the model and the true plant. In the second scenario, a simple transfer learning approach is used to improve the performance of the Q-Learning controller, while parameter estimation is used to aid the MPC. Trajectories and performance indicators are presented for both controllers and for both scenarios. It is found that Q-Learning out-performs MPC in the first scenario. In the second scenario, transfer learning is found to significantly improve the performance of the Q-Learning, and to outperform a comparison controller combining MPC with moving horizon estimation (MHE).

Original language	English
Title of host publication	Computer Aided Chemical Engineering
Pages	1807-1812
Number of pages	6
Volume	53
DOIs	https://doi.org/10.1016/B978-0-443-28824-1.50302-1
Publication status	Published - 26 Jun 2024

Publication series

Name	Computer Aided Chemical Engineering
Volume	53
ISSN (Print)	1570-7946

Access to Document

10.1016/B978-0-443-28824-1.50302-1Licence: Unspecified

Cite this

@inbook{7463c1bcc4254c4fb9616cc5a3304d67,

title = "Q-Learning & Economic NL-MPC for Continuous Biomass Fermentation",

abstract = "An economic optimal control problem for biomass fermentation is presented. Building on this, this work contributes a performance comparison between Q-Learning and non-linear model predictive control (MPC) as applied to a simulated biomass fermentation system. Two different scenarios are considered. In the first scenario, it is assumed an accurate model is available. In the second scenario, it is assumed there is parametric mismatch between the model and the true plant. In the second scenario, a simple transfer learning approach is used to improve the performance of the Q-Learning controller, while parameter estimation is used to aid the MPC. Trajectories and performance indicators are presented for both controllers and for both scenarios. It is found that Q-Learning out-performs MPC in the first scenario. In the second scenario, transfer learning is found to significantly improve the performance of the Q-Learning, and to outperform a comparison controller combining MPC with moving horizon estimation (MHE).",

author = "Tom Vinestock and Hak-Keung Lam and Mark Taylor and Miao Guo",

note = "Publisher Copyright: {\textcopyright} 2024 Elsevier B.V.",

year = "2024",

month = jun,

day = "26",

doi = "10.1016/B978-0-443-28824-1.50302-1",

language = "English",

volume = "53",

series = "Computer Aided Chemical Engineering",

pages = "1807--1812",

booktitle = "Computer Aided Chemical Engineering",

}

TY - CHAP

T1 - Q-Learning & Economic NL-MPC for Continuous Biomass Fermentation

AU - Vinestock, Tom

AU - Lam, Hak-Keung

AU - Taylor, Mark

AU - Guo, Miao

PY - 2024/6/26

Y1 - 2024/6/26

N2 - An economic optimal control problem for biomass fermentation is presented. Building on this, this work contributes a performance comparison between Q-Learning and non-linear model predictive control (MPC) as applied to a simulated biomass fermentation system. Two different scenarios are considered. In the first scenario, it is assumed an accurate model is available. In the second scenario, it is assumed there is parametric mismatch between the model and the true plant. In the second scenario, a simple transfer learning approach is used to improve the performance of the Q-Learning controller, while parameter estimation is used to aid the MPC. Trajectories and performance indicators are presented for both controllers and for both scenarios. It is found that Q-Learning out-performs MPC in the first scenario. In the second scenario, transfer learning is found to significantly improve the performance of the Q-Learning, and to outperform a comparison controller combining MPC with moving horizon estimation (MHE).

AB - An economic optimal control problem for biomass fermentation is presented. Building on this, this work contributes a performance comparison between Q-Learning and non-linear model predictive control (MPC) as applied to a simulated biomass fermentation system. Two different scenarios are considered. In the first scenario, it is assumed an accurate model is available. In the second scenario, it is assumed there is parametric mismatch between the model and the true plant. In the second scenario, a simple transfer learning approach is used to improve the performance of the Q-Learning controller, while parameter estimation is used to aid the MPC. Trajectories and performance indicators are presented for both controllers and for both scenarios. It is found that Q-Learning out-performs MPC in the first scenario. In the second scenario, transfer learning is found to significantly improve the performance of the Q-Learning, and to outperform a comparison controller combining MPC with moving horizon estimation (MHE).

UR - http://www.scopus.com/inward/record.url?scp=85196782948&partnerID=8YFLogxK

U2 - 10.1016/B978-0-443-28824-1.50302-1

DO - 10.1016/B978-0-443-28824-1.50302-1

M3 - Chapter

VL - 53

T3 - Computer Aided Chemical Engineering

SP - 1807

EP - 1812

BT - Computer Aided Chemical Engineering

ER -

Q-Learning & Economic NL-MPC for Continuous Biomass Fermentation

Abstract

Publication series

Access to Document

Other files and links

Fingerprint

Cite this