Exploiting Model Errors for Exploration in Model-Based Reinforcement Learning

Jared Swift; Matteo Leonetti

Exploiting Model Errors for Exploration in Model-Based Reinforcement Learning

Jared Swift^*
, Matteo Leonetti

^*Corresponding author for this work

Informatics

Research output: Other contribution

11 Downloads (Pure)

Abstract

We address the problem of exploration in model-based reinforcement learning (MBRL). We present Model-Corrective eXploration (MCX) a novel approach to exploration in MBRL that is both agnostic to the model representation and scalable to complex environments. MCX learns to generalise model prediction errors in order to make hypotheses about how the model might else be wrong, and uses such hypotheses for performing planning to facilitate exploration. We demonstrate the efficacy of our method in visual control tasks with the state-of-the-art MBRL algorithm, DreamerV3.

Original language	English
Publication status	Accepted/In press - 2025

Keywords

Reinforcement Learning
Exploration
Model-based Reinforcement Learning

Access to Document

Exploiting Model Errors for Exploration in Model-Based Reinforcement LearningAccepted author manuscript, 466 KB

Cite this

@misc{a81083ee556e4f6892718c01346e39fd,

title = "Exploiting Model Errors for Exploration in Model-Based Reinforcement Learning",

abstract = "We address the problem of exploration in model-based reinforcement learning (MBRL). We present Model-Corrective eXploration (MCX) a novel approach to exploration in MBRL that is both agnostic to the model representation and scalable to complex environments. MCX learns to generalise model prediction errors in order to make hypotheses about how the model might else be wrong, and uses such hypotheses for performing planning to facilitate exploration. We demonstrate the efficacy of our method in visual control tasks with the state-of-the-art MBRL algorithm, DreamerV3.",

keywords = "Reinforcement Learning, Exploration, Model-based Reinforcement Learning",

author = "Jared Swift and Matteo Leonetti",

note = "18th European Workshop on Reinforcement Learning (EWRL 2025)",

year = "2025",

language = "English",

type = "Other",

}

TY - GEN

T1 - Exploiting Model Errors for Exploration in Model-Based Reinforcement Learning

AU - Swift, Jared

AU - Leonetti, Matteo

N1 - 18th European Workshop on Reinforcement Learning (EWRL 2025)

PY - 2025

Y1 - 2025

N2 - We address the problem of exploration in model-based reinforcement learning (MBRL). We present Model-Corrective eXploration (MCX) a novel approach to exploration in MBRL that is both agnostic to the model representation and scalable to complex environments. MCX learns to generalise model prediction errors in order to make hypotheses about how the model might else be wrong, and uses such hypotheses for performing planning to facilitate exploration. We demonstrate the efficacy of our method in visual control tasks with the state-of-the-art MBRL algorithm, DreamerV3.

AB - We address the problem of exploration in model-based reinforcement learning (MBRL). We present Model-Corrective eXploration (MCX) a novel approach to exploration in MBRL that is both agnostic to the model representation and scalable to complex environments. MCX learns to generalise model prediction errors in order to make hypotheses about how the model might else be wrong, and uses such hypotheses for performing planning to facilitate exploration. We demonstrate the efficacy of our method in visual control tasks with the state-of-the-art MBRL algorithm, DreamerV3.

KW - Reinforcement Learning

KW - Exploration

KW - Model-based Reinforcement Learning

M3 - Other contribution

ER -

Exploiting Model Errors for Exploration in Model-Based Reinforcement Learning

Abstract

Keywords

Access to Document

Fingerprint

Cite this