Integrating quantitative and qualitative reasoning for value alignment

Jazon Szabo; Jose Such; Natalia Criado Pacheco; Sanjay Modgil

Integrating quantitative and qualitative reasoning for value alignment

Jazon Szabo, Jose Such, Natalia Criado Pacheco, Sanjay Modgil

UPV Polytechnic University of Valencia

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

193 Downloads (Pure)

Abstract

Agents that focus only on achieving their own goals may cause significant harm to society. As a result, when deciding which actions to perform, agents have to consider societal values and how their actions impact these values - the `value alignment problem'. There is therefore a need to integrate quantitative machine reasoning with an ability to reason about the qualitative aspects of human values. In this paper, we present a novel framework for value-based reasoning that aims to bridge the gap between these two modes of reasoning. In particular, our framework extends the theory of grading to model how societal values can trade off with each other or with the agent's goals. Furthermore, our framework introduces the use of hyperreal numbers to represent both quantitative and qualitative aspects of reasoning and help address the value alignment problem.

Original language	English
Title of host publication	The 19th European Conference on Multi-Agent Systems
Publication status	Accepted/In press - 25 Jul 2022

Access to Document

EUMAS_paper_30Accepted author manuscript, 265 KB

Cite this

@inbook{e86e54ac817a4646966d851b5e5ca995,

title = "Integrating quantitative and qualitative reasoning for value alignment",

abstract = "Agents that focus only on achieving their own goals may cause significant harm to society. As a result, when deciding which actions to perform, agents have to consider societal values and how their actions impact these values - the `value alignment problem'. There is therefore a need to integrate quantitative machine reasoning with an ability to reason about the qualitative aspects of human values. In this paper, we present a novel framework for value-based reasoning that aims to bridge the gap between these two modes of reasoning. In particular, our framework extends the theory of grading to model how societal values can trade off with each other or with the agent's goals. Furthermore, our framework introduces the use of hyperreal numbers to represent both quantitative and qualitative aspects of reasoning and help address the value alignment problem.",

author = "Jazon Szabo and Jose Such and {Criado Pacheco}, Natalia and Sanjay Modgil",

year = "2022",

month = jul,

day = "25",

language = "English",

booktitle = "The 19th European Conference on Multi-Agent Systems",

}

TY - CHAP

T1 - Integrating quantitative and qualitative reasoning for value alignment

AU - Szabo, Jazon

AU - Such, Jose

AU - Criado Pacheco, Natalia

AU - Modgil, Sanjay

PY - 2022/7/25

Y1 - 2022/7/25

N2 - Agents that focus only on achieving their own goals may cause significant harm to society. As a result, when deciding which actions to perform, agents have to consider societal values and how their actions impact these values - the `value alignment problem'. There is therefore a need to integrate quantitative machine reasoning with an ability to reason about the qualitative aspects of human values. In this paper, we present a novel framework for value-based reasoning that aims to bridge the gap between these two modes of reasoning. In particular, our framework extends the theory of grading to model how societal values can trade off with each other or with the agent's goals. Furthermore, our framework introduces the use of hyperreal numbers to represent both quantitative and qualitative aspects of reasoning and help address the value alignment problem.

AB - Agents that focus only on achieving their own goals may cause significant harm to society. As a result, when deciding which actions to perform, agents have to consider societal values and how their actions impact these values - the `value alignment problem'. There is therefore a need to integrate quantitative machine reasoning with an ability to reason about the qualitative aspects of human values. In this paper, we present a novel framework for value-based reasoning that aims to bridge the gap between these two modes of reasoning. In particular, our framework extends the theory of grading to model how societal values can trade off with each other or with the agent's goals. Furthermore, our framework introduces the use of hyperreal numbers to represent both quantitative and qualitative aspects of reasoning and help address the value alignment problem.

M3 - Conference paper

BT - The 19th European Conference on Multi-Agent Systems

ER -

Integrating quantitative and qualitative reasoning for value alignment

Abstract

Access to Document

Fingerprint

Cite this