TY - JOUR
T1 - Prompting meaning
T2 - a hermeneutic approach to optimising prompt engineering with ChatGPT
AU - Henrickson, Leah
AU - Meroño-Peñuela, Albert
N1 - Publisher Copyright:
© 2023, The Author(s).
PY - 2023
Y1 - 2023
N2 - Recent advances in natural language generation (NLG), such as public accessibility to ChatGPT, have sparked polarised debates about the societal impact of this technology. Popular discourse tends towards either overoptimistic hype that touts the radically transformative potentials of these systems or pessimistic critique of their technical limitations and general ‘stupidity’. Surprisingly, these debates have largely overlooked the exegetical capacities of these systems, which for many users seem to be producing meaningful texts. In this paper, we take an interdisciplinary approach that combines hermeneutics—the study of meaning and interpretation—with prompt engineering—task descriptions embedded in input to NLG systems—to study the extent to which a specific NLG system, ChatGPT, produces texts of hermeneutic value. We design prompts with the goal of optimising hermeneuticity rather than mere factual accuracy, and apply them in four different use cases combining humans and ChatGPT as readers and writers. In most cases, ChatGPT produces readable texts that respond clearly to our requests. However, increasing the specificity of prompts’ task descriptions leads to texts with intensified neutrality, indicating that ChatGPT’s optimisation for factual accuracy may actually be detrimental to the hermeneuticity of its output.
AB - Recent advances in natural language generation (NLG), such as public accessibility to ChatGPT, have sparked polarised debates about the societal impact of this technology. Popular discourse tends towards either overoptimistic hype that touts the radically transformative potentials of these systems or pessimistic critique of their technical limitations and general ‘stupidity’. Surprisingly, these debates have largely overlooked the exegetical capacities of these systems, which for many users seem to be producing meaningful texts. In this paper, we take an interdisciplinary approach that combines hermeneutics—the study of meaning and interpretation—with prompt engineering—task descriptions embedded in input to NLG systems—to study the extent to which a specific NLG system, ChatGPT, produces texts of hermeneutic value. We design prompts with the goal of optimising hermeneuticity rather than mere factual accuracy, and apply them in four different use cases combining humans and ChatGPT as readers and writers. In most cases, ChatGPT produces readable texts that respond clearly to our requests. However, increasing the specificity of prompts’ task descriptions leads to texts with intensified neutrality, indicating that ChatGPT’s optimisation for factual accuracy may actually be detrimental to the hermeneuticity of its output.
KW - ChatGPT
KW - Hermeneutics
KW - Large language models
KW - Natural language generation
KW - Natural language processing
KW - Prompt engineering
UR - http://www.scopus.com/inward/record.url?scp=85169786535&partnerID=8YFLogxK
U2 - 10.1007/s00146-023-01752-8
DO - 10.1007/s00146-023-01752-8
M3 - Article
AN - SCOPUS:85169786535
SN - 0951-5666
JO - AI and Society
JF - AI and Society
ER -