The Promise and Challenge of Large Language Models for Knowledge Engineering: Insights from a Hackathon

Research output: Chapter in Book/Report/Conference proceedingConference paperpeer-review

Abstract

Knowledge engineering (KE) is the process of managing knowledge in a machine-readable way. This often takes the form of Knowledge Graphs (KGs). The advent of new technologies like Large Language Models (LLMs), besides enhancing automated processes in KG construction, has also changed KE work. We conducted a multiple-methods study exploring user opinions and needs regarding the use of LLMs in KE. We used ethnographic techniques to observe KE workers using LLMs to solve KE tasks during a hackathon, followed by interviews with some of the participants.
This interim study found that despite LLMs' promising capabilities for efficient knowledge acquisition and multimodality, their effective deployment requires an extended set of capabilities and training, particularly in prompting and understanding data. LLMs can be useful for simple quality assessment tasks, but in complex scenarios, the output cannot be controlled and evaluation may require novel approaches.
With this study, we aim to support with evidence the interaction of KE stakeholders with LLMs, identify areas of potential and understand the barriers to their effective use. Copilot approaches may be valuable in developing processes where the human or a team of humans is assisted by generative AI.
Original languageEnglish
Title of host publicationCHIEA'24 Conference on Computer Human Interaction Extended Abstracts Proceedings
PublisherACM
Publication statusAccepted/In press - 21 Mar 2024

Fingerprint

Dive into the research topics of 'The Promise and Challenge of Large Language Models for Knowledge Engineering: Insights from a Hackathon'. Together they form a unique fingerprint.

Cite this