Abstract
Knowledge engineering (KE) is the process of managing knowledge in a machine-readable way. This often takes the form of Knowledge Graphs (KGs). The advent of new technologies like Large Language Models (LLMs), besides enhancing automated processes in KG construction, has also changed KE work. We conducted a multiple-methods study exploring user opinions and needs regarding the use of LLMs in KE. We used ethnographic techniques to observe KE workers using LLMs to solve KE tasks during a hackathon, followed by interviews with some of the participants.
This interim study found that despite LLMs' promising capabilities for efficient knowledge acquisition and multimodality, their effective deployment requires an extended set of capabilities and training, particularly in prompting and understanding data. LLMs can be useful for simple quality assessment tasks, but in complex scenarios, the output cannot be controlled and evaluation may require novel approaches.
With this study, we aim to support with evidence the interaction of KE stakeholders with LLMs, identify areas of potential and understand the barriers to their effective use. Copilot approaches may be valuable in developing processes where the human or a team of humans is assisted by generative AI.
This interim study found that despite LLMs' promising capabilities for efficient knowledge acquisition and multimodality, their effective deployment requires an extended set of capabilities and training, particularly in prompting and understanding data. LLMs can be useful for simple quality assessment tasks, but in complex scenarios, the output cannot be controlled and evaluation may require novel approaches.
With this study, we aim to support with evidence the interaction of KE stakeholders with LLMs, identify areas of potential and understand the barriers to their effective use. Copilot approaches may be valuable in developing processes where the human or a team of humans is assisted by generative AI.
Original language | English |
---|---|
Title of host publication | CHIEA'24 Conference on Computer Human Interaction Extended Abstracts Proceedings |
Publisher | ACM |
Publication status | Accepted/In press - 21 Mar 2024 |