Updating Large Language Models’ Memories with Time Constraints

TY - CHAP

T1 - Updating Large Language Models’ Memories with Time Constraints

AU - Wu, Xin

AU - Bu, Yuqi

AU - Cai, Yi

AU - Wang, Tao

PY - 2024/10/1

Y1 - 2024/10/1

N2 - By incorporating the latest external knowledge, large language models (LLMs) can modify their internal memory. However, in practical applications, LLMs may encounter outdated in- formation, necessitating the filtering of such data and updating of knowledge beyond in- ternal memory. This paper explores whether LLMs can selectively update their memories based on the time constraints between internal memory and external knowledge. We evalu- ate existing LLMs using three types of data that exhibit different time constraints. Our ex- perimental results reveal the challenges most LLMs face with time-constrained knowledge and highlight the differences in how various LLMs handle such information. Additionally, to address the difficulties LLMs encounter in understanding time constraints, we propose a two-stage decoupling framework that separates the identification and computation of time con- straint into a symbolic system. Experimental re- sults demonstrate that the proposed framework yields an improvement of over 60% in Chat- GPT’s performance, and achieves a 12-24% enhancement in state-of-the-art LLM GPT-4.

AB - By incorporating the latest external knowledge, large language models (LLMs) can modify their internal memory. However, in practical applications, LLMs may encounter outdated in- formation, necessitating the filtering of such data and updating of knowledge beyond in- ternal memory. This paper explores whether LLMs can selectively update their memories based on the time constraints between internal memory and external knowledge. We evalu- ate existing LLMs using three types of data that exhibit different time constraints. Our ex- perimental results reveal the challenges most LLMs face with time-constrained knowledge and highlight the differences in how various LLMs handle such information. Additionally, to address the difficulties LLMs encounter in understanding time constraints, we propose a two-stage decoupling framework that separates the identification and computation of time con- straint into a symbolic system. Experimental re- sults demonstrate that the proposed framework yields an improvement of over 60% in Chat- GPT’s performance, and achieves a 12-24% enhancement in state-of-the-art LLM GPT-4.

M3 - Conference paper

BT - The 2024 Conference on Empirical Methods in Natural Language Processing

ER -