Fluid Intake Action Detection Based on Egocentric Videos and YOLOv8 Models

TY - JOUR

T1 - Fluid Intake Action Detection Based on Egocentric Videos and YOLOv8 Models

AU - Chen, Xin

AU - Bao, Xinqi

AU - Kamavuako, Ernest

N1 - Publisher Copyright: © 2013 IEEE.

PY - 2025/3/6

Y1 - 2025/3/6

N2 - Dehydration in older adults poses significant health risks, requiring effective monitoring solutions. This study addresses the challenge of detecting fluid intake accurately using a first-person, vision-based approach with wearable cameras and advanced object detection models. We developed a comprehensive dataset comprising 17 hours of drinking footage (∼3100 events) and 15 hours of nondrinking activities (∼3600 events) recorded as interference, from 36 participants, collected between October 2022 and January 2023 at King's College London. We include various container types and daily activities to enhance the model's robustness and generalizability. YOLOv8 models were used to detect drinking-related objects, and a mechanism was developed to analyse the size and position of the detection output to identify hand-container interactions and movements. The models achieved mAP@50 over 0.97 and F1-score over 0.95 in detecting drinking-related objects. Action detection testing results from video streams demonstrated an F1-score of 0.917, which dropped to 0.863 when interference activities were added. Additionally, the model detected the start of drinking activities with an average latency of 0.24 seconds and the end with 0.04 seconds, indicating high temporal accuracy. These results demonstrate the feasibility of egocentric, vision-based fluidintake detection and its potential application in preventing dehydration. To our knowledge, this is the first vision-based dataset focusing on fluid-intake actions from a first-person viewpoint—offering a novel foundation for advancing hydration monitoring in older adults and various real-world contexts.

AB - Dehydration in older adults poses significant health risks, requiring effective monitoring solutions. This study addresses the challenge of detecting fluid intake accurately using a first-person, vision-based approach with wearable cameras and advanced object detection models. We developed a comprehensive dataset comprising 17 hours of drinking footage (∼3100 events) and 15 hours of nondrinking activities (∼3600 events) recorded as interference, from 36 participants, collected between October 2022 and January 2023 at King's College London. We include various container types and daily activities to enhance the model's robustness and generalizability. YOLOv8 models were used to detect drinking-related objects, and a mechanism was developed to analyse the size and position of the detection output to identify hand-container interactions and movements. The models achieved mAP@50 over 0.97 and F1-score over 0.95 in detecting drinking-related objects. Action detection testing results from video streams demonstrated an F1-score of 0.917, which dropped to 0.863 when interference activities were added. Additionally, the model detected the start of drinking activities with an average latency of 0.24 seconds and the end with 0.04 seconds, indicating high temporal accuracy. These results demonstrate the feasibility of egocentric, vision-based fluidintake detection and its potential application in preventing dehydration. To our knowledge, this is the first vision-based dataset focusing on fluid-intake actions from a first-person viewpoint—offering a novel foundation for advancing hydration monitoring in older adults and various real-world contexts.

KW - Intake monitoring

KW - dehydration

KW - fluid intake

KW - action detection

KW - wearable cameras

UR - http://www.scopus.com/inward/record.url?scp=86000518102&partnerID=8YFLogxK

U2 - 10.1109/JBHI.2025.3548512

DO - 10.1109/JBHI.2025.3548512

M3 - Article

SN - 2168-2194

VL - 29

SP - 4910

EP - 4919

JO - IEEE Journal of Biomedical and Health Informatics

JF - IEEE Journal of Biomedical and Health Informatics

IS - 7

ER -