Task-oriented and Semantics-aware Communications for Augmented Reality

TY - CHAP

T1 - Task-oriented and Semantics-aware Communications for Augmented Reality

AU - Wang, Zhe

AU - Deng, Yansha

N1 - Funding Information: This work was supported in part by UKRI under the UK government’s Horizon Europe funding guarantee (grant number 10061781), as part of the European Commission-funded collaborative project VERGE, under SNS JU program (grant number 101096034). This work is also a contribution by Project REASON, a UK Government funded project under the FONRC sponsored by the DSIT. Publisher Copyright: © 2023 IEEE.

PY - 2023

Y1 - 2023

N2 - Upon the advent of the emerging metaverse and its related applications in Augmented Reality (AR), the current bit-oriented network struggles to support real-time changes for the vast amount of associated information, creating a significant bottleneck in its development. To address the above problem, we present a novel task-oriented and semantics-aware communication framework for augmented reality (TSAR) to enhance communication efficiency and effectiveness significantly. We first present an analysis of traditional wireless AR point cloud communication framework, followed by a detailed summary of our proposed semantic information extraction within the end-to-end communication. Then, we detail the components of the TSAR framework, incorporating semantics extraction with deep learning, task-oriented base knowledge selection, and avatar pose recovery. Through rigorous experimentation, we demonstrate that our proposed TSAR framework considerably outperforms traditional point cloud communication framework, reducing wireless AR application transmission latency by 95.6% and improving communication effectiveness in geometry and color aspects by up to 82.4% and 20.4%, respectively.

AB - Upon the advent of the emerging metaverse and its related applications in Augmented Reality (AR), the current bit-oriented network struggles to support real-time changes for the vast amount of associated information, creating a significant bottleneck in its development. To address the above problem, we present a novel task-oriented and semantics-aware communication framework for augmented reality (TSAR) to enhance communication efficiency and effectiveness significantly. We first present an analysis of traditional wireless AR point cloud communication framework, followed by a detailed summary of our proposed semantic information extraction within the end-to-end communication. Then, we detail the components of the TSAR framework, incorporating semantics extraction with deep learning, task-oriented base knowledge selection, and avatar pose recovery. Through rigorous experimentation, we demonstrate that our proposed TSAR framework considerably outperforms traditional point cloud communication framework, reducing wireless AR application transmission latency by 95.6% and improving communication effectiveness in geometry and color aspects by up to 82.4% and 20.4%, respectively.

KW - augmented reality

KW - Metaverse

KW - point cloud

KW - semantic communication

KW - semantics extraction

UR - http://www.scopus.com/inward/record.url?scp=85187351141&partnerID=8YFLogxK

U2 - 10.1109/GLOBECOM54140.2023.10437075

DO - 10.1109/GLOBECOM54140.2023.10437075

M3 - Conference paper

AN - SCOPUS:85187351141

T3 - Proceedings - IEEE Global Communications Conference, GLOBECOM

SP - 2215

EP - 2220

BT - GLOBECOM 2023 - 2023 IEEE Global Communications Conference

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2023 IEEE Global Communications Conference, GLOBECOM 2023

Y2 - 4 December 2023 through 8 December 2023

ER -