Mapping Inter-cardiovascular Time-frequency Coherence to Harmonic Tension in Sonification of Ensemble Interaction Between a Covid-19 Patient and the Medical Team

TY - CHAP

T1 - Mapping Inter-cardiovascular Time-frequency Coherence to Harmonic Tension in Sonification of Ensemble Interaction Between a Covid-19 Patient and the Medical Team

AU - Frid, Emma

AU - Orini, Michele

AU - Martinelli, Giampaolo

AU - Chew, Elaine

PY - 2021/6/25

Y1 - 2021/6/25

N2 - This paper presents exploratory work on sonic and visual representations of heartbeats of a COVID-19 patient and a medical team. The aim of this work is to sonify heart signals to reflect how a medical team comes together during a COVID-19 treatment, i.e. to highlight other aspects of the COVID-19 pandemic than those usually portrayed through sonification, which often focuses on the number of cases. The proposed framework highlights synergies between sound and heart signals through mapping between timefrequency coherence (TFC) of heart signals and harmonic tension and dissonance in music. Results from a listening experiment suggested that the proposed mapping between TFC and harmonic tension was successful in terms of communicating low versus high coherence between heart signals, with an overall accuracy of 69%, which was significantly higher than chance. In the light of the performed work, we discuss how links between heart-and sound signals can be further explored through sonification to promote understanding of aspects related to cardiovascular health.

AB - This paper presents exploratory work on sonic and visual representations of heartbeats of a COVID-19 patient and a medical team. The aim of this work is to sonify heart signals to reflect how a medical team comes together during a COVID-19 treatment, i.e. to highlight other aspects of the COVID-19 pandemic than those usually portrayed through sonification, which often focuses on the number of cases. The proposed framework highlights synergies between sound and heart signals through mapping between timefrequency coherence (TFC) of heart signals and harmonic tension and dissonance in music. Results from a listening experiment suggested that the proposed mapping between TFC and harmonic tension was successful in terms of communicating low versus high coherence between heart signals, with an overall accuracy of 69%, which was significantly higher than chance. In the light of the performed work, we discuss how links between heart-and sound signals can be further explored through sonification to promote understanding of aspects related to cardiovascular health.

KW - time-frequency coherence

KW - covid 19

KW - harmonic tension

KW - data sonification

KW - electrocardiography

M3 - Conference paper

BT - The 26th International Conference on Auditory Display (ICAD 2021)

ER -