Annotation and Analysis of Recorded Piano Performances on the Web

Lawrence Fyfe; Daniel Bedoya; Elaine Chew

doi:10.17743/jaes.2022.0057

Annotation and Analysis of Recorded Piano Performances on the Web

Lawrence Fyfe^*, Daniel Bedoya, Elaine Chew

^*Corresponding author for this work

Sciences et Technologies de la Musique et du Sons (STMS) Laboratoire – Centre National de la Recherche Scientifique (CNRS), Institut de Recherche et Coordination Acoustique / Musique (IRCAM), Sorbonne Université, Ministère de la Culture

Research output: Contribution to journal › Article › peer-review

2 Citations (Scopus)

55 Downloads (Pure)

Abstract

Advancing knowledge and understanding about performed music is hampered by a lack of annotation data for music expressivity. To enable large-scale collection of annotations and explorations of performed music, the authors have created a workflow that is enabled by CosmoNote, a Web-based citizen science tool for annotating musical structures created by the performer and experienced by the listener during expressive piano performances. To enable annotation tasks with CosmoNote, annotators can listen to the recorded performances and view synchronized music visualization layers including the audio waveform, recorded notes, extracted audio features such as loudness and tempo, and score features such as harmonic tension. Annotators have the ability to zoom into specific parts of a performance and see visuals and listen to the audio from just that part. The annotation of performed musical structures is done by using boundaries of varying strengths, regions, comments, and note groups. By analyzing the annotations collected with CosmoNote, performance decisions will be able to be modeled and analyzed in order to aid in the understanding of expressive choices in musical performances and discover the vocabulary of performed musical structures.

Original language	English
Pages (from-to)	962-978
Journal	JAES Journal of the Audio Engineering Society
Volume	70
Issue number	11
DOIs	https://doi.org/10.17743/jaes.2022.0057
Publication status	Published - Nov 2022

Keywords

music representation
music annotation
citizen science
musical prosody
performance science

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.17743/jaes.2022.0057

22020Final published version, 1.45 MB

1 Active

COSMOS: COSMOS: Computational Shaping and Modeling of Musical Structures
Chew, E.
EC European Commission
1/07/2022 → 30/11/2025
Project: Research

2 Citations
4 Article
3 Chapter
3 Conference paper

Quantifying Karajan: Timing, Dynamics and Harmonic Tension
Chew, E., 1 Sept 2023, "Seine ganze Musikdeutung geht ja vom Klang aus". Musikalische Interpretation bei Herbert von Karajan. Aringer, K., Revers, P. & Wozonig, T. (eds.). Hildesheim: Olms, p. 25-41 17 p. 2
Research output: Chapter in Book/Report/Conference proceeding › Chapter › peer-review

File
A Perceiver-Centered Approach for Representing and Annotating Prosodic Functions in Performed Music
Bedoya, D., Fyfe, L. & Chew, E., 21 Jul 2022, In: Frontiers in Psychology. 13, 886570.
Research output: Contribution to journal › Article › peer-review

Open Access
COSMOS: Computational Shaping and Modeling of Musical Structures
Chew, E., 27 May 2022, In: Frontiers in Psychology. 13, 527539.
Research output: Contribution to journal › Article › peer-review

Open Access
1 Citation (Scopus)

6 Invited talk
1 Types of Public engagement and outreach - Public lecture/debate/seminar

Open Innovation in Science 2024: Debate: Zeroing in on the scientist: Navigating Dynamic Challengesin the Modern Research Landscape
Elaine Chew (Speaker)
22 May 2024
Activity: Talk or presentation › Invited talk
Journées d'Informatique Musicale (JIM24): Keynote: Music, Mathematics, and the Heart: A mellifluous mélange
Elaine Chew (Speaker)
6 May 2024
Activity: Talk or presentation › Invited talk
The International Conference on AI and Musical Creativity: Keynote: Performer-centered AI and Creativity
Elaine Chew (Speaker)
1 Sept 2023
Activity: Talk or presentation › Invited talk

European Research Council (ERC) Advanced Grant
Chew, Elaine (Recipient), Jun 2019
Prize: Fellowship awarded competitively

Cite this

@article{36d5ba65cfbf4ddbb7bcb1181926aa94,

title = "Annotation and Analysis of Recorded Piano Performances on the Web",

abstract = "Advancing knowledge and understanding about performed music is hampered by a lack of annotation data for music expressivity. To enable large-scale collection of annotations and explorations of performed music, the authors have created a workflow that is enabled by CosmoNote, a Web-based citizen science tool for annotating musical structures created by the performer and experienced by the listener during expressive piano performances. To enable annotation tasks with CosmoNote, annotators can listen to the recorded performances and view synchronized music visualization layers including the audio waveform, recorded notes, extracted audio features such as loudness and tempo, and score features such as harmonic tension. Annotators have the ability to zoom into specific parts of a performance and see visuals and listen to the audio from just that part. The annotation of performed musical structures is done by using boundaries of varying strengths, regions, comments, and note groups. By analyzing the annotations collected with CosmoNote, performance decisions will be able to be modeled and analyzed in order to aid in the understanding of expressive choices in musical performances and discover the vocabulary of performed musical structures.",

keywords = "music representation, music annotation, citizen science, musical prosody, performance science",

author = "Lawrence Fyfe and Daniel Bedoya and Elaine Chew",

year = "2022",

month = nov,

doi = "10.17743/jaes.2022.0057",

language = "English",

volume = "70",

pages = "962--978",

journal = "JAES Journal of the Audio Engineering Society",

issn = "1549-4950",

publisher = "Audio Engineering Society",

number = "11",

}

TY - JOUR

T1 - Annotation and Analysis of Recorded Piano Performances on the Web

AU - Fyfe, Lawrence

AU - Bedoya, Daniel

AU - Chew, Elaine

PY - 2022/11

Y1 - 2022/11

N2 - Advancing knowledge and understanding about performed music is hampered by a lack of annotation data for music expressivity. To enable large-scale collection of annotations and explorations of performed music, the authors have created a workflow that is enabled by CosmoNote, a Web-based citizen science tool for annotating musical structures created by the performer and experienced by the listener during expressive piano performances. To enable annotation tasks with CosmoNote, annotators can listen to the recorded performances and view synchronized music visualization layers including the audio waveform, recorded notes, extracted audio features such as loudness and tempo, and score features such as harmonic tension. Annotators have the ability to zoom into specific parts of a performance and see visuals and listen to the audio from just that part. The annotation of performed musical structures is done by using boundaries of varying strengths, regions, comments, and note groups. By analyzing the annotations collected with CosmoNote, performance decisions will be able to be modeled and analyzed in order to aid in the understanding of expressive choices in musical performances and discover the vocabulary of performed musical structures.

AB - Advancing knowledge and understanding about performed music is hampered by a lack of annotation data for music expressivity. To enable large-scale collection of annotations and explorations of performed music, the authors have created a workflow that is enabled by CosmoNote, a Web-based citizen science tool for annotating musical structures created by the performer and experienced by the listener during expressive piano performances. To enable annotation tasks with CosmoNote, annotators can listen to the recorded performances and view synchronized music visualization layers including the audio waveform, recorded notes, extracted audio features such as loudness and tempo, and score features such as harmonic tension. Annotators have the ability to zoom into specific parts of a performance and see visuals and listen to the audio from just that part. The annotation of performed musical structures is done by using boundaries of varying strengths, regions, comments, and note groups. By analyzing the annotations collected with CosmoNote, performance decisions will be able to be modeled and analyzed in order to aid in the understanding of expressive choices in musical performances and discover the vocabulary of performed musical structures.

KW - music representation

KW - music annotation

KW - citizen science

KW - musical prosody

KW - performance science

U2 - 10.17743/jaes.2022.0057

DO - 10.17743/jaes.2022.0057

M3 - Article

SN - 1549-4950

VL - 70

SP - 962

EP - 978

JO - JAES Journal of the Audio Engineering Society

JF - JAES Journal of the Audio Engineering Society

IS - 11

ER -

Annotation and Analysis of Recorded Piano Performances on the Web

Abstract

Keywords

UN SDGs

Access to Document

Fingerprint

Projects

Research output

Activities

Prizes

Cite this