Adaptive Scattering Transforms for Playing Technique Recognition

Changhong Wang; Emmanouil Benetos; Vincent Lostanlen; Elaine Chew

doi:10.1109/TASLP.2022.3156785

Adaptive Scattering Transforms for Playing Technique Recognition

Changhong Wang^*, Emmanouil Benetos, Vincent Lostanlen, Elaine Chew

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

9 Citations (Scopus)

108 Downloads (Pure)

Abstract

Playing techniques contain distinctive information about musical expressivity and interpretation. Yet, current research in music signal analysis suffers from a scarcity of computational models for playing techniques, especially in the context of live performance. To address this problem, our paper develops a general framework for playing technique recognition. We propose the adaptive scattering transform, which refers to any scattering transform that includes a stage of data-driven dimensionality reduction over at least one of its wavelet variables, for representing playing techniques. Two adaptive scattering features are presented: frequency-adaptive scattering and direction-adaptive scattering. We analyse seven playing techniques: vibrato, tremolo, trill, flutter-tongue, acciaccatura, portamento, and glissando. To evaluate the proposed methodology, we create a new dataset containing full-length Chinese bamboo flute performances (CBFdataset) with expert playing technique annotations. Once trained on the proposed scattering representations, a support vector classifier achieves state-of-the-art results. We provide explanatory visualisations of scattering coefficients for each technique and verify the system over three additional datasets with various instrumental and vocal techniques: VPset, SOL, and VocalSet.

Original language	English
Pages (from-to)	1407-1421
Number of pages	15
Journal	IEEE/ACM Transactions on Audio, Speech, and Language Processing
Volume	30
DOIs	https://doi.org/10.1109/TASLP.2022.3156785
Publication status	Published - 7 Mar 2022

Keywords

Scattering
Wavelet transforms
Instruments
Time-frequency analysis
Multiple signal classification
Music
Speech processing
Music performance analysis
music signal analysis
scattering transform

Access to Document

10.1109/TASLP.2022.3156785

TASLP Playing Technique Scattering paper - accepted w copyright notice
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Accepted author manuscript, 3.25 MBLicence: Other
TASLP Playing Technique Scattering supplementary material - accepted w copyright notice
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Accepted author manuscript, 1.34 MBLicence: Other

9 Citations
1 Chapter
1 Conference paper
1 Article

Characterizing and Interpreting Music Expressivity through Rhythm and Loudness Simplices
Lascabettes, P., Chew, E. & Bloch, I., Oct 2023, Proceedings of the International Computer Music Conference. Shenzhen, China, Vol. 47. 8 p.
Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

Open Access
File
Quantifying Karajan: Timing, Dynamics and Harmonic Tension
Chew, E., 1 Sept 2023, "Seine ganze Musikdeutung geht ja vom Klang aus". Musikalische Interpretation bei Herbert von Karajan. Aringer, K., Revers, P. & Wozonig, T. (eds.). Hildesheim: Olms, p. 25-41 17 p. 2
Research output: Chapter in Book/Report/Conference proceeding › Chapter › peer-review

File
A Perceiver-Centered Approach for Representing and Annotating Prosodic Functions in Performed Music
Bedoya, D., Fyfe, L. & Chew, E., 21 Jul 2022, In: Frontiers in Psychology. 13, 886570.
Research output: Contribution to journal › Article › peer-review

Open Access

Cite this

@article{8cd9661b98a545368e8ee30214314e30,

title = "Adaptive Scattering Transforms for Playing Technique Recognition",

abstract = "Playing techniques contain distinctive information about musical expressivity and interpretation. Yet, current research in music signal analysis suffers from a scarcity of computational models for playing techniques, especially in the context of live performance. To address this problem, our paper develops a general framework for playing technique recognition. We propose the adaptive scattering transform, which refers to any scattering transform that includes a stage of data-driven dimensionality reduction over at least one of its wavelet variables, for representing playing techniques. Two adaptive scattering features are presented: frequency-adaptive scattering and direction-adaptive scattering. We analyse seven playing techniques: vibrato, tremolo, trill, flutter-tongue, acciaccatura, portamento, and glissando. To evaluate the proposed methodology, we create a new dataset containing full-length Chinese bamboo flute performances (CBFdataset) with expert playing technique annotations. Once trained on the proposed scattering representations, a support vector classifier achieves state-of-the-art results. We provide explanatory visualisations of scattering coefficients for each technique and verify the system over three additional datasets with various instrumental and vocal techniques: VPset, SOL, and VocalSet.",

keywords = "Scattering, Wavelet transforms, Instruments, Time-frequency analysis, Multiple signal classification, Music, Speech processing, Music performance analysis, music signal analysis, scattering transform",

author = "Changhong Wang and Emmanouil Benetos and Vincent Lostanlen and Elaine Chew",

year = "2022",

month = mar,

day = "7",

doi = "10.1109/TASLP.2022.3156785",

language = "English",

volume = "30",

pages = "1407--1421",

journal = "IEEE/ACM Transactions on Audio, Speech, and Language Processing ",

issn = "2329-9290",

publisher = "IEEE Advancing Technology for Humanity",

}

TY - JOUR

T1 - Adaptive Scattering Transforms for Playing Technique Recognition

AU - Wang, Changhong

AU - Benetos, Emmanouil

AU - Lostanlen, Vincent

AU - Chew, Elaine

PY - 2022/3/7

Y1 - 2022/3/7

N2 - Playing techniques contain distinctive information about musical expressivity and interpretation. Yet, current research in music signal analysis suffers from a scarcity of computational models for playing techniques, especially in the context of live performance. To address this problem, our paper develops a general framework for playing technique recognition. We propose the adaptive scattering transform, which refers to any scattering transform that includes a stage of data-driven dimensionality reduction over at least one of its wavelet variables, for representing playing techniques. Two adaptive scattering features are presented: frequency-adaptive scattering and direction-adaptive scattering. We analyse seven playing techniques: vibrato, tremolo, trill, flutter-tongue, acciaccatura, portamento, and glissando. To evaluate the proposed methodology, we create a new dataset containing full-length Chinese bamboo flute performances (CBFdataset) with expert playing technique annotations. Once trained on the proposed scattering representations, a support vector classifier achieves state-of-the-art results. We provide explanatory visualisations of scattering coefficients for each technique and verify the system over three additional datasets with various instrumental and vocal techniques: VPset, SOL, and VocalSet.

AB - Playing techniques contain distinctive information about musical expressivity and interpretation. Yet, current research in music signal analysis suffers from a scarcity of computational models for playing techniques, especially in the context of live performance. To address this problem, our paper develops a general framework for playing technique recognition. We propose the adaptive scattering transform, which refers to any scattering transform that includes a stage of data-driven dimensionality reduction over at least one of its wavelet variables, for representing playing techniques. Two adaptive scattering features are presented: frequency-adaptive scattering and direction-adaptive scattering. We analyse seven playing techniques: vibrato, tremolo, trill, flutter-tongue, acciaccatura, portamento, and glissando. To evaluate the proposed methodology, we create a new dataset containing full-length Chinese bamboo flute performances (CBFdataset) with expert playing technique annotations. Once trained on the proposed scattering representations, a support vector classifier achieves state-of-the-art results. We provide explanatory visualisations of scattering coefficients for each technique and verify the system over three additional datasets with various instrumental and vocal techniques: VPset, SOL, and VocalSet.

KW - Scattering

KW - Wavelet transforms

KW - Instruments

KW - Time-frequency analysis

KW - Multiple signal classification

KW - Music

KW - Speech processing

KW - Music performance analysis

KW - music signal analysis

KW - scattering transform

U2 - 10.1109/TASLP.2022.3156785

DO - 10.1109/TASLP.2022.3156785

M3 - Article

SN - 2329-9290

VL - 30

SP - 1407

EP - 1421

JO - IEEE/ACM Transactions on Audio, Speech, and Language Processing

JF - IEEE/ACM Transactions on Audio, Speech, and Language Processing

ER -

Adaptive Scattering Transforms for Playing Technique Recognition

Abstract

Keywords

Access to Document

Fingerprint

Research output

Characterizing and Interpreting Music Expressivity through Rhythm and Loudness Simplices

Quantifying Karajan: Timing, Dynamics and Harmonic Tension

A Perceiver-Centered Approach for Representing and Annotating Prosodic Functions in Performed Music

Cite this