Towards Robust Waveform-Based Acoustic Models

Dino Oglic, Zoran Cvetkovic, Peter Sollich, Steve Renals, Bin Yu

Research output: Contribution to journalArticlepeer-review

45 Downloads (Pure)


We study the problem of learning robust acoustic models in adverse environments, characterized by a significant mismatch between training and test conditions. This problem is of paramount importance for the deployment of speech recognition systems that need to perform well in unseen environments. First, we characterize data augmentation theoretically as an instance of vicinal risk minimization, which aims at improving risk estimates during training by replacing the delta functions that define the empirical density over the input space with an approximation of the marginal population density in the vicinity of the training samples. More specifically, we assume that local neighborhoods centered at training samples can be approximated using a mixture of Gaussians, and demonstrate theoretically that this can incorporate robust inductive bias into the learning process. We then specify the individual mixture components implicitly via data augmentation schemes, designed to address common sources of spurious correlations in acoustic models. To avoid potential confounding effects on robustness due to information loss, which has been associated with standard feature extraction techniques (e.g., FBANK and MFCC features), we focus on the waveform- based setting. Our empirical results show that the approach can generalize to unseen noise conditions, with 150% relative improve- ment in out-of-distribution generalization compared to training using the standard risk minimization principle. Moreover, the results demonstrate competitive performance relative to models learned using a training sample designed to match the acoustic conditions characteristic of test utterances.
Original languageEnglish
Pages (from-to)1977 - 1992
Number of pages16
JournalIEEE/ACM Transactions on Audio, Speech, and Language Processing
Publication statusPublished - 5 May 2022


Dive into the research topics of 'Towards Robust Waveform-Based Acoustic Models'. Together they form a unique fingerprint.

Cite this