Identifying indicators of vulnerability from short speech segments using acoustic and textual features

Cui, Xia ORCID: https://orcid.org/0000-0002-1726-3814, Gamage, Amila, Hanley, Terry and Mu, Tingting (2021) Identifying indicators of vulnerability from short speech segments using acoustic and textual features. In: Interspeech 2021, 30 August 2021 - 03 September 2021, Brno, Czechia.

Preview

Published Version Download (176kB) | Preview

Abstract

In order to protect vulnerable people in telemarketing, organisations have to investigate the speech recordings to identify them first. Typically, the investigation is manually conducted. As such, the procedure is costly and time-consuming. With an automatic vulnerability detection system, more vulnerable people can be identified and protected. A standard telephone conversation lasts around 5 minutes, the detection system is expected to be able to identify such a potential vulnerable speaker from speech segments. Due to the complexity of the vulnerability definition and the unavailable annotated vulnerability examples, this paper attempts to address the detection problem as three classification tasks: age classification, accent classification and patient/non-patient classification utilising publicly available datasets. In the proposed system, we trained three sub models using acoustic and textual features for each sub task. Each trained model was evaluated on multiple datasets and achieved competitive results compared to a strong baseline (i.e. in-dataset accuracy).