Traditionally, speech analysis has always relied on a set of very informative features like (Mel) spectrogram, Mel Frequency Cepstral Coefficients (MFCC), pitch or intensity to build speech powered applications. Recently, deep learning-based models for the extraction of acoustic features have allowed significantly improving the state of the art in many speechrelated applications. With this work, we focus the analysis on the cross-lingual transferability of speech analysis features. The idea is to understand whether and how well a classification model trained on speech features in a source language works on an unseen target language. We evaluate these properties analysing models for Parkinson’s disease detection from speech, adapting the models from English to Telugu. Results show that multi-lingual pre-trained deep learning-based features do not require explicit adaptation and work well out-of-thebox. Differently, models not adapting out-of-the-box respond well even to unsupervised adaptation on a small data set.

Cross-lingual transferability of voice analysis models: a Parkinson’s Disease case study

C. Ferrante;V. Scotti
2023-01-01

Abstract

Traditionally, speech analysis has always relied on a set of very informative features like (Mel) spectrogram, Mel Frequency Cepstral Coefficients (MFCC), pitch or intensity to build speech powered applications. Recently, deep learning-based models for the extraction of acoustic features have allowed significantly improving the state of the art in many speechrelated applications. With this work, we focus the analysis on the cross-lingual transferability of speech analysis features. The idea is to understand whether and how well a classification model trained on speech features in a source language works on an unseen target language. We evaluate these properties analysing models for Parkinson’s disease detection from speech, adapting the models from English to Telugu. Results show that multi-lingual pre-trained deep learning-based features do not require explicit adaptation and work well out-of-thebox. Differently, models not adapting out-of-the-box respond well even to unsupervised adaptation on a small data set.
2023
Speech analysis, Deep learning, Parkison's disease detection, Domain adaptation, Cross-language
File in questo prodotto:
File Dimensione Formato  
output-4.pdf

accesso aperto

Descrizione: Extended abstract
: Post-Print (DRAFT o Author’s Accepted Manuscript-AAM)
Dimensione 105.06 kB
Formato Adobe PDF
105.06 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11311/1245957
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
  • OpenAlex ND
social impact