MULTILINGUAL LYRICS-TO-AUDIO ALIGNMENT - Equipe Signal, Statistique et Apprentissage Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

MULTILINGUAL LYRICS-TO-AUDIO ALIGNMENT

Résumé

Lyrics-to-audio alignment methods have recently reported impressive results, opening the door to practical applications such as karaoke and within song navigation. However , most studies focus on a single language-usually En-glish-for which annotated data are abundant. The question of their ability to generalize to other languages, especially in low (or even zero) training resource scenarios has been so far left unexplored. In this paper, we address the lyrics-to-audio alignment task in a generalized multilingual setup. More precisely, this investigation presents the first (to the best of our knowledge) attempt to create a language-independent lyrics-to-audio alignment system. Building on a Recurrent Neural Network (RNN) model trained with a Connectionist Temporal Classification (CTC) algorithm, we study the relevance of different intermediate representations, either character or phoneme, along with several strategies to design a training set. The evaluation is conducted on multiple languages with a varying amount of data available, from plenty to zero. Results show that learning from diverse data and using a universal phoneme set as an intermediate representation yield the best generalization performances.
Fichier principal
Vignette du fichier
101.pdf (275.64 Ko) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte
Loading...

Dates et versions

hal-02996940 , version 1 (09-11-2020)

Licence

Paternité

Identifiants

  • HAL Id : hal-02996940 , version 1

Citer

Andrea Vaglio, Romain Hennequin, Manuel Moussallam, Gael Richard, Florence d'Alché-Buc. MULTILINGUAL LYRICS-TO-AUDIO ALIGNMENT. International Society for Music Information Retrieval Conference (ISMIR), Oct 2020, Montreal, Canada. ⟨hal-02996940⟩
470 Consultations
482 Téléchargements

Partager

Gmail Facebook X LinkedIn More