Principal investigator (ÚFAL): 

Peter Polák

Project Manager (ÚFAL): 

Hana Kubištová

Provider: 

START

Grant id: 

START/SCI/089

Duration: 

2021-2023

Tags: 

Machine Translation

Multilingual

Speech Recognition

People: 

Peter Polák

Ivana Kvapilíková

Dominik Macháček

At present, events like conferences, international negotiations or business meetings employ human interpreters to bridge the language barrier. Major events are simultaneously interpreted into the world languages, but national languages are often neglected for capacity reasons. Latest advances in artificial intelligence have shaped a promising technology - speech translation (aka Babel Fish from The Hitchhiker’s Guide to the Galaxy). Still, the technology suffers from insufficiencies in terms of its robustness against background noise, different speaker accents, disfluent speech or lack of language resources (e.g. unknown words or translations).

We aim to propose Babel Octopus: a robust speech translation system leveraging multiple parallel audio streams in different languages (e.g. an original source and an interpreter). We primarily focus on speech-to-text translation.

This goal will be achieved by: (1) using multiple concurrent speech sources (2) utilizing multilingual input to help with disambiguation and increase translation quality; (3) overcoming insufficient, imbalanced or domain-mismatched text and speech data by unsupervised training and data augmentation.

Publications

• Polák, P., & Bojar, O. (2021). Coarse-To-Fine And Cross-Lingual ASR Transfer. In Proceedings of the 21st Conference Information Technologies – Applications and Theory (ITAT 2021).
• Polák, P., Singh, M., & Bojar, O. (2021). Explainable Quality Estimation: CUNI Eval4NLP Submission. In Proceedings of the 2nd Workshop on Evaluation and Comparison of NLP Systems.