On-line Spoken Language Translation

Guidelines

Machine translation (MT) of text input is generally processed at the level of sentences. This approach makes the application of MT to speech difficult because individual sentences would first have to be identified. Moreover, spontaneous speech often does not lend itself to segmentation into sentences. Techniques of speech normalization or speech reconstructions have been studied in the past to convert the output of automatic speech recognition systems into as fluent individual sentences as possible.

In contrast to such approaches, the goal of the thesis is to propose and evaluate adaptations of training data and regimes of neural machine translation (NMT) models in order support translation of disfluent speech. The main criterion of the performance would be the adequacy of the translation. In terms of fluency, the output is expected to be equally fluent or disfluent as the input was.

The proposed techniques must not neglect context but they must be easily applicable for on-line speech translation, i.e. continuous translation of spontaneous speech.

References

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is All you Need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 6000–6010. Curran Associates, Inc., 2017.

Pusateri, E., Ambati, B.R., Brooks, E., Platek, O., McAllaster, D., Nagesha, V. (2017) A Mostly Data-Driven Approach to Inverse Text Normalization. Proc. Interspeech 2017, 2784-2788, DOI: 10.21437/Interspeech.2017-1274.

Erin Fitzgerald, Frederick Jelinek, Robert Frank. What lies beneath: Semantic and syntactic analysis of manually reconstructed spontaneous speech. ACL/IJCNLP 2009: 746-754.

Goodfellow, I., Y. Bengio, and A. Courville 2016. Deep learning. Cambridge, MA, USA: MIT press.

Helcl Jindřich, Libovický Jindřich, Kocmi Tom, Musil Tomáš, Cífka Ondřej, Variš Dušan, Bojar Ondřej: Neural Monkey: The Current State and Beyond. In: The 13th Conference of The Association for Machine Translation in the Americas, Vol. 1: MT Researchers’ Track, Copyright © The Association for Machine Translation in the Americas, Stroudsburg, PA, USA, pp. 168-176, 2018