Logo ÚFAL Logo MFF

The Prague Bulletin of Mathematical Linguistics


ÚFAL
PBML

Revamping the SLTev Tool for Evaluation of Spoken Language Translation

Michelle Elizabeth, Ondřej Bojar

References:

  1. Chantal Amrhein and Barry Haddow. Don't Discard Fixed-Window Audio Segmentation in Speech-to-Text Translation In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 203–219, Association for Computational Linguistics, Abu Dhabi, United Arab Emirates (Hybrid), 2022.
  2. Antonios Anastasopoulos, Lo Barrault, Luisa Bentivogli, Marcely Zanon Boito, Ondřej Bojar, Roldano Cattoni, Anna Currey, Georgiana Dinu, Kevin Duh, Maha Elbayad, Clara Emmanuel, Yannick Estève, Marcello Federico, Christian Federmann, Souhir Gahbiche, Hongyu Gong, Roman Grundkiewicz, Barry Haddow, Benjamin Hsu, Dávid Javorský, Vĕra Kloudová, Surafel Lakew, Xutai Ma, Prashant Mathur, Paul McNamee, Kenton Murray, Maria Nadejde, Satoshi Nakamura, Matteo Negri, Jan Niehues, Xing Niu, John Ortega, Juan Pino, Elizabeth Salesky, Jiatong Shi, Matthias Sperber, Sebastian Stüker, Katsuhito Sudoh, Marco Turchi, Yogesh Virkar, Alexander Waibel, Changhan Wang, and Shinji Watanabe. Findings of the IWSLT 2022 Evaluation Campaign In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 98–157, Association for Computational Linguistics, Dublin, Ireland (in-person and online), 2022. (http://doi.org/10.18653/v1/2022.iwslt-1.10)
  3. Ebrahim Ansari, Ondřej Bojar, Barry Haddow, and Mohammad Mahmoudi. SLTEV: Comprehensive Evaluation of Spoken Language Translation In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages 71–79, Association for Computational Linguistics, Online, 2021. (http://doi.org/10.18653/v1/2021.eacl-demos.9)
  4. Dario Franceschini, Chiara Canton, Ivan Simonini, Armin Schweinfurth, Adelheid Glott, Sebastian Stüker, Thai-Son Nguyen, Felix Schneider, Thanh-Le Ha, Alex Waibel, Barry Haddow, Philip Williams, Rico Sennrich, Ondřej Bojar, Sangeet Sagar, Dominik Macháček, and Otakar Smrž. Removing European Language Barriers with Innovative Machine Translation Technology In Proceedings of the 1st International Workshop on Language Technology Platforms, pages 44–49, European Language Resources Association, Marseille, France, 2020.
  5. Ryo Fukuda, Yuka Ko, Yasumasa Kano, Kosuke Doi, Hirotaka Tokuyama, Sakriani Sakti, Katsuhito Sudoh, and Satoshi Nakamura. NAIST Simultaneous Speech-to-Text Translation System for IWSLT 2022 In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 286–292, Association for Computational Linguistics, Dublin, Ireland (in-person and online), 2022. (http://doi.org/10.18653/v1/2022.iwslt-1.25)
  6. Marco Gaido, Sara Papi, Dennis Fucci, Giuseppe Fiameni, Matteo Negri, and Marco Turchi. Efficient yet Competitive Speech Translation: FBK@IWSLT2022 In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 177–189, Association for Computational Linguistics, Dublin, Ireland (in-person and online), 2022. (http://doi.org/10.18653/v1/2022.iwslt-1.13)
  7. Javier Iranzo-Sánchez, Javier Jorge Cano, Alejandro Pérez-González-de-Martos, Adrián Giménez Pastor, Gonçal Garcés Díaz-Munío, Pau Baquero-Arnal, Joan Albert Silvestre-Cerdà, Jorge Civera Saiz, Albert Sanchis, and Alfons Juan. MLLP-VRAIN UPV systems for the IWSLT 2022 Simultaneous Speech Translation and Speech-to-Speech Translation tasks In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 255–264, Association for Computational Linguistics, Dublin, Ireland (in-person and online), 2022. (http://doi.org/10.18653/v1/2022.iwslt-1.22)
  8. Evgeny Matusov, Gregor Leusch, Oliver Bender, and Hermann Ney. Evaluating Machine Translation Output with Automatic Sentence Segmentation In Proceedings of the Second International Workshop on Spoken Language Translation, Pittsburgh, Pennsylvania, USA, 2005.
  9. Peter Polák, Ngoc-Quan Pham, Tuan Nam Nguyen, Danni Liu, Carlos Mullov, Jan Niehues, Ondřej Bojar, and Alexander Waibel. CUNI-KIT System for Simultaneous Speech Translation Task at IWSLT 2022 In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 277–285, Association for Computational Linguistics, Dublin, Ireland (in-person and online), 2022. (http://doi.org/10.18653/v1/2022.iwslt-1.24)
  10. Maja Popović. chrF: character n-gram F-score for automatic MT evaluation In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Association for Computational Linguistics, Lisbon, Portugal, 2015. (http://doi.org/10.18653/v1/W15-3049)
  11. Matt Post. A Call for Clarity in Reporting BLEU Scores In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186–191, Association for Computational Linguistics, Belgium, Brussels, 2018. (http://doi.org/10.18653/v1/W18-6319)
  12. Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. COMET: A Neural Framework for MT Evaluation In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685–2702, Association for Computational Linguistics, Online, 2020. (http://doi.org/10.18653/v1/2020.emnlp-main.213)
  13. Ricardo Rei, José G. C. de Souza, Duarte Alves, Chrysoula Zerva, Ana C Farinha, Taisiya Glushkova, Alon Lavie, Luisa Coheur, and André F. T. Martins. COMET-22: Unbabel-IST 2022 Submission for the Metrics Shared Task In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 578–585, Association for Computational Linguistics, Abu Dhabi, United Arab Emirates (Hybrid), 2022.
  14. Xutai Ma, Mohammad Javad Dousti, Changhan Wang, Jiatao Gu, and Juan Pino. Simuleval: An evaluation toolkit for simultaneous translation In Proceedings of the EMNLP, 2020. (http://doi.org/10.18653/v1/2020.emnlp-demos.19)
  15. Minghan Wang, Jiaxin Guo, Yinglu Li, Xiaosong Qiao, Yuxia Wang, Zongyao Li, Chang Su, Yimeng Chen, Min Zhang, Shimin Tao, Hao Yang, and Ying Qin. The HW-TSC's Simultaneous Speech Translation System for IWSLT 2022 Evaluation In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 247–254, Association for Computational Linguistics, Dublin, Ireland (in-person and online), 2022. (http://doi.org/10.18653/v1/2022.iwslt-1.21)