AMALACH

The aim of the project AMALACH (ASR- and MT-based Access to a Large Archive of Cultural Heritage) is to design and implement software tools for facilitating access into a large collection of videos, interviews with holocaust survivors. The archive, now hosted at University of Southern California, Shoah Foundation Institute, contains more than 110 thousand hours of recordings in 32 languages. About half of the interviews are held in English and Czech amounts to approximately one thousand hours.

Current access methods allow to search for keywords listed in a pre-defined dictionary (thesaurus) because snippets of the recordings were manually tagged with these keywords. The coverage of this manual labelling is however insufficient especially in the Czech part of the archive.

The project AMALACH thus aims to use advanced methods of automatic speech recognition (ASR) and machine translation (MT) to enable search in at least all the Czech and English recordings.

Partners

Results

# Result Due Delivered Type Documentation
1 ASR for Czech 31.12.2012 31.12.2012 Software module SEASR-CZE
2 Machine translation (text) 31.12.2013 31.12.2013 Software module see package (TMODS:ENG-CZE)
3 ASR for English 31.12.2014 31.12.2014 Software module SEASR-ENG
4 Machine translation (thesaurus, queries) 30.6.2015 30.6.2015  Software module see package (TMODS:ENG-CZE)
5 Search module 30.6.2015 31.12.2015 Software module WFBAS
6 Integrated system MCLAAS 31.12.2014 31.12.2014 Software module MCLAAS
7 Integrated system deployed 31.12.2015 31.12.2015 Deployed at CVHM and ZM Praha, functional prototype Deployment documentation

Documentation to other results is part of the data package referred to from the above table.

Preliminary and partial results delivered:

  • Thesaurus (part of result #4)
  • USC-SFI MALACH Interviews and Transcripts Czech (software), delivered 16. 3. 2014, documentation

Výsledky vznikly jako součást řešení projektu Ministerstva kultury číslo DF12P01OVV022 a podléhají licenčním podmínkám daného typu projektu. Licence je všem zájemcům poskytována zdarma, avšak nezbytnou podmínkou pro využívání tohoto výsledku je, aby měl uživatel ošetřeno právo přístupu k nahrávkám, nad kterými se vyhledávání provádí, pokud tento požadavek je dle licence na jednotlivé časti systému jejich licencí vyžadován. Veškerá práva k těmto nahrávkám jsou majetkem USC Shoah Foundation. Další informace lze získat na vyžádání na riv@control.zcu.cz.

Publications

  • Galuščáková Petra, Pecina Pavel: Audio Information for Hyperlinking of TV Content. In: Proceedings of the Third Edition Workshop on Speech, Language & Audio in Multimedia, Copyright © ACM, New York, NY, USA, ISBN 978-1-4503-3749-6, pp. 27-30, 2015
  • Urešová Zdeňka, Dušek Ondřej, Fučíková Eva, Hajič Jan, Šindlerová Jana: Bilingual English-Czech Valency Lexicon Linked to a Parallel Corpus. In: Proceedings of the The 9th Linguistic Annotation Workshop (LAW IX 2015) , Copyright © Association for Computational Linguistics, Stroudsburg, PA, USA, ISBN 978-1-941643-47-1, pp. 124-128, 2015
  • Zelinka Jan, Vaněk Jan, Müller Luděk. Neural-Network-based Spectrum Processing for Speech Recognition and Speaker Verification. Statistical Language and Speech Processing, Third International Conference, SLSP 2015, Budapest, Hungary, November 24-26, 2015. Proceedings,288-299. 2015
  • Soutner Daniel, Müller Luděk. On Continuous Space Word Representations as Input of LSTM Language Model. Statistical Language and Speech Processing, Third International Conference, SLSP 2015, Budapest, Hungary, November 24-26, 2015. Proceedings,267-274. 2015
  • Zelinka Jan, Salajka Petr, Müller Luděk. On Deep and Shallow Neural Networks in Speech Recognition from Speech Spectrum. Speech and Computer, 17th International Conference, SPECOM 2015, Athens, Greece, September 20-24,2015, Proceedings,301-308. 2015
  • Zelinka Jan, Vaněk Jan, Müller Luděk. Simultaneously Trained NN-based Acoustic Model and NN-based Feature Extractor. Text, Speech, and Dialogue, 18th International Conference, TSD 2015, Pilsen, Czech Republic, September 14-17, 2015. Proceedings,234-242. 2015
  • Galuščáková Petra, Pecina Pavel: Experiments with Segmentation Strategies for Passage Retrieval in Audio-Visual Documents. In: ICMR '14 Proceedings of International Conference on Multimedia Retrieval , Copyright © ACM, New York, NY, USA, ISBN 978-1-4503-2782-4, pp. 217-225, 2014
  • Urešová Zdeňka, Hajič Jan, Bojar Ondřej: Comparing Czech and English AMRs. In: Proceedings of Workshop on Lexical and Grammatical Resources for Language Processing (LG-LP 2014, at Coling 2014), Copyright © Association for Computational Linguistics and Dublin City University, Dublin, Ireland, ISBN 978-1-873769-44-7, pp. 55-64, 2014
  • Skorkovská Lucie, Zajíc Zbyněk, Müller Luděk. Comparison of Score Normalization Methods Applied to Multi-label Classification. Signal Processing and Information Technology (ISSPIT), 2014 IEEE International Symposium on,433-437. 2014
  • Soutner Daniel, Zelinka Jan, Müller Luděk. On a Hybrid NN/HMM Speech Recognition System with a RNN-Based Language Model. Speech and Computer, 16th International Conference, SPECOM 2014, Novi Sad, Serbia, October 5-9, 2014, Proceedings,315-321. 2014
  • Zajíc Zbyněk, Zelinka Jan, Vaněk Jan, Müller Luděk. Convolutional Neural Network for Refinement of Speaker Adaptation Transformation. Speech and Computer, 16th International Conference, SPECOM 2014, Novi Sad, Serbia, October 5-9, 2014, Proceedings,161-168. 2014
  • Soutner Daniel, Müller Luděk. Continuous Distributed Representations of Words as Input of LSTM Network Language Model. Text, Speech, and Dialogue, 17th International Conference, TSD 2014, Brno, Czech Republic, September 8-12, 2014. Proceedings,150-157. 2014
  • Mareček David, Straka Milan: Stop-probability estimates computed on a large corpus improve Unsupervised Dependency Parsing. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, Copyright © Association for Computational Linguistics, Sofija, Bulgaria, ISBN 978-1-937284-50-3, pp. 281-290, 2013
  • Soutner Daniel, Müller Luděk. Application of LSTM Neural Networks in Language Modelling. Text, Speech, and Dialogue,105-112. 2013
  • Vavruška Jan, Švec Jan, Ircing Pavel. Phonetic Spoken Term Detection in Large Audio Archive Using the WFST Framework. Text, Speech, and Dialogue,402-409. 2013
  • Galuščáková Petra: Application of Topic Segmentation in Audiovisual Information Retrieval. In: WDS'12 Proceedings of Contributed Papers, Copyright © Matfyzpress, Praha, Czechia, ISBN 978-80-7378-224-5, pp. 118-122, 2012
  • Galuščáková Petra, Pecina Pavel, Hajič Jan: Penalty Functions for Evaluation Measures of Unsegmented Speech Retrieval. In: Lecture Notes in Computer Science, Vol. 7488, Information Access Evaluation. Multilinguality, Multimodality, and Visual Analytics - Third International Conference of the CLEF Initiative, Copyright © Springer, Berlin / Heidelberg, ISBN 978-3-642-33246-3, ISSN 0302-9743, pp. 100-111, 2012
  • Galuščáková Petra, Pecina Pavel: CUNI at MediaEval 2012 Search and Hyperlinking Task. In: Working Notes Proceedings of the MediaEval 2012 Workshop, Copyright © CEUR Workshop Proceedings, Aachen, Germany, ISSN 1613-0073, 2012
  • Trmal Jan, Zelinka Jan, Müller Luděk. Unsupervised and semi-supervised adaptation of a hybrid speech recognition system. Proceedings 2012 IEEE 11th International Conference on Signal Processing,527-530. 2012
  • Zelinka Jan, Trmal Jan, Müller Luděk. On Context-Dependent Neural Networks and Speaker Adaptation. Proceedings 2012 IEEE 11th International Conference on Signal Processing,515-518. 2012
  • Zajíc Zbyněk, Machlica Lukáš, Müller Luděk. Initialization of Adaptation by Sufficient Statistics Using Phonetic Tree. Proceedings 2012 IEEE 11th International Conference on Signal Processing,503-506. 2012
  • Zajíc Zbyněk, Machlica Lukáš, Müller Luděk. Robust Adaptation Techniques Dealing with Small Amount of Data. Lecture Notes in Computer Science,7499,neuveden,480-487. 2012