EVALD (Evaluator of Discourse)

Automatic Evaluation of Text Coherence in Czech

The aim of the project is to create a software application (computer programme) that will evaluate the continuity (i.e. coherence) of texts like graduation essays or essays written within the certified tests of Czech for foreigners, including works created by students with special educational needs (such as hearing impaired students whose writing skills demonstrate specificities comparable to the students of Czech as a foreign language). Pupils and students often have problems with writing a comprehensive and continuous text as creating texts is more demanding for them than creating separate sentences. Text is not simply „a cluster of sentences“. Its structure has its own rules and laws whose failure can result in the so called incoherent text, i.e. a text that is not fully functional in communication. The ability of creating text should be encouraged already in the teaching process and it is also important to have appropriate tools for its assessment. Within the project, we will carry out a detailed research on the patterns of text construction and the possibility of automatic monitoring of their maintenance in the texts already created. The main result of the project will be a software application (computer programme) that will serve as a tool for objective assessment of the text coherence, i.e. whether the submitted work fulfills the demands placed on a comprehensive text as a unit of the functional communication process. The main objective of the project is to encourage the skills of creating continuous and coherent written texts in the mother tongue.

EVALD on-line: https://lindat.mff.cuni.cz/services/evald/

EVALD FOR FOREIGNERS on-line: https://lindat.mff.cuni.cz/services/evald-foreign/

EVALD FOR BEGINNERS on-line: https://lindat.mff.cuni.cz/services/evald-begin/

Publications

SOFTWARE

Novák, Michal; Mírovský, Jiří;  Rysová, Kateřina; Rysová, Magdaléna; Hajičová, Eva, 2019, EVALD 4.0 for Beginners – Evaluator of Discourse, LINDAT/CLARIN digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University, http://hdl.handle.net/11234/1-3067.

Novák, Michal; Mírovský, Jiří;  Rysová, Kateřina; Rysová, Magdaléna; Hajičová, Eva, 2019, EVALD 4.0 for Foreigners – Evaluator of Discourse, LINDAT/CLARIN digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University, http://hdl.handle.net/11234/1-3066.

Novák, Michal; Mírovský, Jiří;  Rysová, Kateřina; Rysová, Magdaléna; Hajičová, Eva, 2019, EVALD 4.0 – Evaluator of Discourse, LINDAT/CLARIN digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University, http://hdl.handle.net/11234/1-3065.

Mírovský, Jiří; Novák, Michal; Rysová, Kateřina; Rysová, Magdaléna; Hajičová, Eva, 2018, EVALD 3.0 for Foreigners – Evaluator of Discourse, LINDAT/CLARIN digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University, http://hdl.handle.net/11234/1-2864.

Mírovský, Jiří; Novák, Michal; Rysová, Kateřina; Rysová, Magdaléna; Hajičová, Eva, 2018, EVALD 3.0 – Evaluator of Discourse, LINDAT/CLARIN digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University, http://hdl.handle.net/11234/1-2863.

Novák, Michal; Rysová, Kateřina; Mírovský, Jiří; Rysová, Magdaléna; Hajičová, Eva, 2017, EVALD 2.0 for Foreigners, LINDAT/CLARIN digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University, http://hdl.handle.net/11234/1-2509.

Novák, Michal; Rysová, Kateřina; Mírovský, Jiří; Rysová, Magdaléna; Hajičová, Eva, 2017, EVALD 2.0, LINDAT/CLARIN digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University, http://hdl.handle.net/11234/1-2508.

Rysová, Kateřina; Mírovský, Jiří; Novák, Michal; Rysová, Magdaléna, 2016, EVALD 1.0 for Foreigners, LINDAT/CLARIN digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University, http://hdl.handle.net/11234/1-1821.

Rysová, Kateřina; Mírovský, Jiří; Novák, Michal; Rysová, Magdaléna, 2016, EVALD 1.0, LINDAT/CLARIN digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University, http://hdl.handle.net/11234/1-1820.

 

CONFERENCE PAPERS AND JOURNAL ARTICLES

Rysová Kateřina, Rysová Magdaléna, Novák Michal, Mírovský Jiří, Hajičová Eva: EVALD – a Pioneer Application for Automated Essay Scoring in Czech. The Prague Bulletin of Mathematical Linguistics 113, 2019, pp. 9–30, ISSN 0032-6585. pdf
Novák Michal, Mírovský Jiří, Rysová Kateřina, Rysová Magdaléna: Exploiting Large Unlabeled Data in Automatic Evaluation of Coherence in Czech. In Lecture Notes in Computer Science, Vol. 11697, Proceedings of the 22nd International Conference on Text, Speech and Dialogue – TSD 2019. Cham / Heidelberg / New York / Dordrecht / London: Springer International Publishing, 2019, pp. 197–210, ISSN 0302-9743, ISBN 978-3-030-27946-2. pdf

Rysová Magdaléna, Rysová Kateřina, Mírovský Jiří, Novák Michal: Coherence Errors in Learners’ Essays and a Possibility of Their Improvement through EVALD (Automated Evaluator of Discourse). In Proceedings of the 11th Annual International Conference on Education and New Learning Technologies (EDULEARN 2019). Palma, Spain: IATED Academy, 2019, pp. 6761–6768, 2019, ISBN 978-84-09-12031-4, ISSN 2340-1117. pdf

Novák Michal, Mírovský Jiří, Rysová Kateřina, Rysová Magdaléna: Topic–Focus Articulation: A Third Pillar of Automatic Evaluation of Text Coherence. In Advances in Computational Intelligence (LNAI 11289): 17th Mexican International Conference on Artificial Intelligence, MICAI 2018, Proceedings, Part II, Switzerland: Springer, 2018, pp. 96–108. ISBN 978-3-030-04497-8. pdf

Rysová Magdaléna, Rysová Kateřina, Mírovský Jiří, Novák Michal: Practicing Students‘ Writing Skills through eLearning: Automated Evaluation of Text Coherence in Czech. In EDULEARN18 Proceedings. Valencia, Spain: IATED Academy, 2018, pp. 1963–1970. ISBN 978-84-09-02709-5, ISSN 2340-1117. pdf

Novák Michal, Rysová Kateřina, Rysová Magdaléna, Mírovský Jiří: Incorporating Coreference to Automatic Evaluation of Coherence in Essays. In Statistical Language and Speech Processing. Cham, Switzerland: Springer International Publishing, 2017, pp. 58–69. ISBN 978-3-319-68455-0, ISSN 1611-3349. pdf

Rysová Kateřina: Possibilities of Text Coherence Analysis in the Prague Dependency Treebank. In New perspectives on cohesion and coherence: Implications for translation. Berlin, Germany: Language Science Press, 2017, pp. 35–48. ISBN 978-3-946234-72-2. pdf

Rysová Kateřina, Rysová Magdaléna, Mírovský Jiří, Novák Michal: Introducing EVALD – Software Applications for Automatic Evaluation of Discourse in Czech. In Proceedings of the International Conference Recent Advances in Natural Language Processing, Šumen, Bulgaria: INCOMA Ltd., 2017, pp. 634–641. ISBN 978-954-452-048-9, ISSN 1313-8502. pdf

Rysová Kateřina, Rysová Magdaléna: Koreference a elipsa v Pražském závislostním korpusu. Korpus – gramatika – axiologie 13, 2016, pp. 35–47. ISSN 1804-137X. pdf

Rysová Kateřina, Rysová Magdaléna, Mírovský Jiří: Automatic evaluation of surface coherence in L2 texts in Czech. In Proceedings of the 28th Conference on Computational Linguistics and Speech Processing ROCLING XXVIII (2016). Taipei, Taiwan: The Association for Computational Linguistics and Chinese Language Processing (ACLCLP), 2016, pp. 214–228. ISBN 978-957-30792-9-3. pdf