Institute of Formal and Applied Linguistics
Charles University, Czech Republic
Faculty of Mathematics and Physics
Morphological Derivation in Context Configurations for Class-Specific Word Embeddings
Iveta Kršková
We explore how the knowledge of morphological derivation relations in individual English words affects the performance of a word embedding model (in our case the word2vecf model, Levy et al. 2014).
A word embedding model is a representation of words in a corpus as vectors in a semantic vector space. The similarity of individual vectors is considered to reflect the semantic similarity/relatedness of individual words. This idea draws on the Distributional Hypothesis by Zellig S. Harris (1970), according to which two words occurring in more similar contexts are more semantically related than two words that occur in less similar contexts. The Distributional Hypothesis has a less frequently cited counterpart -- a theory of linguistic transformations, which elaborates on various aspects of context similarity with many examples and which we found particularly interesting.
As Harris' proposed transformations are numerous and go across all levels of linguistic description, we narrowed down our scope to transformations with morphological derivation (e.g. to sing aloud - a loud singer or to love cats - a cat lover). We drew on CELEX, a publicly available database of English morphological derivativation (Baayen et al., 1995), to extract word pairs connected by morphological derivations. Then we took word2vecf and tested different experiment setups to add the morphological derivation information to the regular text input. The parsing scheme was Universal Dependencies (Agic et al., 2015). The baseline was a system reported by Vulic et al. (2017), which contains information on syntactic dependencies between words.
REFERENCES
Agic, Zeljko, Maria Aranzabe, Aitziber Atutxa, Cristina Bosco, Jinho Choi, Marie-Catherine de Marneffe, Timothy Dozat, et al. 2015. Universal Dependencies 1.1. Praha, Czechia: LINDAT/CLARIN digital library at Institute of Formal and Applied Linguistics,Charles University in Prague.
Baayen, R, R Piepenbrock, and L Gulikers. 1995. CELEX2, LDC96L14. Web download, Linguistic Data Consortium, Philadelpha, PA.
Harris, Z. S. (1970) Papers in Structural and Transformational Linguistics. Formal Linguistics series. Reidel.
Levy, O. and Goldberg, Y. 2014. Dependency-Based Word Embeddings. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics 2
Vulic, I. and Schwartz, R. and Rappoport, A. and Reichart, R. and Korhonen, A. 2017. Automatic selection of context configurations for improved (and fast) class-specific word representations. CoNLL 2017.