engleski

Lexicon-Based Morphological Normalisation and its Aplication to Croatian Language

Due to language morphology, words appear in text in various inflectional and derivational forms. This morphological variation has been shown to negatively affect the performance of most information retrieval and text mining systems. Morphological variation may be reduced by performing morphological normalisation, i.e., the conflation of morphological variants of a word into a single representative form. A lexicon-based approach to normalisation allows for high normalisation precision, which for morphologically complex languages may otherwise be difficult to achieve. In this paper we describe a two-stage lexicon-based approach to morphological normalisation that addresses both inflectional and derivational variation. To eliminate the immense effort required to compile a lexicon by hand, we devise a procedure for acquiring automatically an inflectional morphological lexicon from raw corpora. We also propose a convenient and highly expressive morphology representation formalism on which the acquisition procedure is based. We apply our approach to the morphologically complex Croatian language, but our approach should be equally applicable to other languages of similar morphological complexity. A detailed task-independent evaluation reveals that our approach yields good normalisation performance at both inflectional and derivational level.

Morphological normalisation, morphological lexicon, inflection, derivation, lexicon acquisition, Croatian language

nije evidentirano