Improving Sparse Word Representations with Distributional Inference for Semantic Composition

Thomas Kober, Julie Weeds, Jeremy Reffin, David Weir

    Research output: Contribution to journalArticlepeer-review

    10 Citations (Scopus)

    Abstract

    Distributional models are derived from co-occurrences in a corpus, where only a small proportion of all possible plausible co-occurrences will be observed. This results in a very sparse vector space, requiring a mechanism for inferring missing knowledge. Most methods face this challenge in ways that render the resulting word representa-tions uninterpretable, with the consequence that semantic composition becomes hard to model. In this paper we explore an alter-native which involves explicitly inferring un-observed co-occurrences using the distribu-tional neighbourhood. We show that distribu-tional inference improves sparse word repre-sentations on several word similarity bench-marks and demonstrate that our model is com-petitive with the state-of-the-art for adjective-noun, noun-noun and verb-object composi-tions while being fully interpretable.
    Original languageEnglish
    JournalEmnlp
    Publication statusPublished - 2016

    Fingerprint

    Dive into the research topics of 'Improving Sparse Word Representations with Distributional Inference for Semantic Composition'. Together they form a unique fingerprint.

    Cite this