King's College London

Research portal

Improving Sparse Word Representations with Distributional Inference for Semantic Composition

Research output: Contribution to journalArticle

Thomas Kober, Julie Weeds, Jeremy Reffin, David Weir

Original languageEnglish

King's Authors


Distributional models are derived from co-occurrences in a corpus, where only a small proportion of all possible plausible co-occurrences will be observed. This results in a very sparse vector space, requiring a mechanism for inferring missing knowledge. Most methods face this challenge in ways that render the resulting word representa-tions uninterpretable, with the consequence that semantic composition becomes hard to model. In this paper we explore an alter-native which involves explicitly inferring un-observed co-occurrences using the distribu-tional neighbourhood. We show that distribu-tional inference improves sparse word repre-sentations on several word similarity bench-marks and demonstrate that our model is com-petitive with the state-of-the-art for adjective-noun, noun-noun and verb-object composi-tions while being fully interpretable.

View graph of relations

© 2020 King's College London | Strand | London WC2R 2LS | England | United Kingdom | Tel +44 (0)20 7836 5454