Beyond word vectors: adding structure to vector representations of meaning
Steven Schockaert
Cardiff University
While the use of word vectors is now standard in natural language processing, from a knowledge representation point of view such vectors have important shortcomings. In this talk, I will discuss two central issues with word vectors, and outline possible strategies for addressing them.
First, word vectors are inherently limited in how they can express relational information. While vector difference based approaches have been found to successfully model some types of relations, such approaches are provably limited in the kinds of relationships they can capture. As an alternative, in our recent work, we have proposed to combine word vectors with relation vectors, where the latter are explicitly learned to capture how different pairs of words are related.
Second, while representing entities as vectors seems natural, in many applications concepts (or categories) also play a central role. Such concepts formally correspond to sets of entities, hence they are more naturally represented as regions in a vector space. Estimating regions in high-dimensional spaces is challenging however, which may explain why this topics has not yet received much attention. In our recent work, we have proposed a solution based on Bayesian estimation of Gaussian densities, where prior information about the semantic relationships between different concepts is exploited to learn more faithful concept representations.
Week 23 2018/2019
Thursday 9th May 2019
3:00-4:00pm
B78 (DSI Space) InfoLab21
Joint Data Science Group and UCREL talk