![]() ![]() For instance, the famous word2vec algorithm ( Mikolov et al., 2013a, b) learns word representation from each word's local context window (i.e., surrounding words) so that local contextual similarity of words are preserved. ![]() Typically, the mapping function is learned based on the assumption that words sharing similar local contexts are semantically close. Words and phrases, which are originally represented as one-hot vectors, are embedded into a continuous low-dimensional space. Unsupervised word representation learning, or word embedding, has shown remarkable effectiveness in various text analysis tasks, such as named entity recognition ( Lample et al., 2016), text classification ( Kim, 2014) and machine translation ( Cho et al., 2014). Our quantitative analysis and case study show that despite their simplicity, our two proposed models achieve superior performance on word similarity and text classification tasks. We conduct a thorough evaluation on a wide range of benchmark datasets. We provide theoretical interpretations of the proposed models to demonstrate how local and global contexts are jointly modeled, assuming a generative relationship between words and contexts. ![]() We propose two simple yet effective unsupervised word embedding models that jointly model both local and global contexts to learn word representations. Global contexts, referring to the broader semantic units, such as the document or paragraph where the word appears, can capture different aspects of word semantics and complement local contexts. We argue that local contexts can only partially define word semantics in the unsupervised word embedding learning. Word representations are typically learned by modeling local contexts of words, assuming that words sharing similar surrounding words are semantically close. Word embedding has benefited a broad spectrum of text analysis tasks by learning distributed word representations to encode word semantics. 2School of Computational Science and Engineering, College of Computing, Georgia Institute of Technology, Atlanta, GA, United States. ![]()
0 Comments
Leave a Reply. |