Word Embeddings through Hellinger PCA

通过Hellinger PCA方法的词嵌入

Abstract

Word embeddingsresulting from neural language models have been shown to be a great asset for alarge variety of NLP tasks. However, such architecture might be difficult andtime-consuming to train. Instead, we propose to drastically simplify the wordembeddings computation through a Hellinger PCA of the word cooccurence matrix.We compare those new word embeddings with some well-known embeddings on namedentity recognition and movie review tasks and show that we can reach similar oreven better performance. Although deep learning is not really necessary forgenerating good word embeddings, we show that it can provide an easy way toadapt embeddings to specific tasks

单词嵌入产生的自然语言模型已经在NLP任务上展现了非常重大的价值。然而，这种结构的训练可能很难而且费时。取而代之的，我们提出通过对词共现矩阵的彻底简化单词嵌入计算Hellinger-PCA。我们在命名实体识别和电影评论任务上对一些有名的嵌入比较了这些新的词嵌入的优劣，证明我们能实现相似的或者更好的性能。尽管产生好的词嵌入并不是必须的，我们证明深度学习在某些方法上是一个适应嵌入的简单的方法。

这篇文章通过Hellinger-PCA（Hellinger主成分分析）试图解决词嵌入的问题，但是作者针对的是命名实体识别的任务以及电影评论的任务。所以这篇文章当然没有那么大的创新性了。不过总体而言，应该算是一篇中规中矩的文章。这篇文章