A Unified Architecture for Natural Language Processing: Deep NeuralNetworks with Multitask Learning

一个自然语言处理的统一的架构：面向多种任务的深度神经网络

Abstract

We describe asingle convolutional neural network architecture that, given a sentence,outputs a host of language processing predictions: part-of-speech tags, chunks,named entity

tags, semanticroles, semantically similar words and the likelihood that the sentence makessense (grammatically and semantically) using a language model. The entirenetwork is trained jointly on all these tasks using weight-sharing, an instanceof multitask learning. All the tasks use labeled data except the language modelwhich is learnt from unlabeled text and represents a novel form of semi-supervisedlearning for the shared tasks. We show how both multitask learning andsemi-supervised learning improve the generalization of the shared tasks,resulting in state-of-the-art performance.

我们描述了一个单一的卷积神经网络结构，给定一个句子，输出一个可信的语言处理预测：词性标注，分析（分块），命名实体识别，语义角色，语义相似单词以及相似性，句子从语法上和语义上都非常合理地使用一个语言模型。整个网络在所有这些任务上使用权重分享，一个多任务学习进行联合训练。所有的任务都使用打标签的数据，除了语言模型意外，语言模型通过非标注的文本数据学习，而且代表了一个通过共享任务的进行半监督学习新颖形式。我们证明这种多任务学习以及半监督学习改进了共享任务的生成，带来了最佳的效果。