本文为捷克布尔诺理工大学（作者：DISERTACN ˇ ′I PRACE ′）的博士论文，共133页。

统计语言模型是许多成功应用程序的关键部分，例如自动语音识别和统计机器翻译（如著名的谷歌翻译）。估计这些模型的传统技术是基于Ngram计数的。尽管已知N-gram模型的弱点以及许多领域研究团体的巨大努力（语音识别、机器翻译、神经科学、人工智能、自然语言处理、数据压缩、心理学等），N-gram模型基本上仍然是最先进的。本文的目的是提出基于人工神经网络的语言模型的各种体系结构。尽管这些模型的计算成本比N-gram模型高，但使用本文所提供的技术，可以有效地将它们应用到最先进的系统中。相对于最先进的N-gram模型，语音识别系统的误码率降低了20%。所提出的基于递归神经网络的模型在众所周知的Penn Treebank设置上获得了最佳的发布性能。

Statistical language models are crucial part of many successfulapplications, such as automatic speech recognition and statistical machinetranslation (for example well-known Google Translate). Traditional techniquesfor estimating these models are based on Ngramcounts. Despite known weaknesses of N-gramsand huge efforts of research communities across many fields (speechrecognition, machine translation, neuroscience, artificial intelligence,natural language processing, data compression, psychology etc.), N-gramsremained basically the state-of-the-art. The goal of this thesis is to presentvarious architectures of language models that are based on artificial neuralnetworks. Although these models are computationally more expensive than N-grammodels, with the presented techniques it is possible to apply them tostate-of-the-art systems efficiently. Achieved reductions of word error rate ofspeech recognition systems are up to 20%, against state-of-the-art N-grammodel. The presented recurrent neural network based model achieves the bestpublished performance on well-known Penn Treebank setup.

1 引言

2 统计语言模型回顾

3 基于神经网络的语言模型

4 语言建模技术的评估与组合

5 《华尔街日报》实验

6 大规模神经网络语言模型的训练策略

7 其它实验

8 论自然语言的智能模型

9 结论与未来工作展望

更多精彩文章请关注公众号：