大工至善|大学至真分享 http://blog.sciencenet.cn/u/lcj2212916

博文

[转载]【计算机科学】【2016.01】自动编码器能从大脑中学习到什么?

已有 337 次阅读 2019-6-17 21:30 |系统分类:科研笔记|文章来源:转载


本文为Gerben van den Broeke的硕士论文87

 

本文探讨了无监督深度学习算法的基本改进以无监督学习的目的为理论视角选择共同学习的有向生成模型中的学习近似推理作为方法主要问题是如何通过同时重新思考学习方式推理方式来改进这种方法的现有实现特别是自动编码的实现在这种网络体系结构中两个可用的相对路径一个用于推理一个用于生成允许利用它们之间的对称性并允许其中一个向另一个提供反馈信号这些信号可用于仅从本地可用信息来确定连接权重的有效更新从而消除了对传统反向传播路径的需要并缓解了与之相关的问题此外反馈回路可以添加到通常的前馈网络中以改进推理本身大脑新皮质区域之间的相互联系为迭代修正和验证所提出的解释如何能够公平地近似于最佳贝叶斯推理提供了灵感

 

本文从深度学习和皮层功能的研究中提取并结合了潜在的思想对生成模型近似推理局部学习规则目标传播循环横向和偏态竞争预测编码迭代和摊销推理等相关概念进行了探讨通过以上主题我们试图建立一个复杂的见解可以为未来无监督的深度学习方法研究提供指导

 

This thesis explores fundamental improvements in unsupervised deep learning algorithms. Taking a theoretical perspective on the purpose of unsupervised learning, and choosing learnt approximate inference in a jointly learnt directed generative model as the approach, the main question is how existing implementations of this approach, in particular auto-encoders, could be improved by simultaneously rethinking the way they learn and the way they perform inference. In such network architectures, the availability of two opposing pathways, one for inference and one for generation, allows to exploit the symmetry between them and to let either provide feedback signals to the other. The signals can be used to determine helpful updates for the connection weights from only locally available information, removing the need for the conventional back-propagation path and mitigating the issues associated with it. Moreover, feedback loops can be added to the usual usual feed-forward network to improve inference itself. The reciprocal connectivity between regions in the brain's neocortex provides inspiration for how the iterative revision and verification of proposed interpretations could result in a fair approximation to optimal Bayesian inference. While extracting and combining underlying ideas from research in deep learning and cortical functioning, this thesis walks through the concepts of generative models, approximate inference, local learning rules, target propagation, recirculation, lateral and biased competition, predictive coding, iterative and amortised inference, and other related topics, in an attempt to build up a complex of insights that could provide direction to future research in unsupervised deep learning methods.

 

无监督学习概念

无监督学习算法

无监督学习大脑

产生反馈信息用于学习

产生反馈信息用于推理

综合分析 


下载英文原文地址:

http://page3.dfpan.com/fs/2lc3j2221e298161a07/ 


更多精彩文章请关注微信号:qrcode_for_gh_60b944f6c215_258.jpg



http://blog.sciencenet.cn/blog-69686-1185530.html

上一篇:[转载]【图片新闻】美国海军成功实现历史上水下最深18500英尺的残骸回收
下一篇:[转载]【无人机】【2017.05】离散延迟扩展卡尔曼滤波器中的无人机失序测量跟踪系统

0

该博文允许注册用户评论 请点击登录 评论 (0 个评论)

数据加载中...

Archiver|手机版|科学网 ( 京ICP备14006957 )

GMT+8, 2019-7-20 12:00

Powered by ScienceNet.cn

Copyright © 2007- 中国科学报社

返回顶部