Overview of mini-batch gradient descent & tricks
ICML_2013_On the importance of initialization and momentum in deep learning
2012_No more pesky learning rates
1. Mini-batch gradient descent2. A bag of tricks
3. The momentum method
4. A separate, adaptive learning rate for each connection
5. Rmsprop
https://blog.sciencenet.cn/blog-1583812-814649.html
上一篇:
Coursera: Neural Networks for ML- Lecture 5下一篇:
Coursera: Neural Networks for ML- Lecture 7