||
Overfitting happens with excessive power, stochastic/deterministic noise, and limited data.
1. Regularized Hypothesis Set
在实际训练过程中,复杂的模型容易产生过拟合现象,最终导致得到的训练模型的$E_{out}$变得非常大。一种很自然的想法就是给模型加上一些附加限制,用以避免过拟合现象,同时使得模型尽可能的简单化!
Regression with Looser Constraint:
Regression with Softer Constraint:
2. Weight Decay Regularization
The Lagrange Multiplier
(对于上图关于The Lagrange Multiplier的直观解释,让我印象深刻!)
Some Detail: Legendre Polynomials (正则化过程中,可以选择正交的基函数)
3. Regularization and VC Theory
4. General Regularizers
Regularizers: constraint in the "direction" of target function.
The Optimal $\lambda$
Archiver|手机版|科学网 ( 京ICP备07017567号-12 )
GMT+8, 2024-9-1 07:15
Powered by ScienceNet.cn
Copyright © 2007- 中国科学报社