文档下载 文档下载导航
设为首页 | 加入收藏
搜索 请输入内容:  
 导航当前位置: 文档下载 > 所有分类 > 经管营销 > The Bias Variance Tradeoff
侵权投诉

The Bias Variance Tradeoff

Thus the MSE can be decomposed in expectation into the variance of the noiseand the MSE between the true function and the predicted values. This term canbe further composed with the same augmentation trick as above.E(fi yi)

{

2

}{(

=Efi E{yi}+E{yi}i yi=E(fi E{yi})=bias2

{

}{}+2E{(E{y} y)(f E{y})}+Var{y}+2(E{fE{y}} E{E{y}} E{yf}+E{yE{y}})

2

)}

2i

+E(E{yi} yi)

i

2

iiii

2

iii

i

ii

Note: E{fiE{yi}}=fiE{yi} since f is deterministic and E{E{z}}=z

:EE{yi}

{

2

}=E{y} since E{E{z}}=z

2i

2

:E{yifi}=fiE{yi}:E{yiE{yi}}=E{yi}

E(fi yi)

: Thus the last term in the expectation above cancels to zero.

Thus the decomposition of the MSE in expectation becomes:

E(ti yi)

{

2

}=bias

2

+Var{yi}

{

2

}=Var{noise}+bias

2

+Var{yi}

Note that the variance of the noise can not be minimized; it is independent of theneural network. Thus in order to minimize the MSE, we need to minimize boththe bias and the variance. However, this is not trivial to do this. For instance, justneglecting the input data and predicting the output somehow (e.g., just a con-stant), would definitely minimize the variance of our predictions: they would bealways the same, thus the variance would be zero—but the bias of our estimate(i.e., the amount we are off the real function) would be tremendously large. Onthe other hand, the neural network could perfectly interpolate the training data,i.e., it predict y=t for every data point. This will make the bias term vanish en-tirely, since the E(y)=f (insert this above into the squared bias term to verify this),but the variance term will become equal to the variance of the noise, which maybe significant (see also Bishop Chapter 9 and the Geman et al. Paper). In general,finding an optimal bias-variance tradeoff is hard, but acceptable solutions can befound, e.g., by means of cross validation or regularization.

上一页第2页

你可能喜欢

热门文档

相关文档

  • Conceptual complexity and the biasvariance tradeoff

    2–16 Contents lists available at ScienceDirect Cognition journal homepage: www.elsevier.com/locate/COGNIT Conceptual complexity and the bias/variance trade...

  • 支持向量机指南

    对最初 SVMs 发展的驱动发生在 guises-the bias variance tradeoff、学习能力控制,过学 习等——但是基本思想是相同的。粗略的说,对于一个给定的,只有有限训练...

  • Suboptimality of the Truncated SVD for Ill-Conditio...

    This paper analyzes the bias-variance tradeoff of a class of l隐藏>> Suboptimality of the Truncated SVD for Ill-Conditioned Inverse Problems Jeffrey A. ...

  • ...Bias for Smooth Cutsize-Delay Tradeoff in Perfor...

    The Bias Variance Tradeo... 暂无评价 2页 1财富值 Individual investors ... Local Unidirectional Bias for Smooth Cutsize-Delay Tradeoff in Performance-...

  • 支持向量机指南

    对最初 SVMs 发展的驱动发生在 guises-the bias variance tradeoff、学习能力控制,过学 习等——但是基本思想是相同的。粗略的说,对于一个给定的,只有有限训练...

  • Achievable Regions in the Bias-Variance Plane for P...

    Achievable Regions in the Bias-Variance Plane for Parametric Estimation Problems 1 Alfred H Abstract--- In this paper we use the uniform Cramer-Rao (...

  • Outline of presentation

    er Practical algorithms face the bias–variance tradeoff 4 Feature subset selection (2) ? Optimal feature subset de?ned with respect to the induction ...

  • 基于svm的模式识别

    Statistical Learning The problem which drove the initial development of SVMs occurs in several guises –the bias variance tradeoff,capacity control and over ...

  • An application of the bootstrap 632+ rule to ecolog...

    A behavior typical of the bias-variance tradeoff is visible in Figure 2, where the resubstitution error and the relative over?tting rate estimate R are...

  • 机器学习Ch9-Boosting

    3 Fighting the bias-variance tradeoff 4 Voting (Ensemble Methods) 5 Voting (Ensemble Methods) 6 Boosting [Schapire’89] 7 Learning from weighted data ...

站点地图 | 文档上传 | 侵权投诉 | 手机版
本站所有资源均来自互联网,本站只负责收集和整理,均不承担任何法律责任,如有侵权等其它行为请联系我们.
文档下载 Copyright 2013 doc.guandang.net All Rights Reserved.  -  email