文档下载 文档下载导航
设为首页 | 加入收藏
搜索 请输入内容:  
 导航当前位置: 文档下载 > 所有分类 > 经管营销 > The Bias Variance Tradeoff
侵权投诉

The Bias Variance Tradeoff

The Bias-Variance Tradeoff

Given:

- the true function we want to approximate

f=f(x)

- the data set for training

D={(x1,t1),(x2,t2),…,(xN,tN)} where t=f+ε and E{ε}=0

- given D, we train an arbitrary neural network to approximate the function f

by

y=g(x,w)The mean-squared error of this networks is:

1N2

MSE=∑(ti yi)

Ni=1

To assess the effectiveness of the network, we want to know the expectation ofthe MSE if we test the network on arbitrarily many test points drawn from theunknown function.

1N1N2 2

E{MSE}=E ∑(ti yi) =∑E(ti yi)

Ni=1 Ni=1

Let’s investigate the expectation inside the sum, with a little “augmentationtrick”:

{}

E(ti yi)

{

2

}{}

=E{(t f)}+E{(f y)}+2E{(f y)(t f)}

=E{ε}+E{(f y)}+2(E{ft} E{f} E{yt}+E{yf})

=E(ti fi+fi yi)

2

i

i

2

2

i

i

i

i

i

i

2

2

2

i

i

ii

i

ii

ii

Note: E{fiti}=fi2 since f is deterministic and E{ti}=fi

:E{fi2}=fi2 since f is deterministic

:E{yiti}=E{yi(fi+ε)}=E{yifi+yiε}=E{yifi}+0

: (the last term is zero because the noise in the infinite test set over which:we take the expectation is probabilistically independent of the NN

:prediction). Thus the last term in the expectation above cancels to zero.

E(ti yi)

{

2

}

=E{ε2}+E(fi yi)

{

2

}

第1页

你可能喜欢

热门文档

相关文档

  • Conceptual complexity and the biasvariance tradeoff

    2–16 Contents lists available at ScienceDirect Cognition journal homepage: www.elsevier.com/locate/COGNIT Conceptual complexity and the bias/variance trade...

  • 支持向量机指南

    对最初 SVMs 发展的驱动发生在 guises-the bias variance tradeoff、学习能力控制,过学 习等——但是基本思想是相同的。粗略的说,对于一个给定的,只有有限训练...

  • Suboptimality of the Truncated SVD for Ill-Conditio...

    This paper analyzes the bias-variance tradeoff of a class of l隐藏>> Suboptimality of the Truncated SVD for Ill-Conditioned Inverse Problems Jeffrey A. ...

  • ...Bias for Smooth Cutsize-Delay Tradeoff in Perfor...

    The Bias Variance Tradeo... 暂无评价 2页 1财富值 Individual investors ... Local Unidirectional Bias for Smooth Cutsize-Delay Tradeoff in Performance-...

  • 支持向量机指南

    对最初 SVMs 发展的驱动发生在 guises-the bias variance tradeoff、学习能力控制,过学 习等——但是基本思想是相同的。粗略的说,对于一个给定的,只有有限训练...

  • Achievable Regions in the Bias-Variance Plane for P...

    Achievable Regions in the Bias-Variance Plane for Parametric Estimation Problems 1 Alfred H Abstract--- In this paper we use the uniform Cramer-Rao (...

  • Outline of presentation

    er Practical algorithms face the bias–variance tradeoff 4 Feature subset selection (2) ? Optimal feature subset de?ned with respect to the induction ...

  • 基于svm的模式识别

    Statistical Learning The problem which drove the initial development of SVMs occurs in several guises –the bias variance tradeoff,capacity control and over ...

  • An application of the bootstrap 632+ rule to ecolog...

    A behavior typical of the bias-variance tradeoff is visible in Figure 2, where the resubstitution error and the relative over?tting rate estimate R are...

  • 机器学习Ch9-Boosting

    3 Fighting the bias-variance tradeoff 4 Voting (Ensemble Methods) 5 Voting (Ensemble Methods) 6 Boosting [Schapire’89] 7 Learning from weighted data ...

站点地图 | 文档上传 | 侵权投诉 | 手机版
本站所有资源均来自互联网,本站只负责收集和整理,均不承担任何法律责任,如有侵权等其它行为请联系我们.
文档下载 Copyright 2013 doc.guandang.net All Rights Reserved.  -  email