The Bias-Variance Tradeoff
Given:
- the true function we want to approximate
f=f(x)
- the data set for training
D={(x1,t1),(x2,t2),…,(xN,tN)} where t=f+ε and E{ε}=0
- given D, we train an arbitrary neural network to approximate the function f
by
y=g(x,w)The mean-squared error of this networks is:
1N2
MSE=∑(ti yi)
Ni=1
To assess the effectiveness of the network, we want to know the expectation ofthe MSE if we test the network on arbitrarily many test points drawn from theunknown function.
1N1N2 2
E{MSE}=E ∑(ti yi) =∑E(ti yi)
Ni=1 Ni=1
Let’s investigate the expectation inside the sum, with a little “augmentationtrick”:
{}
E(ti yi)
{
2
}{}
=E{(t f)}+E{(f y)}+2E{(f y)(t f)}
=E{ε}+E{(f y)}+2(E{ft} E{f} E{yt}+E{yf})
=E(ti fi+fi yi)
2
i
i
2
2
i
i
i
i
i
i
2
2
2
i
i
ii
i
ii
ii
Note: E{fiti}=fi2 since f is deterministic and E{ti}=fi
:E{fi2}=fi2 since f is deterministic
:E{yiti}=E{yi(fi+ε)}=E{yifi+yiε}=E{yifi}+0
: (the last term is zero because the noise in the infinite test set over which:we take the expectation is probabilistically independent of the NN
:prediction). Thus the last term in the expectation above cancels to zero.
E(ti yi)
{
2
}
=E{ε2}+E(fi yi)
{
2
}
2–16 Contents lists available at ScienceDirect Cognition journal homepage: www.elsevier.com/locate/COGNIT Conceptual complexity and the bias/variance trade...
对最初 SVMs 发展的驱动发生在 guises-the bias variance tradeoff、学习能力控制,过学 习等——但是基本思想是相同的。粗略的说,对于一个给定的,只有有限训练...
This paper analyzes the bias-variance tradeoff of a class of l隐藏>> Suboptimality of the Truncated SVD for Ill-Conditioned Inverse Problems Jeffrey A. ...
The Bias Variance Tradeo... 暂无评价 2页 1财富值 Individual investors ... Local Unidirectional Bias for Smooth Cutsize-Delay Tradeoff in Performance-...
对最初 SVMs 发展的驱动发生在 guises-the bias variance tradeoff、学习能力控制,过学 习等——但是基本思想是相同的。粗略的说,对于一个给定的,只有有限训练...
Achievable Regions in the Bias-Variance Plane for Parametric Estimation Problems 1 Alfred H Abstract--- In this paper we use the uniform Cramer-Rao (...
er Practical algorithms face the bias–variance tradeoff 4 Feature subset selection (2) ? Optimal feature subset de?ned with respect to the induction ...
Statistical Learning The problem which drove the initial development of SVMs occurs in several guises –the bias variance tradeoff,capacity control and over ...
A behavior typical of the bias-variance tradeoff is visible in Figure 2, where the resubstitution error and the relative over?tting rate estimate R are...
3 Fighting the bias-variance tradeoff 4 Voting (Ensemble Methods) 5 Voting (Ensemble Methods) 6 Boosting [Schapire’89] 7 Learning from weighted data ...
热门文档