Note for video Machine Learning and Data Mining——error and noise

Here is the note for lecture four.

error measures

When it comes to final hypothesis, we hope H(hypothesis) approximate f(target) as well as possible.

It can be written as

H ≈ f

Error measures could be defined as

E = 1/n ∑(f(xi)-H(xi))

If xi belongs to x1...xn, that means xi is one of the sample. Then E => Ein, which means sample's error measures.

And there will be a variable Eout, which means ensemble's error measures.

Generally, We hope Eout to be minimize, approximate zero. But it is impossible to get a real Eout.To make hypothesis better,

firstly, we could minimize Ein. And then make Ein approximate Eout, that's OK.

However, there will be a problem. If the model becomes complex to minimize Ein, Ein tracks Eout will much more loosely than it used to.

So remember two points,

1. Minimize Ein

2. Ein is close enough to Eout

Noisy Targets

In reality, when it comes to a problem, identical conditions sometimes have different results. This leads to Noise.

Because of this problem, there will be some change in target function.
So instead of target function y=f(x), we use

target distribution

P (y|x)

Here most of y is possible and rest is impossible. So we can define noise with y - E(y|x).