[ML of Andrew Ng]Week 1 : Linear Regression with One Variable

Week 1 Linear Regression with One Variable

Week 1 Linear Regression with One Variable
Introduction
Two definitions of ML

Two types of ML

Prerequisite for this course

Linear Regression with One Variable
The Hypothesis Function

Cost Function

Gradient Descent

Gradient Descent for Linear Regression

Introduction

Two definitions of ML

The field of study that gives computers the ability to learn without being explicitly programmed.

A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.

Two types of ML

Supervised Learning and Unsupervised Learning.

The keys of supervised learning are the correct output and labeled examples.

While data in Unsupervised learning has no labels,they are same.We don’t know what to do.

Prerequisite for this course

Knowledge of basic computer science principles and skills, at a level sufficient to write a reasonably non-trivial computer program.

Familiarity with the basic probability theory. (CS109 or Stat116 is sufficient but not necessary.)

Familiarity with the basic linear algebra (any one of Math 51, Math 103, Math 113, or CS 205 would be much more than necessary.)

Linear Regression with One Variable

The Hypothesis Function

hθ(x)=θ0+θ1x

or:

hθ(x)=θ0x0+θ1x1whichx0=1

PS: x(i)j means the jrd features and ird examples.

we can get the vectors θ and X as:

θ=[θ0θ1] (2×1)

and:

X=⎡⎣⎢⎢⎢⎢⎢11⋮1x(1)x(2)⋮x(m)⎤⎦⎥⎥⎥⎥⎥ (m×2)

So we get the H=Xθ (m×2)×(2×1)=(m×1) like as:

H=⎡⎣⎢⎢⎢⎢⎢hθ(x(1))hθ(x(2))⋮hθ(x(m))⎤⎦⎥⎥⎥⎥⎥ (m×1)

In matlab:

h = X*theta;

Cost Function

J(θ0,θ1)=12m∑i=1m(hθ(x(i))−y(i))2

Attention: J(θ0,θ1)　is a scalar, just a number.

In matlab, we can use like:

J = 1/(2*m) * sum((X*theta - y).^2);
%.^ means dot product
%sum means sum all elements in matrix

Gradient Descent

θj=θj−α∂∂θjJ(θ0,θ1)

repeat until convergence.

parameter α: Learning rate

parameter ∂: Slope of tangent aka derivative

Gradient Descent for Linear Regression

When specifically applied to the case of linear regression, a new form of the gradient descent equation can be derived.

θj=θj−α1m∑i=1m((hθ(x(i)j)−y(i))x(i)j)

In matlab, we can use like:

theta = theta - (alpha/m * X' * (X*theta - y));
%because of the matrix multiplication, we  need not sum them.