斯坦福大学-回归规则化处理_Exercise Code
2014-04-13 14:19
211 查看
Regularization 规则化(过拟合处理方法:一是减少特征,二是规则化)
http://openclassroom.stanford.edu/MainFolder/DocumentPage.php?course=MachineLearning&doc=exercises/ex5/ex5.html
clear all; close all; clc
x = load('ex5Logx.dat');
y = load('ex5Logy.dat');
% Plot the training data
% Use different markers for positives and negatives
figure
pos = find(y); neg = find(y == 0);
plot(x(pos, 1), x(pos, 2), 'k+','LineWidth', 2, 'MarkerSize', 7)
hold on
plot(x(neg, 1), x(neg, 2), 'ko', 'MarkerFaceColor', 'y', 'MarkerSize', 7)
% Add polynomial features to x by
% calling the feature mapping function
% provided in separate m-file
x = map_feature(x(:,1), x(:,2));
[m, n] = size(x);
% Initialize fitting parameters
theta = zeros(n, 1);
% Define the sigmoid function
g = inline('1.0 ./ (1.0 + exp(-z))');
% setup for Newton's method
MAX_ITR = 15;
J = zeros(MAX_ITR, 1);
% Lambda is the regularization parameter
lambda = 0;
% Newton's Method
for i = 1:MAX_ITR
% Calculate the hypothesis function
z = x * theta;
h = g(z);
% Calculate J (for testing convergence)
J(i) =(1/m)*sum(-y.*log(h) - (1-y).*log(1-h))+ ...
(lambda/(2*m))*norm(theta([2:end]))^2;
% Calculate gradient and hessian.
G = (lambda/m).*theta; G(1) = 0; % extra term for gradient
L = (lambda/m).*eye(n); L(1) = 0;% extra term for Hessian
grad = ((1/m).*x' * (h-y)) + G;
H = ((1/m).*x' * diag(h) * diag(1-h) * x) + L;
% Here is the actual update
theta = theta - H\grad;
end
% Show J to determine if algorithm has converged
J
% display the norm of our parameters
norm_theta = norm(theta)
% Plot the results
% We will evaluate theta*x over a
% grid of features and plot the contour
% where theta*x equals zero
% Here is the grid range
u = linspace(-1, 1.5, 200);
v = linspace(-1, 1.5, 200);
z = zeros(length(u), length(v));
% Evaluate z = theta*x over the grid
for i = 1:length(u)
for j = 1:length(v)
z(i,j) = map_feature(u(i), v(j))*theta;
end
end
z = z'; % important to transpose z before calling contour
% Plot z = 0
% Notice you need to specify the range [0, 0]
contour(u, v, z, [0, 0], 'LineWidth', 2)
legend('y = 1', 'y = 0', 'Decision boundary')
title(sprintf('\\lambda = %g', lambda), 'FontSize', 14)
hold off
% Uncomment to plot J
% figure
% plot(0:MAX_ITR-1, J, 'o--', 'MarkerFaceColor', 'r', 'MarkerSize', 8)
% xlabel('Iteration'); ylabel('J')
http://openclassroom.stanford.edu/MainFolder/DocumentPage.php?course=MachineLearning&doc=exercises/ex5/ex5.html
clear all; close all; clc
x = load('ex5Logx.dat');
y = load('ex5Logy.dat');
% Plot the training data
% Use different markers for positives and negatives
figure
pos = find(y); neg = find(y == 0);
plot(x(pos, 1), x(pos, 2), 'k+','LineWidth', 2, 'MarkerSize', 7)
hold on
plot(x(neg, 1), x(neg, 2), 'ko', 'MarkerFaceColor', 'y', 'MarkerSize', 7)
% Add polynomial features to x by
% calling the feature mapping function
% provided in separate m-file
x = map_feature(x(:,1), x(:,2));
[m, n] = size(x);
% Initialize fitting parameters
theta = zeros(n, 1);
% Define the sigmoid function
g = inline('1.0 ./ (1.0 + exp(-z))');
% setup for Newton's method
MAX_ITR = 15;
J = zeros(MAX_ITR, 1);
% Lambda is the regularization parameter
lambda = 0;
% Newton's Method
for i = 1:MAX_ITR
% Calculate the hypothesis function
z = x * theta;
h = g(z);
% Calculate J (for testing convergence)
J(i) =(1/m)*sum(-y.*log(h) - (1-y).*log(1-h))+ ...
(lambda/(2*m))*norm(theta([2:end]))^2;
% Calculate gradient and hessian.
G = (lambda/m).*theta; G(1) = 0; % extra term for gradient
L = (lambda/m).*eye(n); L(1) = 0;% extra term for Hessian
grad = ((1/m).*x' * (h-y)) + G;
H = ((1/m).*x' * diag(h) * diag(1-h) * x) + L;
% Here is the actual update
theta = theta - H\grad;
end
% Show J to determine if algorithm has converged
J
% display the norm of our parameters
norm_theta = norm(theta)
% Plot the results
% We will evaluate theta*x over a
% grid of features and plot the contour
% where theta*x equals zero
% Here is the grid range
u = linspace(-1, 1.5, 200);
v = linspace(-1, 1.5, 200);
z = zeros(length(u), length(v));
% Evaluate z = theta*x over the grid
for i = 1:length(u)
for j = 1:length(v)
z(i,j) = map_feature(u(i), v(j))*theta;
end
end
z = z'; % important to transpose z before calling contour
% Plot z = 0
% Notice you need to specify the range [0, 0]
contour(u, v, z, [0, 0], 'LineWidth', 2)
legend('y = 1', 'y = 0', 'Decision boundary')
title(sprintf('\\lambda = %g', lambda), 'FontSize', 14)
hold off
% Uncomment to plot J
% figure
% plot(0:MAX_ITR-1, J, 'o--', 'MarkerFaceColor', 'r', 'MarkerSize', 8)
% xlabel('Iteration'); ylabel('J')
相关文章推荐
- 斯坦福大学-线性回归_Exercise Code
- 机器学习基础 维基翻译 保序回归 随机森林 Pipeline处理 及简单的sklearn例子 分类:机器学习Sklearn
- WebKit自动化回归测试之LayoutTest实践 (超时问题的处理)
- 斯坦福大学(吴恩达) 机器学习课后习题详解 第三周 编程题 逻辑回归
- 斯坦福大学-多元线性回归_Exercise Code
- 逻辑回归处理离散变量
- 斯坦福大学机器学习课程笔记:线性回归以及梯度下降
- Activity嵌套多层Fragment 带有回归栈时返回键处理
- IE 11 注册事件处理程序终于回归W3C标准addEventListener
- 斯坦福大学-Logistic回归_Exercise Code
- Machine Learning(Stanford)| 斯坦福大学机器学习笔记--第一周(2.线性回归,代价函数)
- 斯坦福大学机器学习笔记——特征和多项式回归以及正规方程
- 斯坦福大学-朴素贝叶斯_Exercise Code
- 斯坦福机器学习公开课6-x 使用逻辑回归处理多分类
- 【数据清洗】数据异常处理之泰尔森回归
- 斯坦福大学机器学习课程二线性回归编程作业1
- 斯坦福大学机器学习课程笔记: 逻辑回归以及梯度下降
- 斯坦福大学机器学习课程二线性回归编程作业2
- 机器学习入门系列二(关键词:多变量(非)线性回归,批处理,特征缩放,正规方程
- Logistic回归,处理MNIST,举例。