Stanford机器学习笔记-5.神经网络Neural Networks (part two)
2016-04-07 00:03
253 查看
5 Neural Networks (part two)
content:5 Neural Networks (part two)
5.1 cost function
5.2 Back Propagation
5.3 神经网络总结
接上一篇4. Neural Networks (part one). 本文将先定义神经网络的代价函数,然后介绍逆向传播(Back Propagation: BP)算法,它能有效求解代价函数对连接权重的偏导,最后对训练神经网络的过程进行总结。
5.1 cost function
function [J grad] = nnCostFunction(nn_params, ... input_layer_size, ... hidden_layer_size, ... num_labels, ... X, y, lambda) %NNCOSTFUNCTION Implements the neural network cost function for a two layer %neural network which performs classification % [J grad] = NNCOSTFUNCTON(nn_params, hidden_layer_size, num_labels, ... % X, y, lambda) computes the cost and gradient of the neural network. The % parameters for the neural network are "unrolled" into the vector % nn_params and need to be converted back into the weight matrices. % % The returned parameter grad should be a "unrolled" vector of the % partial derivatives of the neural network. % % Reshape nn_params back into the parameters Theta1 and Theta2, the weight matrices % for our 2 layer neural network:Theta1: 1->2; Theta2: 2->3 Theta1 = reshape(nn_params(1:hidden_layer_size * (input_layer_size + 1)), ... hidden_layer_size, (input_layer_size + 1)); Theta2 = reshape(nn_params((1 + (hidden_layer_size * (input_layer_size + 1))):end), ... num_labels, (hidden_layer_size + 1)); % Setup some useful variables m = size(X, 1); J = 0; Theta1_grad = zeros(size(Theta1)); Theta2_grad = zeros(size(Theta2)); % Note: The vector y passed into the function is a vector of labels % containing values from 1..K. You need to map this vector into a % binary vector of 1's and 0's to be used with the neural network % cost function. for i = 1:m % compute activation by Forward Propagation a1 = [1; X(i,:)']; z2 = Theta1 * a1; a2 = [1; sigmoid(z2)]; z3 = Theta2 * a2; h = sigmoid(z3); yy = zeros(num_labels,1); yy(y(i)) = 1; % 训练集的真实值yy J = J + sum(-yy .* log(h) - (1-yy) .* log(1-h)); % Back Propagation delta3 = h - yy; delta2 = (Theta2(:,2:end)' * delta3) .* sigmoidGradient(z2); %注意要除去偏移单元的连接权重 Theta2_grad = Theta2_grad + delta3 * a2'; Theta1_grad = Theta1_grad + delta2 * a1'; end J = J / m + lambda * (sum(sum(Theta1(:,2:end) .^ 2)) + sum(sum(Theta2(:,2:end) .^ 2))) / (2*m); Theta2_grad = Theta2_grad / m; Theta2_grad(:,2:end) = Theta2_grad(:,2:end) + lambda * Theta2(:,2:end) / m; % regularized nn Theta1_grad = Theta1_grad / m; Theta1_grad(:,2:end) = Theta1_grad(:,2:end) + lambda * Theta1(:,2:end) / m; % regularized nn % Unroll gradients grad = [Theta1_grad(:) ; Theta2_grad(:)]; end
View Code
相关文章推荐
- httplib2---python下的http请求终结者(转)
- [10秒学会] - iOS 网络检测(如需要更详细 请用Reachability)
- Response.StatusCode的HTTP状态代码列表 【转】
- http状态码521——记一次网页爬取
- httpclient访问https网站失败(peer not authenticated)的解决办法
- BZOJ 3931 CQOI 2015 网络吞吐量 最大流 最短路
- VMWare的网络三种设置方式比较
- Tcp性能调优 解决Tcp长延时
- Meta http-equiv属性详解(转)
- Apache2.4中开通HTTP基本认证
- TCP/IP基础(四)
- linux网络配置命令之ifconfig、ip和route
- Ping测试网络连接状况
- 一次复杂而简单的网络诊断
- 牛客网每日打卡-网络基础-20160406
- [C#基础]网络编程(三):protobuf-net
- <meta>中的http-equiv的参数值
- https://github.com/oneuijs/You-Dont-Need-jQuery
- bzoj 1061: [Noi2008]志愿者招募(线性规划+网络流)
- hdu 5437 Alisha’s Party(长春网络赛——优先队列)