python使用梯度下降算法实现一个多线性回归
2020-03-25 12:06
393 查看
python使用梯度下降算法实现一个多线性回归,供大家参考,具体内容如下
图示:
import pandas as pd import matplotlib.pylab as plt import numpy as np # Read data from csv pga = pd.read_csv("D:\python3\data\Test.csv") # Normalize the data 归一化值 (x - mean) / (std) pga.AT = (pga.AT - pga.AT.mean()) / pga.AT.std() pga.V = (pga.V - pga.V.mean()) / pga.V.std() pga.AP = (pga.AP - pga.AP.mean()) / pga.AP.std() pga.RH = (pga.RH - pga.RH.mean()) / pga.RH.std() pga.PE = (pga.PE - pga.PE.mean()) / pga.PE.std() def cost(theta0, theta1, theta2, theta3, theta4, x1, x2, x3, x4, y): # Initialize cost J = 0 # The number of observations m = len(x1) # Loop through each observation # 通过每次观察进行循环 for i in range(m): # Compute the hypothesis # 计算假设 h=theta0+x1[i]*theta1+x2[i]*theta2+x3[i]*theta3+x4[i]*theta4 # Add to cost J += (h - y[i])**2 # Average and normalize cost J /= (2*m) return J # The cost for theta0=0 and theta1=1 def partial_cost_theta4(theta0,theta1,theta2,theta3,theta4,x1,x2,x3,x4,y): h = theta0 + x1 * theta1 + x2 * theta2 + x3 * theta3 + x4 * theta4 diff = (h - y) * x4 partial = diff.sum() / (x2.shape[0]) return partial def partial_cost_theta3(theta0,theta1,theta2,theta3,theta4,x1,x2,x3,x4,y): h = theta0 + x1 * theta1 + x2 * theta2 + x3 * theta3 + x4 * theta4 diff = (h - y) * x3 partial = diff.sum() / (x2.shape[0]) return partial def partial_cost_theta2(theta0,theta1,theta2,theta3,theta4,x1,x2,x3,x4,y): h = theta0 + x1 * theta1 + x2 * theta2 + x3 * theta3 + x4 * theta4 diff = (h - y) * x2 partial = diff.sum() / (x2.shape[0]) return partial def partial_cost_theta1(theta0,theta1,theta2,theta3,theta4,x1,x2,x3,x4,y): h = theta0 + x1 * theta1 + x2 * theta2 + x3 * theta3 + x4 * theta4 diff = (h - y) * x1 partial = diff.sum() / (x2.shape[0]) return partial # 对theta0 进行求导 # Partial derivative of cost in terms of theta0 def partial_cost_theta0(theta0, theta1, theta2, theta3, theta4, x1, x2, x3, x4, y): h = theta0 + x1 * theta1 + x2 * theta2 + x3 * theta3 + x4 * theta4 diff = (h - y) partial = diff.sum() / (x2.shape[0]) return partial def gradient_descent(x1,x2,x3,x4,y, alpha=0.1, theta0=0, theta1=0,theta2=0,theta3=0,theta4=0): max_epochs = 1000 # Maximum number of iterations 最大迭代次数 counter = 0 # Intialize a counter 当前第几次 c = cost(theta0, theta1, theta2, theta3, theta4, x1, x2, x3, x4, y) ## Initial cost 当前代价函数 costs = [c] # Lets store each update 每次损失值都记录下来 # Set a convergence threshold to find where the cost function in minimized # When the difference between the previous cost and current cost # is less than this value we will say the parameters converged # 设置一个收敛的阈值 (两次迭代目标函数值相差没有相差多少,就可以停止了) convergence_thres = 0.000001 cprev = c + 10 theta0s = [theta0] theta1s = [theta1] theta2s = [theta2] theta3s = [theta3] theta4s = [theta4] # When the costs converge or we hit a large number of iterations will we stop updating # 两次间隔迭代目标函数值相差没有相差多少(说明可以停止了) while (np.abs(cprev - c) > convergence_thres) and (counter < max_epochs): cprev = c # Alpha times the partial deriviative is our updated # 先求导, 导数相当于步长 update0 = alpha * partial_cost_theta0(theta0, theta1, theta2, theta3, theta4, x1, x2, x3, x4, y) update1 = alpha * partial_cost_theta1(theta0, theta1, theta2, theta3, theta4, x1, x2, x3, x4, y) update2 = alpha * partial_cost_theta2(theta0, theta1, theta2, theta3, theta4, x1, x2, x3, x4, y) update3 = alpha * partial_cost_theta3(theta0, theta1, theta2, theta3, theta4, x1, x2, x3, x4, y) update4 = alpha * partial_cost_theta4(theta0, theta1, theta2, theta3, theta4, x1, x2, x3, x4, y) # Update theta0 and theta1 at the same time # We want to compute the slopes at the same set of hypothesised parameters # so we update after finding the partial derivatives # -= 梯度下降,+=梯度上升 theta0 -= update0 theta1 -= update1 theta2 -= update2 theta3 -= update3 theta4 -= update4 # Store thetas theta0s.append(theta0) theta1s.append(theta1) theta2s.append(theta2) theta3s.append(theta3) theta4s.append(theta4) # Compute the new cost # 当前迭代之后,参数发生更新 c = cost(theta0, theta1, theta2, theta3, theta4, x1, x2, x3, x4, y) # Store updates,可以进行保存当前代价值 costs.append(c) counter += 1 # Count # 将当前的theta0, theta1, costs值都返回去 #return {'theta0': theta0, 'theta1': theta1, 'theta2': theta2, 'theta3': theta3, 'theta4': theta4, "costs": costs} return {'costs':costs} print("costs =", gradient_descent(pga.AT, pga.V,pga.AP,pga.RH,pga.PE)['costs']) descend = gradient_descent(pga.AT, pga.V,pga.AP,pga.RH,pga.PE, alpha=.01) plt.scatter(range(len(descend["costs"])), descend["costs"]) plt.show()
损失函数随迭代次数变换图:
以上就是本文的全部内容,希望对大家的学习有所帮助
您可能感兴趣的文章:
相关文章推荐
- 线性回归的python实现
- 线性回归python实现
- 记录xgboost python 线性回归使用方法
- 吴恩达机器学习线性回归练习题:多变量线性回归(python实现)
- [置顶] 【算法 机器学习】MATLAB、R、python三种编程语言实现简单线性回归算法比较
- 机器学习之线性回归 Linear Regression(二)Python实现
- Python使用sklearn实现的各种回归算法示例
- 神经网络与深度学习 使用Python实现基于梯度下降算法的神经网络和自制仿MNIST数据集的手写数字分类可视化程序 web版本
- 机器学习之线性回归python实现
- 逻辑回归-线性判定边界Python代码实现
- Python实现的简单线性回归算法实例分析
- 局部加权线性回归及岭回归之Python实现
- 神经网络与深度学习 1.6 使用Python实现基于梯度下降算法的神经网络和MNIST数据集的手写数字分类程序
- 多元线性回归的python实现
- 简单线性回归的Python实现
- 线性逻辑回归的Python实现
- python实现简单线性回归
- 线性回归与岭回归python代码实现
- 线性回归总结及python实现
- 【Stanford|斯坦福-机器学习:线性回归-单特征梯度下降+动态图】python3实现