逻辑回归与梯度下降

逻辑回归与梯度下降

已知方程

假设函数：hθ(x(i))=g(θTx)=11+e−θTx

Cost函数：Cost(hθ(x),y)=ylog(hθ(x))+(1−y)log(1−hθ(x))

代价函数：

J(θ)=−1m∑i=1mCost(hθ(x(i)),y(i))=−1m∑i=1m[y(i)log(hθ(x(i)))+(1−y(i))log(1−hθ(x(i)))]

梯度下降公式：θj:=θj−α∂J(θ)∂θj

梯度下降推导

∂J(θ)∂θj∂Cost(hθ(x(i)),y(i))∂θj∂hθ(x(i))∂θj∂Cost(hθ(x(i)),y(i))∂θj∂J(θ)∂θj=−1m∑i=1mCost(hθ(x(i)),y(i))=[y(i)1hθ(x(i))−(1−y(i))11−hθ(x(i))]∂hθ(x(i))∂θj=∂11+e−θTx(i)∂θj=−1[1+e−θTx(i)]2⋅∂[1+e−θTx(i)]∂θj=−1[1+e−θTx(i)]2⋅(−e−θTx(i)⋅∂θTx(i)∂θj)=1[1+e−θTx(i)]2⋅e−θTx(i)⋅x(i)j=11+e−θTx(i)⋅(1−11+e−θTx(i))⋅x(i)j=hθ(x(i))⋅(1−hθ(x(i)))⋅x(i)j=[y(i)1hθ(x(i))−(1−y(i))11−hθ(x(i))]∂hθ(x(i))∂θj=[y(i)1hθ(x(i))−(1−y(i))11−hθ(x(i))]hθ(x(i))⋅(1−hθ(x(i)))⋅x(i)j=[y(i)(1−hθ(x(i)))−(1−y(i))hθ(x(i))]⋅x(i)j=−[hθ(x(i))−y(i)]⋅x(i)j=1m∑i=1m[hθ(x(i))−y(i)]⋅x(i)j