理解梯度下降，随机梯度下降，附电影推荐系统的简单代码小样 2

这是这一title 的下半部分，主要是因为这个浏览器好像缓存不了那么多东西，所以写到某一个临界点的时候，总是崩溃，要死了我都。

最后一部分，老师给了八十万行的数据，让我们自行处理，本来是要按照上面的代码处理一下就好了，我自己写了个三维的图。

import pandas as pd

#three dimensions, x is item y is rating z is the num of people who rating this item

#axis x

x = np.array(list(set(Y4.item)))#could be considered as the name of movie

#axis y

rating_mean = Y4.rating.mean()

#Y4 represents the original dataframe (because there was a same Y as the question 3, I have changed it to make sure that there is no relation between these two questions)

Y4.rating -= Y4.rating.mean()

#get ratings

y = pd.DataFrame(np.linspace(rating_mean,rating_mean,x.shape[0]+1))# because we need to drop one column we need to add extra column

#gY.index=x

y = y.drop(0)#don't need to get a forloop to update the index, delete the column 0 directly

#get user 943

users = np.array(range(1,944))

def movie_stochastic_gradient(Y4, y):

    gy = pd.DataFrame(np.zeros(y.shape), index=y.index)

    random_user = users[np.random.randint(users.shape[0]-1)]#is the same as 'np.random.randint(users.size)'

    items = list(set(Y4.item[Y4.user==random_user]))

    #items = list(Y4.item[Y4.user==1])

    #print(items)

    #get all the ratings from this user (there are some same nums)

    Y4_newform = Y4[Y4.user==random_user]#get a new form only belonged to this random_user and then we could easily get the rating

    for item in items:

        rating = list(set(Y4_newform.rating[Y4_newform.item==item]))[0]# in this form the same items and ratings have repeated several times

        #print(y[item])

        

        gy[0][gy.index==item] +=2*(y[0][y.index==item] - rating)

    return gy

learning_rate = 0.01

iterations = 100

for i in range(iterations):

    gy = movie_stochastic_gradient(Y4, y)

    print('^_^ We have iterated', i, 'times.')

    y -= learning_rate*gy

    #print(y)

# axis z the num of users who rated the same film

z = np.zeros(x.shape) # index means the name of film, z[x] means one movie was rated for x times

for item in Y4.item:

    z[item-1] += 1

#show the 3d map

import pylab as py

import mpl_toolkits.mplot3d.axes3d as p3 

fig = py.figure()

ax = p3.Axes3D(fig)

ax.scatter(x,y,z)

ax.set_xlabel('film')

ax.set_ylabel('rating')

ax.set_zlabel('users_num')

fig.add_axes(ax)

py.show()

主要选取了三个变量，电影的名字，电影被评论的次数以及电影受到用户影响之后的评分。

最后的结果大概是这个样子：