Neural style之torch学习笔记3:神经网络包的用法（1）

总说

torch7框架学习首先还是应该先看最基本的torch包中有什么。在我的前2篇博客torch7学习（一） 和torch7学习（二） 已经较为详细讲了。接下来要看的是神经网络包nn中的内容。可以说，当学习了神经网络包中的内容后，则能看懂给予torch框架的论文代码了，并且已经具备一定的用该框架书写神经网络的能力。

目的

看完这篇博客后，应该能进行最简单的自动训练方式。(⊙o⊙)…，我这样说感觉自己像是个写书的或是老师。 

下面将根据不同模块进行介绍

Overview

首先对整个nn包进行总览。 

神经网络是由不同模块组成。Module是抽象类，子类是container，而container又有三个构建神经网络最重要的子类：Sequential, Parallel和Concat，由这三类构成的神经网络中可以包含简单层,即Linear, Mean, Max和Reshape等，也包括卷积层还有激活函数Tanh,
ReLU等。这些都会在后面分别介绍。

当然你上面只是说了如何构建神经网络，那么怎么进行训练呢？首先你要有一个损失函数，在就是 
Criterion模块。损失函数众多，常用的由MSECriterion, ClassNLLCriterion，第一个就是均方误差MSE，第二个就是分类用的交叉熵。有了损失函数后，那么就要开始用算法进行训练了，训练的方式有很多种，有手动挡和自动挡。手动挡就是自己写如何更新权值，这个一般只针对只有简单层的网络，如果有卷积层的话，不太好写。自动挡就是直接调用内部的优化函数包optim来进行训练，当然了，推荐用自动挡啊。

初识Module

这里说的module是啥啊，就是模块啊，你把神经网络看成很多一小块一小块的模块组成的，有的小块是具有参数的，比如Linear或是卷积模块，有些是不带有参数的，比如Max或是Pooling。对于每个小块，总有输入吧，也有输出吧。那么对于那些具有参数的小块，则我们可以计算dLoss_dParams, 注意dLoss_dParams包含了2个，分别是dLoss_dWeight和dLoss_dBias。如果是不具有参数的小块，那么就没有dLoss_dParams了，不过二者都有dLoss_dInput和dLoss_dOutput。这个挺像matcovnet的思想的。如果没明白可以参考这个，Notes
on MatConvNet ( I ) — Overview

Module是抽象类，包含了4个主要的函数： 

1. forward(input) 根据输入计算该module的输出用的。 

2. backward(input, gradOutput) 这里的gradOutput就是就是dLoss_dOutput。有了gradOutput和input就可以进行反向传播。值得注意的是，这里内部会计算dLoss_dParams和dLoss_dInput。一般来说只返回dLoss_dInput。事实上，一般来说我们最终要的是整个网络的权值，但是这个不用我们操心，我们也不用管如何更新。只要调用这个函数，则内部会更新网络权值。 

3. zeroGradParameters() 这函数一般用于手动挡，自动挡一般用不到。后面会讲解 

4. updateParameters(learningRate) 和上面函数一起用于手动挡，后面会讲解。

只有两个成员变量:output, gradInput。

最简单的自动挡(GSD)训练

这个训练很简单啊。不过对数据集有要求。 

数据集有两点要求。首先dataset[index]返回第Index个数据。其次就是dataset:size()函数返回整个数据集的example的个数。 

example有两个属性。第一个就是data和label。如果是分类，那么label就是数字，如果是MSECriterion，那么label就是GroundTruth对应的图像矩阵。

然后你就可以用最简单的自动挡了！ 

训练模板：

<code class="language-lua hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">criterion = nn.MSECriterion()
trainer = nn.StochasticGradient(mlp,criterion)
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">-- setting trainer's params</span>
trainer.learningRate = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.001</span>
trainer.maxIteration = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">-- train</span>
trainer:train(dataset)</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li></ul>

训练神经网络共五步！ 

1. 加载数据。 这里要加入a:size()方法和运算符[]到trainset中，这两个是用最简单自动挡的前提！也要正规化数据。 

2. 定义网络 

3.定义损失函数 

4. 训练 

5. 测试 

这里简单的自动挡训练代码

<code class="language-lua hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">require</span> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'nn'</span>
<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">require</span> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'cunn'</span>
<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">require</span> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'torch'</span>
<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">require</span> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'cutorch'</span>

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">-- load data</span>
trainset = torch.<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">load</span>(<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'cifar10-train.t7'</span>)
testset = torch.<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">load</span>(<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'cifar10-test.t7'</span>)

<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">setmetatable</span>(trainset,{
__index = <span class="hljs-function" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">function</span><span class="hljs-params" style="color: rgb(102, 0, 102); box-sizing: border-box;">(t,i)</span></span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> {t.data[i], t.label[i]}
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">end</span>
});

<span class="hljs-function" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">function</span> <span class="hljs-title" style="box-sizing: border-box;">trainset:size</span><span class="hljs-params" style="color: rgb(102, 0, 102); box-sizing: border-box;">()</span></span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> self.data:size(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">end</span>

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">-- transfer data to GPU</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">-- Lua cannot handle ByteTensor</span>
trainset.data = trainset.data:cuda()
trainset.label = trainset.label:cuda()

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">-- normalize</span>
meanv = {}
stdv  = {}
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> i = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">do</span>
meanv[i] = trainset.data[ {{}, {i},{}, {}}]:mean()
trainset.data[ {{}, {i},{}, {}}]:add(-meanv[i])
<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">print</span>(<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'mean'</span>..i..<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'_'</span>..meanv[i])

stdv[i] = trainset.data[ {{}, {i},{}, {}}]:std()
trainset.data[ {{}, {i},{}, {}}]:div(stdv[i])
<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">print</span>(<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'std'</span>..i..<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'_'</span>..meanv[i])
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">end</span>

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">------------------ Define Network</span>

net = nn.Sequential()
net:add(nn.SpatialConvolution(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>)) <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">-- 3 input image channels, 6 output channels, 5x5 convolution kernel</span>
net:add(nn.ReLU())                       <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">-- non-linearity </span>
net:add(nn.SpatialMaxPooling(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>))     <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">-- A max-pooling operation that looks at 2x2 windows and finds the max.</span>
net:add(nn.SpatialConvolution(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>))
net:add(nn.ReLU())                       <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">-- non-linearity </span>
net:add(nn.SpatialMaxPooling(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>))
net:add(nn.View(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>*<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>*<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>))                    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">-- reshapes from a 3D tensor of 16x5x5 into 1D tensor of 16*5*5</span>
net:add(nn.Linear(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>*<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>*<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">120</span>))             <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">-- fully connected layer (matrix multiplication between input and weights)</span>
net:add(nn.ReLU())                       <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">-- non-linearity </span>
net:add(nn.Linear(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">120</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">84</span>))
net:add(nn.ReLU())                       <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">-- non-linearity </span>
net:add(nn.Linear(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">84</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span>))                   <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">-- 10 is the number of outputs of the network (in this case, 10 digits)</span>
net:add(nn.LogSoftMax())

<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">print</span>(net)

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">----------------Define criterion-----------</span>

criterion =  nn.ClassNLLCriterion()
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">---------------tranfer net and criterion to GPU-------</span>

net = net:cuda()
criterion = criterion:cuda()
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">-----------------StochasticGradient with Criterion--</span>
timer = torch.Timer()

trainer = nn.StochasticGradient(net, criterion)
trainer.learningRate = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.001</span>
trainer.maxIteration = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>
trainer:train(trainset)

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">-------------------test-------------</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">--Firstly, normalize testset</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> i = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">do</span>
testset.data[{{},{i},{},{}}]:add(-meanv[i])
testset.data[{{},{i},{},{}}]:div(stdv[i])

<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">end</span>

testset.data = testset.data:cuda()
testset.label = testset.label:cuda()

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">--calculate accuracies</span>
accuracies = {<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>}

<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> i=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10000</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">do</span>
predict = net:forward(trainset.data[i])
sampleLabel = trainset.label[i]
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">local</span> confidences , indice = torch.sort(predict,<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">true</span>)
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> indice[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>] == sampleLabel <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">then</span>
accuracies[sampleLabel] = accuracies[sampleLabel]+<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">end</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">end</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">-- start,end,step, different from matlab which with start, step, end</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> i =<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">do</span>
<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">print</span>(<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'accuracies: '</span>..i..<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'_'</span>..(accuracies[i])/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span> ..<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'%'</span>)
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">end</span>

<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">print</span>(<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'Elapsed time:'</span>..timer:time().real..<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'seconds'</span>)

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">--[[注意，如果用GPU的话，则3样东西一定要转换！ dataset, net和criterion.
否则，就就会出现问题。
]]</span>
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li><li style="box-sizing: border-box; padding: 0px 5px;">61</li><li style="box-sizing: border-box; padding: 0px 5px;">62</li><li style="box-sizing: border-box; padding: 0px 5px;">63</li><li style="box-sizing: border-box; padding: 0px 5px;">64</li><li style="box-sizing: border-box; padding: 0px 5px;">65</li><li style="box-sizing: border-box; padding: 0px 5px;">66</li><li style="box-sizing: border-box; padding: 0px 5px;">67</li><li style="box-sizing: border-box; padding: 0px 5px;">68</li><li style="box-sizing: border-box; padding: 0px 5px;">69</li><li style="box-sizing: border-box; padding: 0px 5px;">70</li><li style="box-sizing: border-box; padding: 0px 5px;">71</li><li style="box-sizing: border-box; padding: 0px 5px;">72</li><li style="box-sizing: border-box; padding: 0px 5px;">73</li><li style="box-sizing: border-box; padding: 0px 5px;">74</li><li style="box-sizing: border-box; padding: 0px 5px;">75</li><li style="box-sizing: border-box; padding: 0px 5px;">76</li><li style="box-sizing: border-box; padding: 0px 5px;">77</li><li style="box-sizing: border-box; padding: 0px 5px;">78</li><li style="box-sizing: border-box; padding: 0px 5px;">79</li><li style="box-sizing: border-box; padding: 0px 5px;">80</li><li style="box-sizing: border-box; padding: 0px 5px;">81</li><li style="box-sizing: border-box; padding: 0px 5px;">82</li><li style="box-sizing: border-box; padding: 0px 5px;">83</li><li style="box-sizing: border-box; padding: 0px 5px;">84</li><li style="box-sizing: border-box; padding: 0px 5px;">85</li><li style="box-sizing: border-box; padding: 0px 5px;">86</li><li style="box-sizing: border-box; padding: 0px 5px;">87</li><li style="box-sizing: border-box; padding: 0px 5px;">88</li><li style="box-sizing: border-box; padding: 0px 5px;">89</li><li style="box-sizing: border-box; padding: 0px 5px;">90</li><li style="box-sizing: border-box; padding: 0px 5px;">91</li><li style="box-sizing: border-box; padding: 0px 5px;">92</li><li style="box-sizing: border-box; padding: 0px 5px;">93</li><li style="box-sizing: border-box; padding: 0px 5px;">94</li><li style="box-sizing: border-box; padding: 0px 5px;">95</li><li style="box-sizing: border-box; padding: 0px 5px;">96</li><li style="box-sizing: border-box; padding: 0px 5px;">97</li><li style="box-sizing: border-box; padding: 0px 5px;">98</li><li style="box-sizing: border-box; padding: 0px 5px;">99</li><li style="box-sizing: border-box; padding: 0px 5px;">100</li><li style="box-sizing: border-box; padding: 0px 5px;">101</li><li style="box-sizing: border-box; padding: 0px 5px;">102</li><li style="box-sizing: border-box; padding: 0px 5px;">103</li><li style="box-sizing: border-box; padding: 0px 5px;">104</li><li style="box-sizing: border-box; padding: 0px 5px;">105</li><li style="box-sizing: border-box; padding: 0px 5px;">106</li><li style="box-sizing: border-box; padding: 0px 5px;">107</li><li style="box-sizing: border-box; padding: 0px 5px;">108</li><li style="box-sizing: border-box; padding: 0px 5px;">109</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li><li style="box-sizing: border-box; padding: 0px 5px;">61</li><li style="box-sizing: border-box; padding: 0px 5px;">62</li><li style="box-sizing: border-box; padding: 0px 5px;">63</li><li style="box-sizing: border-box; padding: 0px 5px;">64</li><li style="box-sizing: border-box; padding: 0px 5px;">65</li><li style="box-sizing: border-box; padding: 0px 5px;">66</li><li style="box-sizing: border-box; padding: 0px 5px;">67</li><li style="box-sizing: border-box; padding: 0px 5px;">68</li><li style="box-sizing: border-box; padding: 0px 5px;">69</li><li style="box-sizing: border-box; padding: 0px 5px;">70</li><li style="box-sizing: border-box; padding: 0px 5px;">71</li><li style="box-sizing: border-box; padding: 0px 5px;">72</li><li style="box-sizing: border-box; padding: 0px 5px;">73</li><li style="box-sizing: border-box; padding: 0px 5px;">74</li><li style="box-sizing: border-box; padding: 0px 5px;">75</li><li style="box-sizing: border-box; padding: 0px 5px;">76</li><li style="box-sizing: border-box; padding: 0px 5px;">77</li><li style="box-sizing: border-box; padding: 0px 5px;">78</li><li style="box-sizing: border-box; padding: 0px 5px;">79</li><li style="box-sizing: border-box; padding: 0px 5px;">80</li><li style="box-sizing: border-box; padding: 0px 5px;">81</li><li style="box-sizing: border-box; padding: 0px 5px;">82</li><li style="box-sizing: border-box; padding: 0px 5px;">83</li><li style="box-sizing: border-box; padding: 0px 5px;">84</li><li style="box-sizing: border-box; padding: 0px 5px;">85</li><li style="box-sizing: border-box; padding: 0px 5px;">86</li><li style="box-sizing: border-box; padding: 0px 5px;">87</li><li style="box-sizing: border-box; padding: 0px 5px;">88</li><li style="box-sizing: border-box; padding: 0px 5px;">89</li><li style="box-sizing: border-box; padding: 0px 5px;">90</li><li style="box-sizing: border-box; padding: 0px 5px;">91</li><li style="box-sizing: border-box; padding: 0px 5px;">92</li><li style="box-sizing: border-box; padding: 0px 5px;">93</li><li style="box-sizing: border-box; padding: 0px 5px;">94</li><li style="box-sizing: border-box; padding: 0px 5px;">95</li><li style="box-sizing: border-box; padding: 0px 5px;">96</li><li style="box-sizing: border-box; padding: 0px 5px;">97</li><li style="box-sizing: border-box; padding: 0px 5px;">98</li><li style="box-sizing: border-box; padding: 0px 5px;">99</li><li style="box-sizing: border-box; padding: 0px 5px;">100</li><li style="box-sizing: border-box; padding: 0px 5px;">101</li><li style="box-sizing: border-box; padding: 0px 5px;">102</li><li style="box-sizing: border-box; padding: 0px 5px;">103</li><li style="box-sizing: border-box; padding: 0px 5px;">104</li><li style="box-sizing: border-box; padding: 0px 5px;">105</li><li style="box-sizing: border-box; padding: 0px 5px;">106</li><li style="box-sizing: border-box; padding: 0px 5px;">107</li><li style="box-sizing: border-box; padding: 0px 5px;">108</li><li style="box-sizing: border-box; padding: 0px 5px;">109</li></ul>

结果如下：

<code class="language-lua hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">mean1_125<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.83173370361</span>
std1_125<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.83173370361</span>
mean2_123<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.26066589355</span>
std2_123<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.26066589355</span>
mean3_114<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0306854248</span>
std3_114<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0306854248</span>
nn.Sequential {
[input -> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>) -> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>) -> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>) -> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span>) -> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>) -> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span>) -> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7</span>) -> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span>) -> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">9</span>) -> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span>) -> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11</span>) -> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">12</span>) -> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>) -> output]
(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>): nn.SpatialConvolution(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span> -> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>x5)
(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>): nn.ReLU
(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>): nn.SpatialMaxPooling(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>x2, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)
(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span>): nn.SpatialConvolution(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span> -> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>x5)
(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>): nn.ReLU
(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span>): nn.SpatialMaxPooling(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>x2, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)
(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7</span>): nn.View(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">400</span>)
(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span>): nn.Linear(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">400</span> -> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">120</span>)
(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">9</span>): nn.ReLU
(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span>): nn.Linear(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">120</span> -> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">84</span>)
(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11</span>): nn.ReLU
(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">12</span>): nn.Linear(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">84</span> -> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span>)
(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>): nn.LogSoftMax
}
# StochasticGradient: training
# current <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">error</span> = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.2138581074953</span>
# current <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">error</span> = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.8626037351131</span>
# current <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">error</span> = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.6799513678432</span>
# current <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">error</span> = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.5545474862456</span>
# current <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">error</span> = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.4643953397989</span>
# StochasticGradient: you have reached the maximum number of iterations
# training <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">error</span> = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.4643953397989</span>
accuracies: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>_34<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.2</span>%
accuracies: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>_67%
accuracies: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>_30<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.6</span>%
accuracies: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span>_36<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.6</span>%
accuracies: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>_31<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.6</span>%
accuracies: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span>_25%
accuracies: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7</span>_68<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.2</span>%
accuracies: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span>_58%
accuracies: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">9</span>_72<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.7</span>%
accuracies: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span>_57<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.6</span>%
Elapsed time:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">42.098989009857</span>seconds </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li></ul>

到此，应该能进行最简单的手动挡GSD的训练。后面将会用optim包进行更加专业更加自定义的训练方式。