卷积神经网络(CNN)的简单实现(MNIST)
2016-03-06 20:42
513 查看
卷积神经网络(CNN)的基础介绍见http://blog.csdn.net/fengbingchun/article/details/50529500，这里主要以代码实现为主。
CNN是一个多层的神经网络，每层由多个二维平面组成，而每个平面由多个独立神经元组成。
以MNIST作为数据库，仿照LeNet-5和tiny-cnn( http://blog.csdn.net/fengbingchun/article/details/50573841 )
设计一个简单的7层CNN结构如下：
输入层Input：神经元数量32*32=1024；
C1层：卷积窗大小5*5，输出特征图数量6，卷积窗种类6，输出特征图大小28*28，可训练参数(权值+阈值(偏置))5*5*6+6=150+6，神经元数量28*28*6=4704；
S2层：卷积窗大小2*2，输出下采样图数量6，卷积窗种类6，输出下采样图大小14*14，可训练参数1*6+6=6+6，神经元数量14*14*6=1176；
C3层：卷积窗大小5*5，输出特征图数量16，卷积窗种类6*16=96，输出特征图大小10*10，可训练参数5*5*(6*16)+16=2400+16，神经元数量10*10*16=1600；
S4层：卷积窗大小2*2，输出下采样图数量16，卷积窗种类16，输出下采样图大小5*5，可训练参数1*16+16=16+16，神经元数量5*5*16=400；
C5层：卷积窗大小5*5，输出特征图数量120，卷积窗种类16*120=1920，输出特征图大小1*1，可训练参数5*5*(16*120)+120=48000+120，神经元数量1*1*120=120；
输出层Output：卷积窗大小1*1，输出特征图数量10，卷积窗种类120*10=1200，输出特征图大小1*1，可训练参数1*(120*10)+10=1200+10，神经元数量1*1*10=10。
下面对实现执行过程进行描述说明：
1. 从MNIST数据库中分别获取训练样本和测试样本数据：
(1)、原有MNIST库中图像大小为28*28，这里缩放为32*32，数据值范围为[-1,1]，扩充值均取-1；总共60000个32*32训练样本，10000个32*32测试样本；
(2)、输出层有10个输出节点，在训练阶段，对应位置的节点值设为0.8，其它节点设为-0.8.
2. 初始化权值和阈值(偏置)：权值就是卷积图像，每一个特征图上的神经元共享相同的权值和阈值，特征图的数量等于阈值的个数
(1)、权值采用uniform rand的方法初始化；
(2)、阈值均初始化为0.
3. 前向传播：根据权值和阈值，主要计算每层神经元的值
(1)、输入层：每次输入一个32*32数据。
(2)、C1层：分别用每一个5*5的卷积图像去乘以32*32的图像，获得一个28*28的图像，即对应位置相加再求和，stride长度为1；一共6个5*5的卷积图像，然后对每一个神经元加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
(3)、S2层：对C1中6个28*28的特征图生成6个14*14的下采样图，相邻四个神经元分别进行相加求和，然后乘以一个权值，再求均值即除以4，然后再加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
(4)、C3层：由S2中的6个14*14下采样图生成16个10*10特征图，对于生成的每一个10*10的特征图，是由6个5*5的卷积图像去乘以6个14*14的下采样图，然后对应位置相加求和，然后对每一个神经元加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
(5)、S4层：由C3中16个10*10的特征图生成16个5*5下采样图，相邻四个神经元分别进行相加求和，然后乘以一个权值，再求均值即除以4，然后再加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
(6)、C5层：由S4中16个5*5下采样图生成120个1*1特征图，对于生成的每一个1*1的特征图，是由16个5*5的卷积图像去乘以16个5*5的下采用图，然后相加求和，然后对每一个神经元加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
(7)、输出层：即全连接层，输出层中的每一个神经元均是由C5层中的120个神经元乘以相对应的权值，然后相加求和；然后对每一个神经元加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
4. 反向传播：主要计算每层神经元、权值和阈值的误差，以用来更新权值和阈值
(1)、输出层：计算输出层神经元误差；通过mse损失函数的导数函数和tanh激活函数的导数函数来计算输出层神经元误差。
(2)、C5层：计算C5层神经元误差、输出层权值误差、输出层阈值误差；通过输出层神经元误差乘以输出层权值，求和，结果再乘以C5层神经元的tanh激活函数的导数，获得C5层每一个神经元误差；通过输出层神经元误差乘以C5层神经元获得输出层权值误差；输出层误差即为输出层阈值误差。
(3)、S4层：计算S4层神经元误差、C5层权值误差、C5层阈值误差；通过C5层权值乘以C5层神经元误差，求和，结果再乘以S4层神经元的tanh激活函数的导数，获得S4层每一个神经元误差；通过S4层神经元乘以C5层神经元误差，求和，获得C5层权值误差；C5层神经元误差即为C5层阈值误差。
(4)、C3层：计算C3层神经元误差、S4层权值误差、S4层阈值误差；
(5)、S2层：计算S2层神经元误差、C3层权值误差、C3层阈值误差；
(6)、C1层：计算C1层神经元误差、S2层权值误差、S2层阈值误差；
(7)、输入层：计算C1层权值误差、C1层阈值误差.
代码文件：
CNN.hpp：
CNN.cpp：
CNN是一个多层的神经网络，每层由多个二维平面组成，而每个平面由多个独立神经元组成。
以MNIST作为数据库，仿照LeNet-5和tiny-cnn( http://blog.csdn.net/fengbingchun/article/details/50573841 )
设计一个简单的7层CNN结构如下：
输入层Input：神经元数量32*32=1024；
C1层：卷积窗大小5*5，输出特征图数量6，卷积窗种类6，输出特征图大小28*28，可训练参数(权值+阈值(偏置))5*5*6+6=150+6，神经元数量28*28*6=4704；
S2层：卷积窗大小2*2，输出下采样图数量6，卷积窗种类6，输出下采样图大小14*14，可训练参数1*6+6=6+6，神经元数量14*14*6=1176；
C3层：卷积窗大小5*5，输出特征图数量16，卷积窗种类6*16=96，输出特征图大小10*10，可训练参数5*5*(6*16)+16=2400+16，神经元数量10*10*16=1600；
S4层：卷积窗大小2*2，输出下采样图数量16，卷积窗种类16，输出下采样图大小5*5，可训练参数1*16+16=16+16，神经元数量5*5*16=400；
C5层：卷积窗大小5*5，输出特征图数量120，卷积窗种类16*120=1920，输出特征图大小1*1，可训练参数5*5*(16*120)+120=48000+120，神经元数量1*1*120=120；
输出层Output：卷积窗大小1*1，输出特征图数量10，卷积窗种类120*10=1200，输出特征图大小1*1，可训练参数1*(120*10)+10=1200+10，神经元数量1*1*10=10。
下面对实现执行过程进行描述说明：
1. 从MNIST数据库中分别获取训练样本和测试样本数据：
(1)、原有MNIST库中图像大小为28*28，这里缩放为32*32，数据值范围为[-1,1]，扩充值均取-1；总共60000个32*32训练样本，10000个32*32测试样本；
(2)、输出层有10个输出节点，在训练阶段，对应位置的节点值设为0.8，其它节点设为-0.8.
2. 初始化权值和阈值(偏置)：权值就是卷积图像，每一个特征图上的神经元共享相同的权值和阈值，特征图的数量等于阈值的个数
(1)、权值采用uniform rand的方法初始化；
(2)、阈值均初始化为0.
3. 前向传播：根据权值和阈值，主要计算每层神经元的值
(1)、输入层：每次输入一个32*32数据。
(2)、C1层：分别用每一个5*5的卷积图像去乘以32*32的图像，获得一个28*28的图像，即对应位置相加再求和，stride长度为1；一共6个5*5的卷积图像，然后对每一个神经元加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
(3)、S2层：对C1中6个28*28的特征图生成6个14*14的下采样图，相邻四个神经元分别进行相加求和，然后乘以一个权值，再求均值即除以4，然后再加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
(4)、C3层：由S2中的6个14*14下采样图生成16个10*10特征图，对于生成的每一个10*10的特征图，是由6个5*5的卷积图像去乘以6个14*14的下采样图，然后对应位置相加求和，然后对每一个神经元加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
(5)、S4层：由C3中16个10*10的特征图生成16个5*5下采样图，相邻四个神经元分别进行相加求和，然后乘以一个权值，再求均值即除以4，然后再加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
(6)、C5层：由S4中16个5*5下采样图生成120个1*1特征图，对于生成的每一个1*1的特征图，是由16个5*5的卷积图像去乘以16个5*5的下采用图，然后相加求和，然后对每一个神经元加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
(7)、输出层：即全连接层，输出层中的每一个神经元均是由C5层中的120个神经元乘以相对应的权值，然后相加求和；然后对每一个神经元加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
4. 反向传播：主要计算每层神经元、权值和阈值的误差，以用来更新权值和阈值
(1)、输出层：计算输出层神经元误差；通过mse损失函数的导数函数和tanh激活函数的导数函数来计算输出层神经元误差。
(2)、C5层：计算C5层神经元误差、输出层权值误差、输出层阈值误差；通过输出层神经元误差乘以输出层权值，求和，结果再乘以C5层神经元的tanh激活函数的导数，获得C5层每一个神经元误差；通过输出层神经元误差乘以C5层神经元获得输出层权值误差；输出层误差即为输出层阈值误差。
(3)、S4层：计算S4层神经元误差、C5层权值误差、C5层阈值误差；通过C5层权值乘以C5层神经元误差，求和，结果再乘以S4层神经元的tanh激活函数的导数，获得S4层每一个神经元误差；通过S4层神经元乘以C5层神经元误差，求和，获得C5层权值误差；C5层神经元误差即为C5层阈值误差。
(4)、C3层：计算C3层神经元误差、S4层权值误差、S4层阈值误差；
(5)、S2层：计算S2层神经元误差、C3层权值误差、C3层阈值误差；
(6)、C1层：计算C1层神经元误差、S2层权值误差、S2层阈值误差；
(7)、输入层：计算C1层权值误差、C1层阈值误差.
代码文件：
CNN.hpp：
<li class="alt"><span><span class="preprocessor">#ifndef _CNN_HPP_</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define _CNN_HPP_</span><span> </span></span></li><li class="alt"><span> </span></li><li class=""><span><span class="keyword">namespace</span><span> ANN { </span></span></li><li class="alt"><span> </span></li><li class=""><span><span class="preprocessor">#define width_image_input_CNN 32 //归一化图像宽</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define height_image_input_CNN 32 //归一化图像高</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define width_image_C1_CNN 28</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define height_image_C1_CNN 28</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define width_image_S2_CNN 14</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define height_image_S2_CNN 14</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define width_image_C3_CNN 10</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define height_image_C3_CNN 10</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define width_image_S4_CNN 5</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define height_image_S4_CNN 5</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define width_image_C5_CNN 1</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define height_image_C5_CNN 1</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define width_image_output_CNN 1</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define height_image_output_CNN 1</span><span> </span></span></li><li class=""><span> </span></li><li class="alt"><span><span class="preprocessor">#define width_kernel_conv_CNN 5 //卷积核大小</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define height_kernel_conv_CNN 5</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define width_kernel_pooling_CNN 2</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define height_kernel_pooling_CNN 2</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define size_pooling_CNN 2</span><span> </span></span></li><li class=""><span> </span></li><li class="alt"><span><span class="preprocessor">#define num_map_input_CNN 1 //输入层map个数</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define num_map_C1_CNN 6 //C1层map个数</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define num_map_S2_CNN 6 //S2层map个数</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define num_map_C3_CNN 16 //C3层map个数</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define num_map_S4_CNN 16 //S4层map个数</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define num_map_C5_CNN 120 //C5层map个数</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define num_map_output_CNN 10 //输出层map个数</span><span> </span></span></li><li class=""><span> </span></li><li class="alt"><span><span class="preprocessor">#define num_patterns_train_CNN 60000 //训练模式对数(总数)</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define num_patterns_test_CNN 10000 //测试模式对数(总数)</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define num_epochs_CNN 100 //最大迭代次数</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define accuracy_rate_CNN 0.97 //要求达到的准确率</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define learning_rate_CNN 0.01 //学习率</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define eps_CNN 1e-8</span><span> </span></span></li><li class="alt"><span> </span></li><li class=""><span><span class="preprocessor">#define len_weight_C1_CNN 150 //C1层权值数，5*5*6=150</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define len_bias_C1_CNN 6 //C1层阈值数，6</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define len_weight_S2_CNN 6 //S2层权值数,1*6=6</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define len_bias_S2_CNN 6 //S2层阈值数,6</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define len_weight_C3_CNN 2400 //C3层权值数，5*5*6*16</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define len_bias_C3_CNN 16 //C3层阈值数,16</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define len_weight_S4_CNN 16 //S4层权值数，1*16=16</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define len_bias_S4_CNN 16 //S4层阈值数，16</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define len_weight_C5_CNN 48000 //C5层权值数，5*5*16*120=48000</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define len_bias_C5_CNN 120 //C5层阈值数，120</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define len_weight_output_CNN 1200 //输出层权值数，120*10=1200</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define len_bias_output_CNN 10 //输出层阈值数，10</span><span> </span></span></li><li class=""><span> </span></li><li class="alt"><span><span class="preprocessor">#define num_neuron_input_CNN 1024 //输入层神经元数，32*32=1024</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define num_neuron_C1_CNN 4704 //C1层神经元数，28*28*6=4704</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define num_neuron_S2_CNN 1176 //S2层神经元数，14*14*6=1176</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define num_neuron_C3_CNN 1600 //C3层神经元数，10*10*16=1600</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define num_neuron_S4_CNN 400 //S4层神经元数，5*5*16=400</span><span> </span></span></li><li class=""><span><span class="preprocessor">#define num_neuron_C5_CNN 120 //C5层神经元数，1*120=120</span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#define num_neuron_output_CNN 10 //输出层神经元数，1*10=10</span><span> </span></span></li><li class=""><span> </span></li><li class="alt"><span><span class="keyword">class</span><span> CNN { </span></span></li><li class=""><span><span class="keyword">public</span><span>: </span></span></li><li class="alt"><span> CNN(); </span></li><li class=""><span> ~CNN(); </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">void</span><span> init(); </span><span class="comment">//初始化，分配空间</span><span> </span></span></li><li class="alt"><span> <span class="datatypes">bool</span><span> train(); </span><span class="comment">//训练</span><span> </span></span></li><li class=""><span> <span class="datatypes">int</span><span> predict(</span><span class="keyword">const</span><span> unsigned </span><span class="datatypes">char</span><span>* data, </span><span class="datatypes">int</span><span> width, </span><span class="datatypes">int</span><span> height); </span><span class="comment">//预测</span><span> </span></span></li><li class="alt"><span> <span class="datatypes">bool</span><span> readModelFile(</span><span class="keyword">const</span><span> </span><span class="datatypes">char</span><span>* name); </span><span class="comment">//读取已训练好的BP model</span><span> </span></span></li><li class=""><span> </span></li><li class="alt"><span><span class="keyword">protected</span><span>: </span></span></li><li class=""><span> <span class="keyword">typedef</span><span> std::vector<std::pair<</span><span class="datatypes">int</span><span>, </span><span class="datatypes">int</span><span>> > wi_connections; </span></span></li><li class="alt"><span> <span class="keyword">typedef</span><span> std::vector<std::pair<</span><span class="datatypes">int</span><span>, </span><span class="datatypes">int</span><span>> > wo_connections; </span></span></li><li class=""><span> <span class="keyword">typedef</span><span> std::vector<std::pair<</span><span class="datatypes">int</span><span>, </span><span class="datatypes">int</span><span>> > io_connections; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">void</span><span> release(); </span><span class="comment">//释放申请的空间</span><span> </span></span></li><li class="alt"><span> <span class="datatypes">bool</span><span> saveModelFile(</span><span class="keyword">const</span><span> </span><span class="datatypes">char</span><span>* name); </span><span class="comment">//将训练好的model保存起来，包括各层的节点数，权值和阈值</span><span> </span></span></li><li class=""><span> <span class="datatypes">bool</span><span> initWeightThreshold(); </span><span class="comment">//初始化，产生[-1, 1]之间的随机小数</span><span> </span></span></li><li class="alt"><span> <span class="datatypes">bool</span><span> getSrcData(); </span><span class="comment">//读取MNIST数据</span><span> </span></span></li><li class=""><span> <span class="datatypes">float</span><span> test(); </span><span class="comment">//训练完一次计算一次准确率</span><span> </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> activation_function_tanh(</span><span class="datatypes">float</span><span> x); </span><span class="comment">//激活函数:tanh</span><span> </span></span></li><li class=""><span> <span class="datatypes">float</span><span> activation_function_tanh_derivative(</span><span class="datatypes">float</span><span> x); </span><span class="comment">//激活函数tanh的导数</span><span> </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> activation_function_identity(</span><span class="datatypes">float</span><span> x); </span></span></li><li class=""><span> <span class="datatypes">float</span><span> activation_function_identity_derivative(</span><span class="datatypes">float</span><span> x); </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> loss_function_mse(</span><span class="datatypes">float</span><span> y, </span><span class="datatypes">float</span><span> t); </span><span class="comment">//损失函数:mean squared error</span><span> </span></span></li><li class=""><span> <span class="datatypes">float</span><span> loss_function_mse_derivative(</span><span class="datatypes">float</span><span> y, </span><span class="datatypes">float</span><span> t); </span></span></li><li class="alt"><span> <span class="keyword">void</span><span> loss_function_gradient(</span><span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* y, </span><span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* t, </span><span class="datatypes">float</span><span>* dst, </span><span class="datatypes">int</span><span> len); </span></span></li><li class=""><span> <span class="datatypes">float</span><span> dot_product(</span><span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* s1, </span><span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* s2, </span><span class="datatypes">int</span><span> len); </span><span class="comment">//点乘</span><span> </span></span></li><li class="alt"><span> <span class="datatypes">bool</span><span> muladd(</span><span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* src, </span><span class="datatypes">float</span><span> c, </span><span class="datatypes">int</span><span> len, </span><span class="datatypes">float</span><span>* dst); </span><span class="comment">//dst[i] += c * src[i]</span><span> </span></span></li><li class=""><span> <span class="keyword">void</span><span> init_variable(</span><span class="datatypes">float</span><span>* val, </span><span class="datatypes">float</span><span> c, </span><span class="datatypes">int</span><span> len); </span></span></li><li class="alt"><span> <span class="datatypes">bool</span><span> uniform_rand(</span><span class="datatypes">float</span><span>* src, </span><span class="datatypes">int</span><span> len, </span><span class="datatypes">float</span><span> min, </span><span class="datatypes">float</span><span> max); </span></span></li><li class=""><span> <span class="datatypes">float</span><span> uniform_rand(</span><span class="datatypes">float</span><span> min, </span><span class="datatypes">float</span><span> max); </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> get_index(</span><span class="datatypes">int</span><span> x, </span><span class="datatypes">int</span><span> y, </span><span class="datatypes">int</span><span> channel, </span><span class="datatypes">int</span><span> width, </span><span class="datatypes">int</span><span> height, </span><span class="datatypes">int</span><span> depth); </span></span></li><li class=""><span> <span class="keyword">void</span><span> calc_out2wi(</span><span class="datatypes">int</span><span> width_in, </span><span class="datatypes">int</span><span> height_in, </span><span class="datatypes">int</span><span> width_out, </span><span class="datatypes">int</span><span> height_out, </span><span class="datatypes">int</span><span> depth_out, std::vector<wi_connections>& out2wi); </span></span></li><li class="alt"><span> <span class="keyword">void</span><span> calc_out2bias(</span><span class="datatypes">int</span><span> width, </span><span class="datatypes">int</span><span> height, </span><span class="datatypes">int</span><span> depth, std::vector<</span><span class="datatypes">int</span><span>>& out2bias); </span></span></li><li class=""><span> <span class="keyword">void</span><span> calc_in2wo(</span><span class="datatypes">int</span><span> width_in, </span><span class="datatypes">int</span><span> height_in, </span><span class="datatypes">int</span><span> width_out, </span><span class="datatypes">int</span><span> height_out, </span><span class="datatypes">int</span><span> depth_in, </span><span class="datatypes">int</span><span> depth_out, std::vector<wo_connections>& in2wo); </span></span></li><li class="alt"><span> <span class="keyword">void</span><span> calc_weight2io(</span><span class="datatypes">int</span><span> width_in, </span><span class="datatypes">int</span><span> height_in, </span><span class="datatypes">int</span><span> width_out, </span><span class="datatypes">int</span><span> height_out, </span><span class="datatypes">int</span><span> depth_in, </span><span class="datatypes">int</span><span> depth_out, std::vector<io_connections>& weight2io); </span></span></li><li class=""><span> <span class="keyword">void</span><span> calc_bias2out(</span><span class="datatypes">int</span><span> width_in, </span><span class="datatypes">int</span><span> height_in, </span><span class="datatypes">int</span><span> width_out, </span><span class="datatypes">int</span><span> height_out, </span><span class="datatypes">int</span><span> depth_in, </span><span class="datatypes">int</span><span> depth_out, std::vector<std::vector<</span><span class="datatypes">int</span><span>> >& bias2out); </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="datatypes">bool</span><span> Forward_C1(); </span><span class="comment">//前向传播</span><span> </span></span></li><li class="alt"><span> <span class="datatypes">bool</span><span> Forward_S2(); </span></span></li><li class=""><span> <span class="datatypes">bool</span><span> Forward_C3(); </span></span></li><li class="alt"><span> <span class="datatypes">bool</span><span> Forward_S4(); </span></span></li><li class=""><span> <span class="datatypes">bool</span><span> Forward_C5(); </span></span></li><li class="alt"><span> <span class="datatypes">bool</span><span> Forward_output(); </span></span></li><li class=""><span> <span class="datatypes">bool</span><span> Backward_output(); </span></span></li><li class="alt"><span> <span class="datatypes">bool</span><span> Backward_C5(); </span><span class="comment">//反向传播</span><span> </span></span></li><li class=""><span> <span class="datatypes">bool</span><span> Backward_S4(); </span></span></li><li class="alt"><span> <span class="datatypes">bool</span><span> Backward_C3(); </span></span></li><li class=""><span> <span class="datatypes">bool</span><span> Backward_S2(); </span></span></li><li class="alt"><span> <span class="datatypes">bool</span><span> Backward_C1(); </span></span></li><li class=""><span> <span class="datatypes">bool</span><span> Backward_input(); </span></span></li><li class="alt"><span> <span class="datatypes">bool</span><span> UpdateWeights(); </span><span class="comment">//更新权值、阈值</span><span> </span></span></li><li class=""><span> <span class="keyword">void</span><span> update_weights_bias(</span><span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* delta, </span><span class="datatypes">float</span><span>* weight, </span><span class="datatypes">int</span><span> len); </span></span></li><li class="alt"><span> </span></li><li class=""><span><span class="keyword">private</span><span>: </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span>* data_input_train; </span><span class="comment">//原始标准输入数据，训练,范围：[-1, 1]</span><span> </span></span></li><li class=""><span> <span class="datatypes">float</span><span>* data_output_train; </span><span class="comment">//原始标准期望结果，训练,范围：[-0.9, 0.9]</span><span> </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span>* data_input_test; </span><span class="comment">//原始标准输入数据，测试,范围：[-1, 1]</span><span> </span></span></li><li class=""><span> <span class="datatypes">float</span><span>* data_output_test; </span><span class="comment">//原始标准期望结果，测试,范围：[-0.9, 0.9]</span><span> </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span>* data_single_image; </span></span></li><li class=""><span> <span class="datatypes">float</span><span>* data_single_label; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="datatypes">float</span><span> weight_C1[len_weight_C1_CNN]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> bias_C1[len_bias_C1_CNN]; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> weight_S2[len_weight_S2_CNN]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> bias_S2[len_bias_S2_CNN]; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> weight_C3[len_weight_C3_CNN]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> bias_C3[len_bias_C3_CNN]; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> weight_S4[len_weight_S4_CNN]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> bias_S4[len_bias_S4_CNN]; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> weight_C5[len_weight_C5_CNN]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> bias_C5[len_bias_C5_CNN]; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> weight_output[len_weight_output_CNN]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> bias_output[len_bias_output_CNN]; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="datatypes">float</span><span> neuron_input[num_neuron_input_CNN]; </span><span class="comment">//data_single_image</span><span> </span></span></li><li class=""><span> <span class="datatypes">float</span><span> neuron_C1[num_neuron_C1_CNN]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> neuron_S2[num_neuron_S2_CNN]; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> neuron_C3[num_neuron_C3_CNN]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> neuron_S4[num_neuron_S4_CNN]; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> neuron_C5[num_neuron_C5_CNN]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> neuron_output[num_neuron_output_CNN]; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="datatypes">float</span><span> delta_neuron_output[num_neuron_output_CNN]; </span><span class="comment">//神经元误差</span><span> </span></span></li><li class=""><span> <span class="datatypes">float</span><span> delta_neuron_C5[num_neuron_C5_CNN]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> delta_neuron_S4[num_neuron_S4_CNN]; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> delta_neuron_C3[num_neuron_C3_CNN]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> delta_neuron_S2[num_neuron_S2_CNN]; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> delta_neuron_C1[num_neuron_C1_CNN]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> delta_neuron_input[num_neuron_input_CNN]; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="datatypes">float</span><span> delta_weight_C1[len_weight_C1_CNN]; </span><span class="comment">//权值、阈值误差</span><span> </span></span></li><li class=""><span> <span class="datatypes">float</span><span> delta_bias_C1[len_bias_C1_CNN]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> delta_weight_S2[len_weight_S2_CNN]; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> delta_bias_S2[len_bias_S2_CNN]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> delta_weight_C3[len_weight_C3_CNN]; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> delta_bias_C3[len_bias_C3_CNN]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> delta_weight_S4[len_weight_S4_CNN]; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> delta_bias_S4[len_bias_S4_CNN]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> delta_weight_C5[len_weight_C5_CNN]; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> delta_bias_C5[len_bias_C5_CNN]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> delta_weight_output[len_weight_output_CNN]; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> delta_bias_output[len_bias_output_CNN]; </span></span></li><li class="alt"><span> </span></li><li class=""><span> std::vector<wi_connections> out2wi_S2; <span class="comment">// out_id -> [(weight_id, in_id)]</span><span> </span></span></li><li class="alt"><span> std::vector<<span class="datatypes">int</span><span>> out2bias_S2; </span></span></li><li class=""><span> std::vector<wi_connections> out2wi_S4; </span></li><li class="alt"><span> std::vector<<span class="datatypes">int</span><span>> out2bias_S4; </span></span></li><li class=""><span> std::vector<wo_connections> in2wo_C3; <span class="comment">// in_id -> [(weight_id, out_id)]</span><span> </span></span></li><li class="alt"><span> std::vector<io_connections> weight2io_C3; <span class="comment">// weight_id -> [(in_id, out_id)]</span><span> </span></span></li><li class=""><span> std::vector<std::vector<<span class="datatypes">int</span><span>> > bias2out_C3; </span></span></li><li class="alt"><span> std::vector<wo_connections> in2wo_C1; </span></li><li class=""><span> std::vector<io_connections> weight2io_C1; </span></li><li class="alt"><span> std::vector<std::vector<<span class="datatypes">int</span><span>> > bias2out_C1; </span></span></li><li class=""><span>}; </span></li><li class="alt"><span> </span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="preprocessor">#endif //_CNN_HPP_</span><span> </span></span></li>CNN.cpp：
<li class="alt"><span><span class="preprocessor">#include <assert.h></span><span> </span></span></li><li class=""><span><span class="preprocessor">#include <time.h></span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#include <iostream></span><span> </span></span></li><li class=""><span><span class="preprocessor">#include <fstream></span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#include <numeric></span><span> </span></span></li><li class=""><span><span class="preprocessor">#include <windows.h></span><span> </span></span></li><li class="alt"><span><span class="preprocessor">#include <random></span><span> </span></span></li><li class=""><span><span class="preprocessor">#include <algorithm></span><span> </span></span></li><li class="alt"><span> </span></li><li class=""><span><span class="preprocessor">#include <CNN.hpp></span><span> </span></span></li><li class="alt"><span> </span></li><li class=""><span><span class="keyword">namespace</span><span> ANN { </span></span></li><li class="alt"><span> </span></li><li class=""><span>CNN::CNN() </span></li><li class="alt"><span>{ </span></li><li class=""><span> data_input_train = NULL; </span></li><li class="alt"><span> data_output_train = NULL; </span></li><li class=""><span> data_input_test = NULL; </span></li><li class="alt"><span> data_output_test = NULL; </span></li><li class=""><span> data_single_image = NULL; </span></li><li class="alt"><span> data_single_label = NULL; </span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span>CNN::~CNN() </span></li><li class="alt"><span>{ </span></li><li class=""><span> release(); </span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="keyword">void</span><span> CNN::release() </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> <span class="keyword">if</span><span> (data_input_train) { </span></span></li><li class=""><span> <span class="keyword">delete</span><span>[] data_input_train; </span></span></li><li class="alt"><span> data_input_train = NULL; </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">if</span><span> (data_output_train) { </span></span></li><li class="alt"><span> <span class="keyword">delete</span><span>[] data_output_train; </span></span></li><li class=""><span> data_output_train = NULL; </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">if</span><span> (data_input_test) { </span></span></li><li class=""><span> <span class="keyword">delete</span><span>[] data_input_test; </span></span></li><li class="alt"><span> data_input_test = NULL; </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">if</span><span> (data_output_test) { </span></span></li><li class="alt"><span> <span class="keyword">delete</span><span>[] data_output_test; </span></span></li><li class=""><span> data_output_test = NULL; </span></li><li class="alt"><span> } </span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="keyword">void</span><span> CNN::init_variable(</span><span class="datatypes">float</span><span>* val, </span><span class="datatypes">float</span><span> c, </span><span class="datatypes">int</span><span> len) </span></span></li><li class="alt"><span>{ </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < len; i++) { </span></span></li><li class="alt"><span> val[i] = c; </span></li><li class=""><span> } </span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="keyword">void</span><span> CNN::init() </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> <span class="datatypes">int</span><span> len1 = width_image_input_CNN * height_image_input_CNN * num_patterns_train_CNN; </span></span></li><li class=""><span> data_input_train = <span class="keyword">new</span><span> </span><span class="datatypes">float</span><span>[len1]; </span></span></li><li class="alt"><span> init_variable(data_input_train, -1.0, len1); </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="datatypes">int</span><span> len2 = num_map_output_CNN * num_patterns_train_CNN; </span></span></li><li class=""><span> data_output_train = <span class="keyword">new</span><span> </span><span class="datatypes">float</span><span>[len2]; </span></span></li><li class="alt"><span> init_variable(data_output_train, -0.9, len2); </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="datatypes">int</span><span> len3 = width_image_input_CNN * height_image_input_CNN * num_patterns_test_CNN; </span></span></li><li class=""><span> data_input_test = <span class="keyword">new</span><span> </span><span class="datatypes">float</span><span>[len3]; </span></span></li><li class="alt"><span> init_variable(data_input_test, -1.0, len3); </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="datatypes">int</span><span> len4 = num_map_output_CNN * num_patterns_test_CNN; </span></span></li><li class=""><span> data_output_test = <span class="keyword">new</span><span> </span><span class="datatypes">float</span><span>[len4]; </span></span></li><li class="alt"><span> init_variable(data_output_test, -0.9, len4); </span></li><li class=""><span> </span></li><li class="alt"><span> initWeightThreshold(); </span></li><li class=""><span> getSrcData(); </span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">float</span><span> CNN::uniform_rand(</span><span class="datatypes">float</span><span> min, </span><span class="datatypes">float</span><span> max) </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> <span class="keyword">static</span><span> std::mt19937 gen(1); </span></span></li><li class=""><span> std::uniform_real_distribution<<span class="datatypes">float</span><span>> dst(min, max); </span></span></li><li class="alt"><span> <span class="keyword">return</span><span> dst(gen); </span></span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="datatypes">bool</span><span> CNN::uniform_rand(</span><span class="datatypes">float</span><span>* src, </span><span class="datatypes">int</span><span> len, </span><span class="datatypes">float</span><span> min, </span><span class="datatypes">float</span><span> max) </span></span></li><li class="alt"><span>{ </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < len; i++) { </span></span></li><li class="alt"><span> src[i] = uniform_rand(min, max); </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">bool</span><span> CNN::initWeightThreshold() </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> srand(time(0) + rand()); </span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span> scale = 6.0; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="comment">//const float_t weight_base = std::sqrt(scale_ / (fan_in + fan_out));</span><span> </span></span></li><li class="alt"><span> <span class="comment">//fan_in = width_kernel_conv_CNN * height_kernel_conv_CNN * num_map_input_CNN = 5 * 5 * 1</span><span> </span></span></li><li class=""><span> <span class="comment">//fan_out = width_kernel_conv_CNN * height_kernel_conv_CNN * num_map_C1_CNN = 5 * 5 * 6</span><span> </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> min_ = -std::sqrt(scale / (25.0 + 150.0)); </span></span></li><li class=""><span> <span class="datatypes">float</span><span> max_ = std::sqrt(scale / (25.0 + 150.0)); </span></span></li><li class="alt"><span> uniform_rand(weight_C1, len_weight_C1_CNN, min_, max_); </span></li><li class=""><span> <span class="comment">//for (int i = 0; i < len_weight_C1_CNN; i++) {</span><span> </span></span></li><li class="alt"><span> <span class="comment">// weight_C1[i] = -1 + 2 * ((float)rand()) / RAND_MAX; //[-1, 1]</span><span> </span></span></li><li class=""><span> <span class="comment">//}</span><span> </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < len_bias_C1_CNN; i++) { </span></span></li><li class=""><span> bias_C1[i] = -1 + 2 * ((<span class="datatypes">float</span><span>)rand()) / RAND_MAX;</span><span class="comment">//0.0;//</span><span> </span></span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> min_ = -std::sqrt(scale / (4.0 + 1.0)); </span></li><li class=""><span> max_ = std::sqrt(scale / (4.0 + 1.0)); </span></li><li class="alt"><span> uniform_rand(weight_S2, len_weight_S2_CNN, min_, max_); </span></li><li class=""><span> <span class="comment">//for (int i = 0; i < len_weight_S2_CNN; i++) {</span><span> </span></span></li><li class="alt"><span> <span class="comment">// weight_S2[i] = -1 + 2 * ((float)rand()) / RAND_MAX;</span><span> </span></span></li><li class=""><span> <span class="comment">//}</span><span> </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < len_bias_S2_CNN; i++) { </span></span></li><li class=""><span> bias_S2[i] = -1 + 2 * ((<span class="datatypes">float</span><span>)rand()) / RAND_MAX;</span><span class="comment">//0.0;// </span><span> </span></span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> min_ = -std::sqrt(scale / (150.0 + 400.0)); </span></li><li class=""><span> max_ = std::sqrt(scale / (150.0 + 400.0)); </span></li><li class="alt"><span> uniform_rand(weight_C3, len_weight_C3_CNN, min_, max_); </span></li><li class=""><span> <span class="comment">//for (int i = 0; i < len_weight_C3_CNN; i++) {</span><span> </span></span></li><li class="alt"><span> <span class="comment">// weight_C3[i] = -1 + 2 * ((float)rand()) / RAND_MAX;</span><span> </span></span></li><li class=""><span> <span class="comment">//}</span><span> </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < len_bias_C3_CNN; i++) { </span></span></li><li class=""><span> bias_C3[i] = -1 + 2 * ((<span class="datatypes">float</span><span>)rand()) / RAND_MAX;</span><span class="comment">//0.0;// </span><span> </span></span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> min_ = -std::sqrt(scale / (4.0 + 1.0)); </span></li><li class=""><span> max_ = std::sqrt(scale / (4.0 + 1.0)); </span></li><li class="alt"><span> uniform_rand(weight_S4, len_weight_S4_CNN, min_, max_); </span></li><li class=""><span> <span class="comment">//for (int i = 0; i < len_weight_S4_CNN; i++) {</span><span> </span></span></li><li class="alt"><span> <span class="comment">// weight_S4[i] = -1 + 2 * ((float)rand()) / RAND_MAX;</span><span> </span></span></li><li class=""><span> <span class="comment">//}</span><span> </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < len_bias_S4_CNN; i++) { </span></span></li><li class=""><span> bias_S4[i] = -1 + 2 * ((<span class="datatypes">float</span><span>)rand()) / RAND_MAX; </span><span class="comment">//0.0;//</span><span> </span></span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> min_ = -std::sqrt(scale / (400.0 + 3000.0)); </span></li><li class=""><span> max_ = std::sqrt(scale / (400.0 + 3000.0)); </span></li><li class="alt"><span> uniform_rand(weight_C5, len_weight_C5_CNN, min_, max_); </span></li><li class=""><span> <span class="comment">//for (int i = 0; i < len_weight_C5_CNN; i++) {</span><span> </span></span></li><li class="alt"><span> <span class="comment">// weight_C5[i] = -1 + 2 * ((float)rand()) / RAND_MAX;</span><span> </span></span></li><li class=""><span> <span class="comment">//}</span><span> </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < len_bias_C5_CNN; i++) { </span></span></li><li class=""><span> bias_C5[i] =-1 + 2 * ((<span class="datatypes">float</span><span>)rand()) / RAND_MAX; </span><span class="comment">//0.0;// </span><span> </span></span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> min_ = -std::sqrt(scale / (120.0 + 10.0)); </span></li><li class=""><span> max_ = std::sqrt(scale / (120.0 + 10.0)); </span></li><li class="alt"><span> uniform_rand(weight_output, len_weight_output_CNN, min_, max_); </span></li><li class=""><span> <span class="comment">//for (int i = 0; i < len_weight_output_CNN; i++) {</span><span> </span></span></li><li class="alt"><span> <span class="comment">// weight_output[i] = -1 + 2 * ((float)rand()) / RAND_MAX;</span><span> </span></span></li><li class=""><span> <span class="comment">//}</span><span> </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < len_bias_output_CNN; i++) { </span></span></li><li class=""><span> bias_output[i] = -1 + 2 * ((<span class="datatypes">float</span><span>)rand()) / RAND_MAX;</span><span class="comment">//0.0;// </span><span> </span></span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="keyword">static</span><span> </span><span class="datatypes">int</span><span> reverseInt(</span><span class="datatypes">int</span><span> i) </span></span></li><li class="alt"><span>{ </span></li><li class=""><span> unsigned <span class="datatypes">char</span><span> ch1, ch2, ch3, ch4; </span></span></li><li class="alt"><span> ch1 = i & 255; </span></li><li class=""><span> ch2 = (i >> 8) & 255; </span></li><li class="alt"><span> ch3 = (i >> 16) & 255; </span></li><li class=""><span> ch4 = (i >> 24) & 255; </span></li><li class="alt"><span> <span class="keyword">return</span><span>((</span><span class="datatypes">int</span><span>)ch1 << 24) + ((</span><span class="datatypes">int</span><span>)ch2 << 16) + ((</span><span class="datatypes">int</span><span>)ch3 << 8) + ch4; </span></span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="keyword">static</span><span> </span><span class="keyword">void</span><span> readMnistImages(std::string filename, </span><span class="datatypes">float</span><span>* data_dst, </span><span class="datatypes">int</span><span> num_image) </span></span></li><li class="alt"><span>{ </span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">int</span><span> width_src_image = 28; </span></span></li><li class="alt"><span> <span class="keyword">const</span><span> </span><span class="datatypes">int</span><span> height_src_image = 28; </span></span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">int</span><span> x_padding = 2; </span></span></li><li class="alt"><span> <span class="keyword">const</span><span> </span><span class="datatypes">int</span><span> y_padding = 2; </span></span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span> scale_min = -1; </span></span></li><li class="alt"><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span> scale_max = 1; </span></span></li><li class=""><span> </span></li><li class="alt"><span> std::ifstream file(filename, std::ios::binary); </span></li><li class=""><span> assert(file.is_open()); </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="datatypes">int</span><span> magic_number = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> number_of_images = 0; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> n_rows = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> n_cols = 0; </span></span></li><li class=""><span> file.read((<span class="datatypes">char</span><span>*)&magic_number, </span><span class="keyword">sizeof</span><span>(magic_number)); </span></span></li><li class="alt"><span> magic_number = reverseInt(magic_number); </span></li><li class=""><span> file.read((<span class="datatypes">char</span><span>*)&number_of_images, </span><span class="keyword">sizeof</span><span>(number_of_images)); </span></span></li><li class="alt"><span> number_of_images = reverseInt(number_of_images); </span></li><li class=""><span> assert(number_of_images == num_image); </span></li><li class="alt"><span> file.read((<span class="datatypes">char</span><span>*)&n_rows, </span><span class="keyword">sizeof</span><span>(n_rows)); </span></span></li><li class=""><span> n_rows = reverseInt(n_rows); </span></li><li class="alt"><span> file.read((<span class="datatypes">char</span><span>*)&n_cols, </span><span class="keyword">sizeof</span><span>(n_cols)); </span></span></li><li class=""><span> n_cols = reverseInt(n_cols); </span></li><li class="alt"><span> assert(n_rows == height_src_image && n_cols == width_src_image); </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="datatypes">int</span><span> size_single_image = width_image_input_CNN * height_image_input_CNN; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < number_of_images; ++i) { </span></span></li><li class=""><span> <span class="datatypes">int</span><span> addr = size_single_image * i; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> r = 0; r < n_rows; ++r) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> c = 0; c < n_cols; ++c) { </span></span></li><li class=""><span> unsigned <span class="datatypes">char</span><span> temp = 0; </span></span></li><li class="alt"><span> file.read((<span class="datatypes">char</span><span>*)&temp, </span><span class="keyword">sizeof</span><span>(temp)); </span></span></li><li class=""><span> data_dst[addr + width_image_input_CNN * (r + y_padding) + c + x_padding] = (temp / 255.0) * (scale_max - scale_min) + scale_min; </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="keyword">static</span><span> </span><span class="keyword">void</span><span> readMnistLabels(std::string filename, </span><span class="datatypes">float</span><span>* data_dst, </span><span class="datatypes">int</span><span> num_image) </span></span></li><li class="alt"><span>{ </span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span> scale_min = -0.9; </span></span></li><li class="alt"><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span> scale_max = 0.9; </span></span></li><li class=""><span> </span></li><li class="alt"><span> std::ifstream file(filename, std::ios::binary); </span></li><li class=""><span> assert(file.is_open()); </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="datatypes">int</span><span> magic_number = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> number_of_images = 0; </span></span></li><li class=""><span> file.read((<span class="datatypes">char</span><span>*)&magic_number, </span><span class="keyword">sizeof</span><span>(magic_number)); </span></span></li><li class="alt"><span> magic_number = reverseInt(magic_number); </span></li><li class=""><span> file.read((<span class="datatypes">char</span><span>*)&number_of_images, </span><span class="keyword">sizeof</span><span>(number_of_images)); </span></span></li><li class="alt"><span> number_of_images = reverseInt(number_of_images); </span></li><li class=""><span> assert(number_of_images == num_image); </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < number_of_images; ++i) { </span></span></li><li class="alt"><span> unsigned <span class="datatypes">char</span><span> temp = 0; </span></span></li><li class=""><span> file.read((<span class="datatypes">char</span><span>*)&temp, </span><span class="keyword">sizeof</span><span>(temp)); </span></span></li><li class="alt"><span> data_dst[i * num_map_output_CNN + temp] = scale_max; </span></li><li class=""><span> } </span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">bool</span><span> CNN::getSrcData() </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> assert(data_input_train && data_output_train && data_input_test && data_output_test); </span></li><li class=""><span> </span></li><li class="alt"><span> std::string filename_train_images = <span class="string">"D:/Download/MNIST/train-images.idx3-ubyte"</span><span>; </span></span></li><li class=""><span> std::string filename_train_labels = <span class="string">"D:/Download/MNIST/train-labels.idx1-ubyte"</span><span>; </span></span></li><li class="alt"><span> readMnistImages(filename_train_images, data_input_train, num_patterns_train_CNN); </span></li><li class=""><span> <span class="comment">/*unsigned char* p = new unsigned char[num_neuron_input_CNN];</span> </span></li><li class="alt"><span><span class="comment"> memset(p, 0, sizeof(unsigned char) * num_neuron_input_CNN);</span> </span></li><li class=""><span><span class="comment"> for (int j = 0, i = 59998 * num_neuron_input_CNN; j< num_neuron_input_CNN; j++, i++) {</span> </span></li><li class="alt"><span><span class="comment"> p[j] = (unsigned char)((data_input_train[i] + 1.0) / 2.0 * 255.0);</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> delete[] p;*/</span><span> </span></span></li><li class=""><span> readMnistLabels(filename_train_labels, data_output_train, num_patterns_train_CNN); </span></li><li class="alt"><span> <span class="comment">/*float* q = new float[num_neuron_output_CNN];</span> </span></li><li class=""><span><span class="comment"> memset(q, 0, sizeof(float) * num_neuron_output_CNN);</span> </span></li><li class="alt"><span><span class="comment"> for (int j = 0, i = 59998 * num_neuron_output_CNN; j < num_neuron_output_CNN; j++, i++) {</span> </span></li><li class=""><span><span class="comment"> q[j] = data_output_train[i];</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> delete[] q;*/</span><span> </span></span></li><li class="alt"><span> </span></li><li class=""><span> std::string filename_test_images = <span class="string">"D:/Download/MNIST/t10k-images.idx3-ubyte"</span><span>; </span></span></li><li class="alt"><span> std::string filename_test_labels = <span class="string">"D:/Download/MNIST/t10k-labels.idx1-ubyte"</span><span>; </span></span></li><li class=""><span> readMnistImages(filename_test_images, data_input_test, num_patterns_test_CNN); </span></li><li class="alt"><span> readMnistLabels(filename_test_labels, data_output_test, num_patterns_test_CNN); </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="datatypes">bool</span><span> CNN::train() </span></span></li><li class="alt"><span>{ </span></li><li class=""><span> out2wi_S2.clear(); </span></li><li class="alt"><span> out2bias_S2.clear(); </span></li><li class=""><span> out2wi_S4.clear(); </span></li><li class="alt"><span> out2bias_S4.clear(); </span></li><li class=""><span> in2wo_C3.clear(); </span></li><li class="alt"><span> weight2io_C3.clear(); </span></li><li class=""><span> bias2out_C3.clear(); </span></li><li class="alt"><span> in2wo_C1.clear(); </span></li><li class=""><span> weight2io_C1.clear(); </span></li><li class="alt"><span> bias2out_C1.clear(); </span></li><li class=""><span> </span></li><li class="alt"><span> calc_out2wi(width_image_C1_CNN, height_image_C1_CNN, width_image_S2_CNN, height_image_S2_CNN, num_map_S2_CNN, out2wi_S2); </span></li><li class=""><span> calc_out2bias(width_image_S2_CNN, height_image_S2_CNN, num_map_S2_CNN, out2bias_S2); </span></li><li class="alt"><span> calc_out2wi(width_image_C3_CNN, height_image_C3_CNN, width_image_S4_CNN, height_image_S4_CNN, num_map_S4_CNN, out2wi_S4); </span></li><li class=""><span> calc_out2bias(width_image_S4_CNN, height_image_S4_CNN, num_map_S4_CNN, out2bias_S4); </span></li><li class="alt"><span> calc_in2wo(width_image_C3_CNN, height_image_C3_CNN, width_image_S4_CNN, height_image_S4_CNN, num_map_C3_CNN, num_map_S4_CNN, in2wo_C3); </span></li><li class=""><span> calc_weight2io(width_image_C3_CNN, height_image_C3_CNN, width_image_S4_CNN, height_image_S4_CNN, num_map_C3_CNN, num_map_S4_CNN, weight2io_C3); </span></li><li class="alt"><span> calc_bias2out(width_image_C3_CNN, height_image_C3_CNN, width_image_S4_CNN, height_image_S4_CNN, num_map_C3_CNN, num_map_S4_CNN, bias2out_C3); </span></li><li class=""><span> calc_in2wo(width_image_C1_CNN, height_image_C1_CNN, width_image_S2_CNN, height_image_S2_CNN, num_map_C1_CNN, num_map_C3_CNN, in2wo_C1); </span></li><li class="alt"><span> calc_weight2io(width_image_C1_CNN, height_image_C1_CNN, width_image_S2_CNN, height_image_S2_CNN, num_map_C1_CNN, num_map_C3_CNN, weight2io_C1); </span></li><li class=""><span> calc_bias2out(width_image_C1_CNN, height_image_C1_CNN, width_image_S2_CNN, height_image_S2_CNN, num_map_C1_CNN, num_map_C3_CNN, bias2out_C1); </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="datatypes">int</span><span> iter = 0; </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (iter = 0; iter < num_epochs_CNN; iter++) { </span></span></li><li class=""><span> std::cout << <span class="string">"epoch: "</span><span> << iter; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="datatypes">float</span><span> accuracyRate = test();</span><span class="comment">//0;</span><span> </span></span></li><li class="alt"><span> std::cout << <span class="string">", accuray rate: "</span><span> << accuracyRate << std::endl; </span></span></li><li class=""><span> <span class="keyword">if</span><span> (accuracyRate > accuracy_rate_CNN) { </span></span></li><li class="alt"><span> saveModelFile(<span class="string">"cnn.model"</span><span>); </span></span></li><li class=""><span> std::cout << <span class="string">"generate cnn model"</span><span> << std::endl; </span></span></li><li class="alt"><span> <span class="keyword">break</span><span>; </span></span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < num_patterns_train_CNN; i++) { </span></span></li><li class="alt"><span> data_single_image = data_input_train + i * num_neuron_input_CNN; </span></li><li class=""><span> data_single_label = data_output_train + i * num_neuron_output_CNN; </span></li><li class="alt"><span> </span></li><li class=""><span> Forward_C1(); </span></li><li class="alt"><span> Forward_S2(); </span></li><li class=""><span> Forward_C3(); </span></li><li class="alt"><span> Forward_S4(); </span></li><li class=""><span> Forward_C5(); </span></li><li class="alt"><span> Forward_output(); </span></li><li class=""><span> </span></li><li class="alt"><span> Backward_output(); </span></li><li class=""><span> Backward_C5(); </span></li><li class="alt"><span> Backward_S4(); </span></li><li class=""><span> Backward_C3(); </span></li><li class="alt"><span> Backward_S2(); </span></li><li class=""><span> Backward_C1(); </span></li><li class="alt"><span> Backward_input(); </span></li><li class=""><span> </span></li><li class="alt"><span> UpdateWeights(); </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">if</span><span> (iter == num_epochs_CNN) { </span></span></li><li class=""><span> saveModelFile(<span class="string">"cnn.model"</span><span>); </span></span></li><li class="alt"><span> std::cout << <span class="string">"generate cnn model"</span><span> << std::endl; </span></span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">float</span><span> CNN::activation_function_tanh(</span><span class="datatypes">float</span><span> x) </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> <span class="datatypes">float</span><span> ep = std::exp(x); </span></span></li><li class=""><span> <span class="datatypes">float</span><span> em = std::exp(-x); </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">return</span><span> (ep - em) / (ep + em); </span></span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">float</span><span> CNN::activation_function_tanh_derivative(</span><span class="datatypes">float</span><span> x) </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> <span class="keyword">return</span><span> (1.0 - x * x); </span></span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="datatypes">float</span><span> CNN::activation_function_identity(</span><span class="datatypes">float</span><span> x) </span></span></li><li class="alt"><span>{ </span></li><li class=""><span> <span class="keyword">return</span><span> x; </span></span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">float</span><span> CNN::activation_function_identity_derivative(</span><span class="datatypes">float</span><span> x) </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> <span class="keyword">return</span><span> 1; </span></span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="datatypes">float</span><span> CNN::loss_function_mse(</span><span class="datatypes">float</span><span> y, </span><span class="datatypes">float</span><span> t) </span></span></li><li class="alt"><span>{ </span></li><li class=""><span> <span class="keyword">return</span><span> (y - t) * (y - t) / 2; </span></span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">float</span><span> CNN::loss_function_mse_derivative(</span><span class="datatypes">float</span><span> y, </span><span class="datatypes">float</span><span> t) </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> <span class="keyword">return</span><span> (y - t); </span></span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="keyword">void</span><span> CNN::loss_function_gradient(</span><span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* y, </span><span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* t, </span><span class="datatypes">float</span><span>* dst, </span><span class="datatypes">int</span><span> len) </span></span></li><li class="alt"><span>{ </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < len; i++) { </span></span></li><li class="alt"><span> dst[i] = loss_function_mse_derivative(y[i], t[i]); </span></li><li class=""><span> } </span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">float</span><span> CNN::dot_product(</span><span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* s1, </span><span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* s2, </span><span class="datatypes">int</span><span> len) </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> <span class="datatypes">float</span><span> result = 0.0; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < len; i++) { </span></span></li><li class=""><span> result += s1[i] * s2[i]; </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">return</span><span> result; </span></span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="datatypes">bool</span><span> CNN::muladd(</span><span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* src, </span><span class="datatypes">float</span><span> c, </span><span class="datatypes">int</span><span> len, </span><span class="datatypes">float</span><span>* dst) </span></span></li><li class="alt"><span>{ </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < len; i++) { </span></span></li><li class="alt"><span> dst[i] += (src[i] * c); </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">int</span><span> CNN::get_index(</span><span class="datatypes">int</span><span> x, </span><span class="datatypes">int</span><span> y, </span><span class="datatypes">int</span><span> channel, </span><span class="datatypes">int</span><span> width, </span><span class="datatypes">int</span><span> height, </span><span class="datatypes">int</span><span> depth) </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> assert(x >= 0 && x < width); </span></li><li class=""><span> assert(y >= 0 && y < height); </span></li><li class="alt"><span> assert(channel >= 0 && channel < depth); </span></li><li class=""><span> <span class="keyword">return</span><span> (height * channel + y) * width + x; </span></span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">bool</span><span> CNN::Forward_C1() </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> init_variable(neuron_C1, 0.0, num_neuron_C1_CNN); </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="comment">/*for (int i = 0; i < num_map_C1_CNN; i++) {</span> </span></li><li class=""><span><span class="comment"> int addr1 = i * width_image_C1_CNN * height_image_C1_CNN;</span> </span></li><li class="alt"><span><span class="comment"> int addr2 = i * width_kernel_conv_CNN * height_kernel_conv_CNN;</span> </span></li><li class=""><span><span class="comment"> float* image = &neuron_C1[0] + addr1;</span> </span></li><li class="alt"><span><span class="comment"> const float* weight = &weight_C1[0] + addr2;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int y = 0; y < height_image_C1_CNN; y++) {</span> </span></li><li class=""><span><span class="comment"> for (int x = 0; x < width_image_C1_CNN; x++) {</span> </span></li><li class="alt"><span><span class="comment"> float sum = 0.0;</span> </span></li><li class=""><span><span class="comment"> const float* image_input = data_single_image + y * width_image_input_CNN + x;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int m = 0; m < height_kernel_conv_CNN; m++) {</span> </span></li><li class="alt"><span><span class="comment"> for (int n = 0; n < width_kernel_conv_CNN; n++) {</span> </span></li><li class=""><span><span class="comment"> sum += weight[m * width_kernel_conv_CNN + n] * image_input[m * width_image_input_CNN + n];</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> image[y * width_image_C1_CNN + x] = activation_function_tanh(sum + bias_C1[i]); //tanh((w*x + b))</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }*/</span><span> </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> o = 0; o < num_map_C1_CNN; o++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> inc = 0; inc < num_map_input_CNN; inc++) { </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> addr1 = get_index(0, 0, num_map_input_CNN * o + inc, width_kernel_conv_CNN, height_kernel_conv_CNN, num_map_C1_CNN); </span></span></li><li class=""><span> <span class="datatypes">int</span><span> addr2 = get_index(0, 0, inc, width_image_input_CNN, height_image_input_CNN, num_map_input_CNN); </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> addr3 = get_index(0, 0, o, width_image_C1_CNN, height_image_C1_CNN, num_map_C1_CNN); </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* pw = &weight_C1[0] + addr1; </span></span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* pi = data_single_image + addr2; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span>* pa = &neuron_C1[0] + addr3; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height_image_C1_CNN; y++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> x = 0; x < width_image_C1_CNN; x++) { </span></span></li><li class="alt"><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* ppw = pw; </span></span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* ppi = pi + y * width_image_input_CNN + x; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> sum = 0.0; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> wy = 0; wy < height_kernel_conv_CNN; wy++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> wx = 0; wx < width_kernel_conv_CNN; wx++) { </span></span></li><li class="alt"><span> sum += *ppw++ * ppi[wy * width_image_input_CNN + wx]; </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> pa[y * width_image_C1_CNN + x] += sum; </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="datatypes">int</span><span> addr3 = get_index(0, 0, o, width_image_C1_CNN, height_image_C1_CNN, num_map_C1_CNN); </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span>* pa = &neuron_C1[0] + addr3; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> b = bias_C1[o]; </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height_image_C1_CNN; y++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> x = 0; x < width_image_C1_CNN; x++) { </span></span></li><li class="alt"><span> pa[y * width_image_C1_CNN + x] += b; </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < num_neuron_C1_CNN; i++) { </span></span></li><li class="alt"><span> neuron_C1[i] = activation_function_tanh(neuron_C1[i]); </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="keyword">void</span><span> CNN::calc_out2wi(</span><span class="datatypes">int</span><span> width_in, </span><span class="datatypes">int</span><span> height_in, </span><span class="datatypes">int</span><span> width_out, </span><span class="datatypes">int</span><span> height_out, </span><span class="datatypes">int</span><span> depth_out, std::vector<wi_connections>& out2wi) </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < depth_out; i++) { </span></span></li><li class=""><span> <span class="datatypes">int</span><span> block = width_in * height_in * i; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height_out; y++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> x = 0; x < width_out; x++) { </span></span></li><li class=""><span> <span class="datatypes">int</span><span> rows = y * width_kernel_pooling_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> cols = x * height_kernel_pooling_CNN; </span></span></li><li class=""><span> </span></li><li class="alt"><span> wi_connections wi_connections_; </span></li><li class=""><span> std::pair<<span class="datatypes">int</span><span>, </span><span class="datatypes">int</span><span>> pair_; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> m = 0; m < width_kernel_pooling_CNN; m++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> n = 0; n < height_kernel_pooling_CNN; n++) { </span></span></li><li class=""><span> pair_.first = i; </span></li><li class="alt"><span> pair_.second = (rows + m) * width_in + cols + n + block; </span></li><li class=""><span> wi_connections_.push_back(pair_); </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> out2wi.push_back(wi_connections_); </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="keyword">void</span><span> CNN::calc_out2bias(</span><span class="datatypes">int</span><span> width, </span><span class="datatypes">int</span><span> height, </span><span class="datatypes">int</span><span> depth, std::vector<</span><span class="datatypes">int</span><span>>& out2bias) </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < depth; i++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height; y++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> x = 0; x < width; x++) { </span></span></li><li class=""><span> out2bias.push_back(i); </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="keyword">void</span><span> CNN::calc_in2wo(</span><span class="datatypes">int</span><span> width_in, </span><span class="datatypes">int</span><span> height_in, </span><span class="datatypes">int</span><span> width_out, </span><span class="datatypes">int</span><span> height_out, </span><span class="datatypes">int</span><span> depth_in, </span><span class="datatypes">int</span><span> depth_out, std::vector<wo_connections>& in2wo) </span></span></li><li class="alt"><span>{ </span></li><li class=""><span> <span class="datatypes">int</span><span> len = width_in * height_in * depth_in; </span></span></li><li class="alt"><span> in2wo.resize(len); </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> c = 0; c < depth_in; c++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height_in; y += height_kernel_pooling_CNN) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> x = 0; x < width_in; x += width_kernel_pooling_CNN) { </span></span></li><li class=""><span> <span class="datatypes">int</span><span> dymax = min(size_pooling_CNN, height_in - y); </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> dxmax = min(size_pooling_CNN, width_in - x); </span></span></li><li class=""><span> <span class="datatypes">int</span><span> dstx = x / width_kernel_pooling_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> dsty = y / height_kernel_pooling_CNN; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> dy = 0; dy < dymax; dy++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> dx = 0; dx < dxmax; dx++) { </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> index_in = get_index(x + dx, y + dy, c, width_in, height_in, depth_in); </span></span></li><li class=""><span> <span class="datatypes">int</span><span> index_out = get_index(dstx, dsty, c, width_out, height_out, depth_out); </span></span></li><li class="alt"><span> </span></li><li class=""><span> wo_connections wo_connections_; </span></li><li class="alt"><span> std::pair<<span class="datatypes">int</span><span>, </span><span class="datatypes">int</span><span>> pair_; </span></span></li><li class=""><span> pair_.first = c; </span></li><li class="alt"><span> pair_.second = index_out; </span></li><li class=""><span> wo_connections_.push_back(pair_); </span></li><li class="alt"><span> </span></li><li class=""><span> in2wo[index_in] = wo_connections_; </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="keyword">void</span><span> CNN::calc_weight2io(</span><span class="datatypes">int</span><span> width_in, </span><span class="datatypes">int</span><span> height_in, </span><span class="datatypes">int</span><span> width_out, </span><span class="datatypes">int</span><span> height_out, </span><span class="datatypes">int</span><span> depth_in, </span><span class="datatypes">int</span><span> depth_out, std::vector<io_connections>& weight2io) </span></span></li><li class="alt"><span>{ </span></li><li class=""><span> <span class="datatypes">int</span><span> len = depth_in; </span></span></li><li class="alt"><span> weight2io.resize(len); </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> c = 0; c < depth_in; c++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height_in; y += height_kernel_pooling_CNN) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> x = 0; x < width_in; x += width_kernel_pooling_CNN) { </span></span></li><li class=""><span> <span class="datatypes">int</span><span> dymax = min(size_pooling_CNN, height_in - y); </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> dxmax = min(size_pooling_CNN, width_in - x); </span></span></li><li class=""><span> <span class="datatypes">int</span><span> dstx = x / width_kernel_pooling_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> dsty = y / height_kernel_pooling_CNN; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> dy = 0; dy < dymax; dy++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> dx = 0; dx < dxmax; dx++) { </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> index_in = get_index(x + dx, y + dy, c, width_in, height_in, depth_in); </span></span></li><li class=""><span> <span class="datatypes">int</span><span> index_out = get_index(dstx, dsty, c, width_out, height_out, depth_out); </span></span></li><li class="alt"><span> </span></li><li class=""><span> std::pair<<span class="datatypes">int</span><span>, </span><span class="datatypes">int</span><span>> pair_; </span></span></li><li class="alt"><span> pair_.first = index_in; </span></li><li class=""><span> pair_.second = index_out; </span></li><li class="alt"><span> </span></li><li class=""><span> weight2io[c].push_back(pair_); </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="keyword">void</span><span> CNN::calc_bias2out(</span><span class="datatypes">int</span><span> width_in, </span><span class="datatypes">int</span><span> height_in, </span><span class="datatypes">int</span><span> width_out, </span><span class="datatypes">int</span><span> height_out, </span><span class="datatypes">int</span><span> depth_in, </span><span class="datatypes">int</span><span> depth_out, std::vector<std::vector<</span><span class="datatypes">int</span><span>> >& bias2out) </span></span></li><li class="alt"><span>{ </span></li><li class=""><span> <span class="datatypes">int</span><span> len = depth_in; </span></span></li><li class="alt"><span> bias2out.resize(len); </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> c = 0; c < depth_in; c++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height_out; y++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> x = 0; x < width_out; x++) { </span></span></li><li class=""><span> <span class="datatypes">int</span><span> index_out = get_index(x, y, c, width_out, height_out, depth_out); </span></span></li><li class="alt"><span> bias2out[c].push_back(index_out); </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">bool</span><span> CNN::Forward_S2() </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> init_variable(neuron_S2, 0.0, num_neuron_S2_CNN); </span></li><li class=""><span> <span class="datatypes">float</span><span> scale_factor = 1.0 / (width_kernel_pooling_CNN * height_kernel_pooling_CNN); </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="comment">/*for (int i = 0; i < num_map_S2_CNN; i++) {</span> </span></li><li class="alt"><span><span class="comment"> int addr1 = i * width_image_S2_CNN * height_image_S2_CNN;</span> </span></li><li class=""><span><span class="comment"> int addr2 = i * width_image_C1_CNN * height_image_C1_CNN;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> float* image = &neuron_S2[0] + addr1;</span> </span></li><li class="alt"><span><span class="comment"> const float* image_input = &neuron_C1[0] + addr2;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int y = 0; y < height_image_S2_CNN; y++) {</span> </span></li><li class=""><span><span class="comment"> for (int x = 0; x < width_image_S2_CNN; x++) {</span> </span></li><li class="alt"><span><span class="comment"> float sum = 0.0;</span> </span></li><li class=""><span><span class="comment"> int rows = y * height_kernel_pooling_CNN;</span> </span></li><li class="alt"><span><span class="comment"> int cols = x * width_kernel_pooling_CNN;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int m = 0; m < height_kernel_pooling_CNN; m++) {</span> </span></li><li class=""><span><span class="comment"> for (int n = 0; n < width_kernel_pooling_CNN; n++) {</span> </span></li><li class="alt"><span><span class="comment"> sum += image_input[(rows + m) * width_image_C1_CNN + cols + n];</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> image[y * width_image_S2_CNN + x] = activation_function_tanh(sum * weight_S2[i] * scale_factor + bias_S2[i]);</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }*/</span><span> </span></span></li><li class="alt"><span> </span></li><li class=""><span> assert(out2wi_S2.size() == num_neuron_S2_CNN); </span></li><li class="alt"><span> assert(out2bias_S2.size() == num_neuron_S2_CNN); </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < num_neuron_S2_CNN; i++) { </span></span></li><li class=""><span> <span class="keyword">const</span><span> wi_connections& connections = out2wi_S2[i]; </span></span></li><li class="alt"><span> neuron_S2[i] = 0; </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> index = 0; index < connections.size(); index++) { </span></span></li><li class=""><span> neuron_S2[i] += weight_S2[connections[index].first] * neuron_C1[connections[index].second]; </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> neuron_S2[i] *= scale_factor; </span></li><li class=""><span> neuron_S2[i] += bias_S2[out2bias_S2[i]]; </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < num_neuron_S2_CNN; i++) { </span></span></li><li class=""><span> neuron_S2[i] = activation_function_tanh(neuron_S2[i]); </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="datatypes">bool</span><span> CNN::Forward_C3() </span></span></li><li class="alt"><span>{ </span></li><li class=""><span> init_variable(neuron_C3, 0.0, num_neuron_C3_CNN); </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="comment">/*for (int i = 0; i < num_map_C3_CNN; i++) {</span> </span></li><li class="alt"><span><span class="comment"> int addr1 = i * width_image_C3_CNN * height_image_C3_CNN;</span> </span></li><li class=""><span><span class="comment"> int addr2 = i * width_kernel_conv_CNN * height_kernel_conv_CNN * num_map_S2_CNN;</span> </span></li><li class="alt"><span><span class="comment"> float* image = &neuron_C3[0] + addr1;</span> </span></li><li class=""><span><span class="comment"> const float* weight = &weight_C3[0] + addr2;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int j = 0; j < num_map_S2_CNN; j++) {</span> </span></li><li class="alt"><span><span class="comment"> int addr3 = j * width_image_S2_CNN * height_image_S2_CNN;</span> </span></li><li class=""><span><span class="comment"> int addr4 = j * width_kernel_conv_CNN * height_kernel_conv_CNN;</span> </span></li><li class="alt"><span><span class="comment"> const float* image_input = &neuron_S2[0] + addr3;</span> </span></li><li class=""><span><span class="comment"> const float* weight_ = weight + addr4;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int y = 0; y < height_image_C3_CNN; y++) {</span> </span></li><li class="alt"><span><span class="comment"> for (int x = 0; x < width_image_C3_CNN; x++) {</span> </span></li><li class=""><span><span class="comment"> float sum = 0.0;</span> </span></li><li class="alt"><span><span class="comment"> const float* image_input_ = image_input + y * width_image_S2_CNN + x;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int m = 0; m < height_kernel_conv_CNN; m++) {</span> </span></li><li class=""><span><span class="comment"> for (int n = 0; n < width_kernel_conv_CNN; n++) {</span> </span></li><li class="alt"><span><span class="comment"> sum += weight_[m * width_kernel_conv_CNN + n] * image_input_[m * width_image_S2_CNN + n];</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> image[y * width_image_C3_CNN + x] += sum;</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int y = 0; y < height_image_C3_CNN; y++) {</span> </span></li><li class="alt"><span><span class="comment"> for (int x = 0; x < width_image_C3_CNN; x++) {</span> </span></li><li class=""><span><span class="comment"> image[y * width_image_C3_CNN + x] = activation_function_tanh(image[y * width_image_C3_CNN + x] + bias_C3[i]);</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }*/</span><span> </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> o = 0; o < num_map_C3_CNN; o++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> inc = 0; inc < num_map_S2_CNN; inc++) { </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> addr1 = get_index(0, 0, num_map_S2_CNN * o + inc, width_kernel_conv_CNN, height_kernel_conv_CNN, num_map_C3_CNN * num_map_S2_CNN); </span></span></li><li class=""><span> <span class="datatypes">int</span><span> addr2 = get_index(0, 0, inc, width_image_S2_CNN, height_image_S2_CNN, num_map_S2_CNN); </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> addr3 = get_index(0, 0, o, width_image_C3_CNN, height_image_C3_CNN, num_map_C3_CNN); </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* pw = &weight_C3[0] + addr1; </span></span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* pi = &neuron_S2[0] + addr2; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span>* pa = &neuron_C3[0] + addr3; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height_image_C3_CNN; y++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> x = 0; x < width_image_C3_CNN; x++) { </span></span></li><li class="alt"><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* ppw = pw; </span></span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* ppi = pi + y * width_image_S2_CNN + x; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> sum = 0.0; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> wy = 0; wy < height_kernel_conv_CNN; wy++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> wx = 0; wx < width_kernel_conv_CNN; wx++) { </span></span></li><li class="alt"><span> sum += *ppw++ * ppi[wy * width_image_S2_CNN + wx]; </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> pa[y * width_image_C3_CNN + x] += sum; </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="datatypes">int</span><span> addr3 = get_index(0, 0, o, width_image_C3_CNN, height_image_C3_CNN, num_map_C3_CNN); </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span>* pa = &neuron_C3[0] + addr3; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> b = bias_C3[o]; </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height_image_C3_CNN; y++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> x = 0; x < width_image_C3_CNN; x++) { </span></span></li><li class="alt"><span> pa[y * width_image_C3_CNN + x] += b; </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < num_neuron_C3_CNN; i++) { </span></span></li><li class="alt"><span> neuron_C3[i] = activation_function_tanh(neuron_C3[i]); </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">bool</span><span> CNN::Forward_S4() </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> <span class="datatypes">float</span><span> scale_factor = 1.0 / (width_kernel_pooling_CNN * height_kernel_pooling_CNN); </span></span></li><li class=""><span> init_variable(neuron_S4, 0.0, num_neuron_S4_CNN); </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="comment">/*for (int i = 0; i < num_map_S4_CNN; i++) {</span> </span></li><li class="alt"><span><span class="comment"> int addr1 = i * width_image_S4_CNN * height_image_S4_CNN;</span> </span></li><li class=""><span><span class="comment"> int addr2 = i * width_image_C3_CNN * height_image_C3_CNN;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> float* image = &neuron_S4[0] + addr1;</span> </span></li><li class="alt"><span><span class="comment"> const float* image_input = &neuron_C3[0] + addr2;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int y = 0; y < height_image_S4_CNN; y++) {</span> </span></li><li class=""><span><span class="comment"> for (int x = 0; x < width_image_S4_CNN; x++) {</span> </span></li><li class="alt"><span><span class="comment"> float sum = 0.0;</span> </span></li><li class=""><span><span class="comment"> int rows = y * height_kernel_pooling_CNN;</span> </span></li><li class="alt"><span><span class="comment"> int cols = x * width_kernel_pooling_CNN;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int m = 0; m < height_kernel_pooling_CNN; m++) {</span> </span></li><li class=""><span><span class="comment"> for (int n = 0; n < width_kernel_pooling_CNN; n++) {</span> </span></li><li class="alt"><span><span class="comment"> sum += image_input[(rows + m) * width_image_C3_CNN + cols + n];</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> image[y * width_image_S4_CNN + x] = activation_function_tanh(sum * weight_S4[i] * scale_factor + bias_S4[i]);</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }*/</span><span> </span></span></li><li class="alt"><span> </span></li><li class=""><span> assert(out2wi_S4.size() == num_neuron_S4_CNN); </span></li><li class="alt"><span> assert(out2bias_S4.size() == num_neuron_S4_CNN); </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < num_neuron_S4_CNN; i++) { </span></span></li><li class=""><span> <span class="keyword">const</span><span> wi_connections& connections = out2wi_S4[i]; </span></span></li><li class="alt"><span> neuron_S4[i] = 0.0; </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> index = 0; index < connections.size(); index++) { </span></span></li><li class=""><span> neuron_S4[i] += weight_S4[connections[index].first] * neuron_C3[connections[index].second]; </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> neuron_S4[i] *= scale_factor; </span></li><li class=""><span> neuron_S4[i] += bias_S4[out2bias_S4[i]]; </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < num_neuron_S4_CNN; i++) { </span></span></li><li class=""><span> neuron_S4[i] = activation_function_tanh(neuron_S4[i]); </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="comment">//int count_num = 0;</span><span> </span></span></li><li class=""><span> <span class="comment">//for (int i = 0; i < num_neuron_S4_CNN; i++) {</span><span> </span></span></li><li class="alt"><span> <span class="comment">// if (fabs(neuron_S4[i] - Tmp_neuron_S4[i]) > 0.0000001/*0.0000000001*/) {</span><span> </span></span></li><li class=""><span> <span class="comment">// count_num++;</span><span> </span></span></li><li class="alt"><span> <span class="comment">// std::cout << "i = " << i << " , old: " << neuron_S4[i] << " , new: " << Tmp_neuron_S4[i] << std::endl;</span><span> </span></span></li><li class=""><span> <span class="comment">// }</span><span> </span></span></li><li class="alt"><span> <span class="comment">//}</span><span> </span></span></li><li class=""><span> <span class="comment">//std::cout << "count_num: " << count_num << std::endl;</span><span> </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">bool</span><span> CNN::Forward_C5() </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> init_variable(neuron_C5, 0.0, num_neuron_C5_CNN); </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="comment">/*for (int i = 0; i < num_map_C5_CNN; i++) {</span> </span></li><li class=""><span><span class="comment"> int addr1 = i * width_image_C5_CNN * height_image_C5_CNN;</span> </span></li><li class="alt"><span><span class="comment"> int addr2 = i * width_kernel_conv_CNN * height_kernel_conv_CNN * num_map_S4_CNN;</span> </span></li><li class=""><span><span class="comment"> float* image = &neuron_C5[0] + addr1;</span> </span></li><li class="alt"><span><span class="comment"> const float* weight = &weight_C5[0] + addr2;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int j = 0; j < num_map_S4_CNN; j++) {</span> </span></li><li class=""><span><span class="comment"> int addr3 = j * width_kernel_conv_CNN * height_kernel_conv_CNN;</span> </span></li><li class="alt"><span><span class="comment"> int addr4 = j * width_image_S4_CNN * height_image_S4_CNN;</span> </span></li><li class=""><span><span class="comment"> const float* weight_ = weight + addr3;</span> </span></li><li class="alt"><span><span class="comment"> const float* image_input = &neuron_S4[0] + addr4;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int y = 0; y < height_image_C5_CNN; y++) {</span> </span></li><li class=""><span><span class="comment"> for (int x = 0; x < width_image_C5_CNN; x++) {</span> </span></li><li class="alt"><span><span class="comment"> float sum = 0.0;</span> </span></li><li class=""><span><span class="comment"> const float* image_input_ = image_input + y * width_image_S4_CNN + x;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int m = 0; m < height_kernel_conv_CNN; m++) {</span> </span></li><li class="alt"><span><span class="comment"> for (int n = 0; n < width_kernel_conv_CNN; n++) {</span> </span></li><li class=""><span><span class="comment"> sum += weight_[m * width_kernel_conv_CNN + n] * image_input_[m * width_image_S4_CNN + n];</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> image[y * width_image_C5_CNN + x] += sum;</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int y = 0; y < height_image_C5_CNN; y++) {</span> </span></li><li class=""><span><span class="comment"> for (int x = 0; x < width_image_C5_CNN; x++) {</span> </span></li><li class="alt"><span><span class="comment"> image[y * width_image_C5_CNN + x] = activation_function_tanh(image[y * width_image_C5_CNN + x] + bias_C5[i]);</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }*/</span><span> </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> o = 0; o < num_map_C5_CNN; o++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> inc = 0; inc < num_map_S4_CNN; inc++) { </span></span></li><li class=""><span> <span class="datatypes">int</span><span> addr1 = get_index(0, 0, num_map_S4_CNN * o + inc, width_kernel_conv_CNN, height_kernel_conv_CNN, num_map_C5_CNN * num_map_S4_CNN); </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> addr2 = get_index(0, 0, inc, width_image_S4_CNN, height_image_S4_CNN, num_map_S4_CNN); </span></span></li><li class=""><span> <span class="datatypes">int</span><span> addr3 = get_index(0, 0, o, width_image_C5_CNN, height_image_C5_CNN, num_map_C5_CNN); </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span> *pw = &weight_C5[0] + addr1; </span></span></li><li class="alt"><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span> *pi = &neuron_S4[0] + addr2; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> *pa = &neuron_C5[0] + addr3; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height_image_C5_CNN; y++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> x = 0; x < width_image_C5_CNN; x++) { </span></span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span> *ppw = pw; </span></span></li><li class="alt"><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span> *ppi = pi + y * width_image_S4_CNN + x; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> sum = 0.0; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> wy = 0; wy < height_kernel_conv_CNN; wy++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> wx = 0; wx < width_kernel_conv_CNN; wx++) { </span></span></li><li class=""><span> sum += *ppw++ * ppi[wy * width_image_S4_CNN + wx]; </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> pa[y * width_image_C5_CNN + x] += sum; </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="datatypes">int</span><span> addr3 = get_index(0, 0, o, width_image_C5_CNN, height_image_C5_CNN, num_map_C5_CNN); </span></span></li><li class=""><span> <span class="datatypes">float</span><span> *pa = &neuron_C5[0] + addr3; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> b = bias_C5[o]; </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height_image_C5_CNN; y++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> x = 0; x < width_image_C5_CNN; x++) { </span></span></li><li class=""><span> pa[y * width_image_C5_CNN + x] += b; </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < num_neuron_C5_CNN; i++) { </span></span></li><li class=""><span> neuron_C5[i] = activation_function_tanh(neuron_C5[i]); </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="datatypes">bool</span><span> CNN::Forward_output() </span></span></li><li class="alt"><span>{ </span></li><li class=""><span> init_variable(neuron_output, 0.0, num_neuron_output_CNN); </span></li><li class="alt"><span> <span class="comment">/*float* image = &neuron_output[0];</span> </span></li><li class=""><span><span class="comment"> const float* weight = &weight_output[0];</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int i = 0; i < num_neuron_output_CNN; i++) {</span> </span></li><li class="alt"><span><span class="comment"> for (int j = 0; j < num_neuron_C5_CNN; j++) {</span> </span></li><li class=""><span><span class="comment"> image[i] += (weight[j * num_neuron_output_CNN + i] * neuron_C5[j]);</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> image[i] = activation_function_tanh(image[i] + bias_output[i]);</span> </span></li><li class=""><span><span class="comment"> }*/</span><span> </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < num_neuron_output_CNN; i++) { </span></span></li><li class="alt"><span> neuron_output[i] = 0.0; </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> c = 0; c < num_neuron_C5_CNN; c++) { </span></span></li><li class=""><span> neuron_output[i] += weight_output[c * num_neuron_output_CNN + i] * neuron_C5[c]; </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> neuron_output[i] += bias_output[i]; </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < num_neuron_output_CNN; i++) { </span></span></li><li class="alt"><span> neuron_output[i] = activation_function_tanh(neuron_output[i]); </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">bool</span><span> CNN::Backward_output() </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> init_variable(delta_neuron_output, 0.0, num_neuron_output_CNN); </span></li><li class=""><span> <span class="comment">/*float gradient[num_neuron_output_CNN];</span> </span></li><li class="alt"><span><span class="comment"> const float* t = &data_single_label[0];</span> </span></li><li class=""><span><span class="comment"> float tmp[num_neuron_output_CNN];</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int i = 0; i < num_neuron_output_CNN; i++) {</span> </span></li><li class="alt"><span><span class="comment"> gradient[i] = loss_function_mse_derivative(neuron_output[i], t[i]);</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int i = 0; i < num_neuron_output_CNN; i++) {</span> </span></li><li class="alt"><span><span class="comment"> init_variable(tmp, 0.0, num_neuron_output_CNN);</span> </span></li><li class=""><span><span class="comment"> tmp[i] = activation_function_tanh_derivative(neuron_output[i]);</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> delta_neuron_output[i] = dot_product(gradient, tmp, num_neuron_output_CNN);</span> </span></li><li class="alt"><span><span class="comment"> }*/</span><span> </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="datatypes">float</span><span> dE_dy[num_neuron_output_CNN]; </span></span></li><li class=""><span> init_variable(dE_dy, 0.0, num_neuron_output_CNN); </span></li><li class="alt"><span> loss_function_gradient(neuron_output, data_single_label, dE_dy, num_neuron_output_CNN); </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="comment">// delta = dE/da = (dE/dy) * (dy/da)</span><span> </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < num_neuron_output_CNN; i++) { </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> dy_da[num_neuron_output_CNN]; </span></span></li><li class=""><span> init_variable(dy_da, 0.0, num_neuron_output_CNN); </span></li><li class="alt"><span> </span></li><li class=""><span> dy_da[i] = activation_function_tanh_derivative(neuron_output[i]); </span></li><li class="alt"><span> delta_neuron_output[i] = dot_product(dE_dy, dy_da, num_neuron_output_CNN); </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">bool</span><span> CNN::Backward_C5() </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> init_variable(delta_neuron_C5, 0.0, num_neuron_C5_CNN); </span></li><li class=""><span> init_variable(delta_weight_output, 0.0, len_weight_output_CNN); </span></li><li class="alt"><span> init_variable(delta_bias_output, 0.0, len_bias_output_CNN); </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="comment">/*for (int i = 0; i < num_neuron_C5_CNN; i++) {</span> </span></li><li class=""><span><span class="comment"> delta_neuron_C5[i] = dot_product(&delta_neuron_output[0], &weight_output[0] + i * num_neuron_output_CNN, num_neuron_output_CNN);</span> </span></li><li class="alt"><span><span class="comment"> delta_neuron_C5[i] *= activation_function_tanh_derivative(neuron_C5[i]);</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int j = 0; j < num_neuron_C5_CNN; j++) {</span> </span></li><li class="alt"><span><span class="comment"> muladd(&delta_neuron_output[0], neuron_C5[j], num_neuron_output_CNN, &delta_weight_output[0] + j * num_neuron_output_CNN);</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int i = 0; i < num_neuron_output_CNN; i++) {</span> </span></li><li class="alt"><span><span class="comment"> delta_bias_output[i] += delta_neuron_output[i];</span> </span></li><li class=""><span><span class="comment"> }*/</span><span> </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> c = 0; c < num_neuron_C5_CNN; c++) { </span></span></li><li class="alt"><span> <span class="comment">// propagate delta to previous layer</span><span> </span></span></li><li class=""><span> <span class="comment">// prev_delta[c] += current_delta[r] * W_[c * out_size_ + r]</span><span> </span></span></li><li class="alt"><span> delta_neuron_C5[c] = dot_product(&delta_neuron_output[0], &weight_output[c * num_neuron_output_CNN], num_neuron_output_CNN); </span></li><li class=""><span> delta_neuron_C5[c] *= activation_function_tanh_derivative(neuron_C5[c]); </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="comment">// accumulate weight-step using delta</span><span> </span></span></li><li class=""><span> <span class="comment">// dW[c * out_size + i] += current_delta[i] * prev_out[c]</span><span> </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> c = 0; c < num_neuron_C5_CNN; c++) { </span></span></li><li class=""><span> muladd(&delta_neuron_output[0], neuron_C5[c], num_neuron_output_CNN, &delta_weight_output[0] + c * num_neuron_output_CNN); </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < len_bias_output_CNN; i++) { </span></span></li><li class=""><span> delta_bias_output[i] += delta_neuron_output[i]; </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="comment">//int count_num = 0;</span><span> </span></span></li><li class=""><span> <span class="comment">//for (int i = 0; i < num_neuron_C5_CNN; i++) {</span><span> </span></span></li><li class="alt"><span> <span class="comment">// if (fabs(delta_neuron_C5[i] - Tmp_delta_neuron_C5[i]) > 0.0000001/*0.0000000001*/) {</span><span> </span></span></li><li class=""><span> <span class="comment">// count_num++;</span><span> </span></span></li><li class="alt"><span> <span class="comment">// }</span><span> </span></span></li><li class=""><span> <span class="comment">//}</span><span> </span></span></li><li class="alt"><span> <span class="comment">//std::cout << "delta_neuron count_num: " << count_num << std::endl;</span><span> </span></span></li><li class=""><span> <span class="comment">//count_num = 0;</span><span> </span></span></li><li class="alt"><span> <span class="comment">//for (int i = 0; i < len_weight_output_CNN; i++) {</span><span> </span></span></li><li class=""><span> <span class="comment">// if (fabs(delta_weight_output[i] - Tmp_delta_weight_output[i]) > 0.0000001/*0.0000000001*/) {</span><span> </span></span></li><li class="alt"><span> <span class="comment">// count_num++;</span><span> </span></span></li><li class=""><span> <span class="comment">// }</span><span> </span></span></li><li class="alt"><span> <span class="comment">//}</span><span> </span></span></li><li class=""><span> <span class="comment">//std::cout << "delta_weight count_num: " << count_num << std::endl;</span><span> </span></span></li><li class="alt"><span> <span class="comment">//count_num = 0;</span><span> </span></span></li><li class=""><span> <span class="comment">//for (int i = 0; i < len_bias_output_CNN; i++) {</span><span> </span></span></li><li class="alt"><span> <span class="comment">// if (fabs(delta_bias_output[i] - Tmp_delta_bias_output[i]) > 0.0000001/*0.0000000001*/) {</span><span> </span></span></li><li class=""><span> <span class="comment">// count_num++;</span><span> </span></span></li><li class="alt"><span> <span class="comment">// }</span><span> </span></span></li><li class=""><span> <span class="comment">//}</span><span> </span></span></li><li class="alt"><span> <span class="comment">//std::cout << "delta_bias count_num: " << count_num << std::endl;</span><span> </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="datatypes">bool</span><span> CNN::Backward_S4() </span></span></li><li class="alt"><span>{ </span></li><li class=""><span> init_variable(delta_neuron_S4, 0.0, num_neuron_S4_CNN); </span></li><li class="alt"><span> init_variable(delta_weight_C5, 0.0, len_weight_C5_CNN); </span></li><li class=""><span> init_variable(delta_bias_C5, 0.0, len_bias_C5_CNN); </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="comment">/*for (int i = 0; i < num_map_S4_CNN; i++) {</span> </span></li><li class="alt"><span><span class="comment"> for (int j = 0; j < num_map_C5_CNN; j++) {</span> </span></li><li class=""><span><span class="comment"> int addr1 = width_kernel_conv_CNN * height_kernel_conv_CNN * (num_map_S4_CNN * j + i);</span> </span></li><li class="alt"><span><span class="comment"> int addr2 = width_image_S4_CNN * height_image_S4_CNN * i;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> const float* weight_c5 = &weight_C5[0] + addr1;</span> </span></li><li class=""><span><span class="comment"> const float* delta_c5 = &delta_neuron_C5[0] + width_image_C5_CNN * height_image_C5_CNN * j;</span> </span></li><li class="alt"><span><span class="comment"> float* delta_s4 = &delta_neuron_S4[0] + addr2;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int y = 0; y < height_image_C5_CNN; y++) {</span> </span></li><li class=""><span><span class="comment"> for (int x = 0; x < width_image_C5_CNN; x++) {</span> </span></li><li class="alt"><span><span class="comment"> const float* weight_c5_ = weight_c5;</span> </span></li><li class=""><span><span class="comment"> const float delta_c5_ = delta_c5[y * width_image_C5_CNN + x];</span> </span></li><li class="alt"><span><span class="comment"> float* delta_s4_ = delta_s4 + y * width_image_S4_CNN + x;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int m = 0; m < height_kernel_conv_CNN; m++) {</span> </span></li><li class=""><span><span class="comment"> for (int n = 0; n < width_kernel_conv_CNN; n++) {</span> </span></li><li class="alt"><span><span class="comment"> delta_s4_[m * width_image_S4_CNN + n] += weight_c5_[m * width_kernel_conv_CNN + n] * delta_c5_;</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int i = 0; i < num_neuron_S4_CNN; i++) {</span> </span></li><li class=""><span><span class="comment"> delta_neuron_S4[i] *= activation_function_tanh_derivative(neuron_S4[i]);</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int i = 0; i < num_map_S4_CNN; i++) {////////</span> </span></li><li class=""><span><span class="comment"> for (int j = 0; j < num_map_C5_CNN; j++) {</span> </span></li><li class="alt"><span><span class="comment"> for (int y = 0; y < height_kernel_conv_CNN; y++) {</span> </span></li><li class=""><span><span class="comment"> for (int x = 0; x < width_kernel_conv_CNN; x++) {</span> </span></li><li class="alt"><span><span class="comment"> int addr1 = (height_image_S4_CNN * i + y) * width_image_S4_CNN + x;</span> </span></li><li class=""><span><span class="comment"> int addr2 = (height_kernel_conv_CNN * (num_map_S4_CNN * j + i) + y) * width_kernel_conv_CNN + x;</span> </span></li><li class="alt"><span><span class="comment"> int addr3 = height_image_C5_CNN * j * width_image_C5_CNN;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> float dst = 0;</span> </span></li><li class=""><span><span class="comment"> const float* neuron_s4 = &neuron_S4[0] + addr1;</span> </span></li><li class="alt"><span><span class="comment"> const float* delta_c5 = &delta_neuron_C5[0] + addr3;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int m = 0; m < height_image_C5_CNN; m++) {</span> </span></li><li class=""><span><span class="comment"> dst += dot_product(neuron_s4 + m * width_image_S4_CNN, delta_c5 + y * width_image_C5_CNN, width_image_C5_CNN);</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> delta_weight_C5[addr2] += dst;</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int i = 0; i < num_map_C5_CNN; i++) {</span> </span></li><li class=""><span><span class="comment"> delta_bias_C5[i] += delta_neuron_C5[i];</span> </span></li><li class="alt"><span><span class="comment"> }*/</span><span> </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="comment">// propagate delta to previous layer</span><span> </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> inc = 0; inc < num_map_S4_CNN; inc++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> outc = 0; outc < num_map_C5_CNN; outc++) { </span></span></li><li class=""><span> <span class="datatypes">int</span><span> addr1 = get_index(0, 0, num_map_S4_CNN * outc + inc, width_kernel_conv_CNN, height_kernel_conv_CNN, num_map_S4_CNN * num_map_C5_CNN); </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> addr2 = get_index(0, 0, outc, width_image_C5_CNN, height_image_C5_CNN, num_map_C5_CNN); </span></span></li><li class=""><span> <span class="datatypes">int</span><span> addr3 = get_index(0, 0, inc, width_image_S4_CNN, height_image_S4_CNN, num_map_S4_CNN); </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* pw = &weight_C5[0] + addr1; </span></span></li><li class="alt"><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* pdelta_src = &delta_neuron_C5[0] + addr2; </span></span></li><li class=""><span> <span class="datatypes">float</span><span>* pdelta_dst = &delta_neuron_S4[0] + addr3; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height_image_C5_CNN; y++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> x = 0; x < width_image_C5_CNN; x++) { </span></span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* ppw = pw; </span></span></li><li class="alt"><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span> ppdelta_src = pdelta_src[y * width_image_C5_CNN + x]; </span></span></li><li class=""><span> <span class="datatypes">float</span><span>* ppdelta_dst = pdelta_dst + y * width_image_S4_CNN + x; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> wy = 0; wy < height_kernel_conv_CNN; wy++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> wx = 0; wx < width_kernel_conv_CNN; wx++) { </span></span></li><li class=""><span> ppdelta_dst[wy * width_image_S4_CNN + wx] += *ppw++ * ppdelta_src; </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < num_neuron_S4_CNN; i++) { </span></span></li><li class="alt"><span> delta_neuron_S4[i] *= activation_function_tanh_derivative(neuron_S4[i]); </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="comment">// accumulate dw</span><span> </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> inc = 0; inc < num_map_S4_CNN; inc++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> outc = 0; outc < num_map_C5_CNN; outc++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> wy = 0; wy < height_kernel_conv_CNN; wy++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> wx = 0; wx < width_kernel_conv_CNN; wx++) { </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> addr1 = get_index(wx, wy, inc, width_image_S4_CNN, height_image_S4_CNN, num_map_S4_CNN); </span></span></li><li class=""><span> <span class="datatypes">int</span><span> addr2 = get_index(0, 0, outc, width_image_C5_CNN, height_image_C5_CNN, num_map_C5_CNN); </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> addr3 = get_index(wx, wy, num_map_S4_CNN * outc + inc, width_kernel_conv_CNN, height_kernel_conv_CNN, num_map_S4_CNN * num_map_C5_CNN); </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="datatypes">float</span><span> dst = 0.0; </span></span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* prevo = &neuron_S4[0] + addr1; </span></span></li><li class="alt"><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* delta = &delta_neuron_C5[0] + addr2; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height_image_C5_CNN; y++) { </span></span></li><li class=""><span> dst += dot_product(prevo + y * width_image_S4_CNN, delta + y * width_image_C5_CNN, width_image_C5_CNN); </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> delta_weight_C5[addr3] += dst; </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="comment">// accumulate db</span><span> </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> outc = 0; outc < num_map_C5_CNN; outc++) { </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> addr2 = get_index(0, 0, outc, width_image_C5_CNN, height_image_C5_CNN, num_map_C5_CNN); </span></span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* delta = &delta_neuron_C5[0] + addr2; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height_image_C5_CNN; y++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> x = 0; x < width_image_C5_CNN; x++) { </span></span></li><li class=""><span> delta_bias_C5[outc] += delta[y * width_image_C5_CNN + x]; </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="datatypes">bool</span><span> CNN::Backward_C3() </span></span></li><li class="alt"><span>{ </span></li><li class=""><span> init_variable(delta_neuron_C3, 0.0, num_neuron_C3_CNN); </span></li><li class="alt"><span> init_variable(delta_weight_S4, 0.0, len_weight_S4_CNN); </span></li><li class=""><span> init_variable(delta_bias_S4, 0.0, len_bias_S4_CNN); </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="datatypes">float</span><span> scale_factor = 1.0 / (width_kernel_pooling_CNN * height_kernel_pooling_CNN); </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="comment">/*for (int i = 0; i < num_map_C3_CNN; i++) {</span> </span></li><li class="alt"><span><span class="comment"> int addr1 = width_image_S4_CNN * height_image_S4_CNN * i;</span> </span></li><li class=""><span><span class="comment"> int addr2 = width_image_C3_CNN * height_image_C3_CNN * i;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> const float* delta_s4 = &delta_neuron_S4[0] + addr1;</span> </span></li><li class="alt"><span><span class="comment"> float* delta_c3 = &delta_neuron_C3[0] + addr2;</span> </span></li><li class=""><span><span class="comment"> const float* neuron_c3 = &neuron_C3[0] + addr2;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int y = 0; y < height_image_C3_CNN; y++) {</span> </span></li><li class="alt"><span><span class="comment"> for (int x = 0; x < width_image_C3_CNN; x++) {</span> </span></li><li class=""><span><span class="comment"> float delta = 0.0;</span> </span></li><li class="alt"><span><span class="comment"> int index = width_image_S4_CNN * (y / height_kernel_pooling_CNN) + x / width_kernel_pooling_CNN;</span> </span></li><li class=""><span><span class="comment"> delta = weight_S4[i] * delta_s4[index];</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> delta_c3[y * width_image_C3_CNN + x] = delta * scale_factor * activation_function_tanh_derivative(neuron_c3[y * width_image_C3_CNN + x]);</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int i = 0; i < len_weight_S4_CNN; i++) {</span> </span></li><li class=""><span><span class="comment"> int addr1 = width_image_C3_CNN * height_image_C3_CNN * i;</span> </span></li><li class="alt"><span><span class="comment"> int addr2 = width_image_S4_CNN * height_image_S4_CNN * i;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> const float* neuron_c3 = &neuron_C3[0] + addr1;</span> </span></li><li class=""><span><span class="comment"> const float* delta_s4 = &delta_neuron_S4[0] + addr2;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> float diff = 0.0;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int y = 0; y < height_image_C3_CNN; y++) {</span> </span></li><li class="alt"><span><span class="comment"> for (int x = 0; x < width_image_C3_CNN; x++) {</span> </span></li><li class=""><span><span class="comment"> int index = y / height_kernel_pooling_CNN * height_image_S4_CNN + x / width_kernel_pooling_CNN;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> diff += neuron_c3[y * width_image_C3_CNN + x] * delta_s4[index];</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> delta_weight_S4[i] += diff * scale_factor;</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int i = 0; i < len_bias_S4_CNN; i++) {</span> </span></li><li class=""><span><span class="comment"> int addr1 = width_image_S4_CNN * height_image_S4_CNN * i;</span> </span></li><li class="alt"><span><span class="comment"> const float* delta_s4 = &delta_neuron_S4[0] + addr1;</span> </span></li><li class=""><span><span class="comment"> float diff = 0;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int y = 0; y < height_image_S4_CNN; y++) {</span> </span></li><li class="alt"><span><span class="comment"> for (int x = 0; x < width_image_S4_CNN; x++) {</span> </span></li><li class=""><span><span class="comment"> diff += delta_s4[y * width_image_S4_CNN + x];</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> delta_bias_S4[i] += diff;</span> </span></li><li class="alt"><span><span class="comment"> }*/</span><span> </span></span></li><li class=""><span> </span></li><li class="alt"><span> assert(in2wo_C3.size() == num_neuron_C3_CNN); </span></li><li class=""><span> assert(weight2io_C3.size() == len_weight_S4_CNN); </span></li><li class="alt"><span> assert(bias2out_C3.size() == len_bias_S4_CNN); </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < num_neuron_C3_CNN; i++) { </span></span></li><li class=""><span> <span class="keyword">const</span><span> wo_connections& connections = in2wo_C3[i]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> delta = 0.0; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> j = 0; j < connections.size(); j++) { </span></span></li><li class=""><span> delta += weight_S4[connections[j].first] * delta_neuron_S4[connections[j].second]; </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> delta_neuron_C3[i] = delta * scale_factor * activation_function_tanh_derivative(neuron_C3[i]); </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < len_weight_S4_CNN; i++) { </span></span></li><li class="alt"><span> <span class="keyword">const</span><span> io_connections& connections = weight2io_C3[i]; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> diff = 0; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> j = 0; j < connections.size(); j++) { </span></span></li><li class="alt"><span> diff += neuron_C3[connections[j].first] * delta_neuron_S4[connections[j].second]; </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> delta_weight_S4[i] += diff * scale_factor; </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < len_bias_S4_CNN; i++) { </span></span></li><li class=""><span> <span class="keyword">const</span><span> std::vector<</span><span class="datatypes">int</span><span>>& outs = bias2out_C3[i]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> diff = 0; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> o = 0; o < outs.size(); o++) { </span></span></li><li class=""><span> diff += delta_neuron_S4[outs[o]]; </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> delta_bias_S4[i] += diff; </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">bool</span><span> CNN::Backward_S2() </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> init_variable(delta_neuron_S2, 0.0, num_neuron_S2_CNN); </span></li><li class=""><span> init_variable(delta_weight_C3, 0.0, len_weight_C3_CNN); </span></li><li class="alt"><span> init_variable(delta_bias_C3, 0.0, len_bias_C3_CNN); </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="comment">/*for (int i = 0; i < num_map_S2_CNN; i++) {////////////////</span> </span></li><li class=""><span><span class="comment"> int addr1 = width_kernel_conv_CNN * height_kernel_conv_CNN * num_map_C3_CNN * i;</span> </span></li><li class="alt"><span><span class="comment"> int addr2 = width_kernel_conv_CNN * height_kernel_conv_CNN * i;</span> </span></li><li class=""><span><span class="comment"> for (int j = 0; j < num_map_C3_CNN; j++) {</span> </span></li><li class="alt"><span><span class="comment"> const float* weight_c3 = &weight_C3[0] + addr1 + j * width_kernel_conv_CNN * height_kernel_conv_CNN;</span> </span></li><li class=""><span><span class="comment"> const float* delta_c3 = &delta_neuron_C3[0] + width_image_C3_CNN * height_image_C3_CNN * j;</span> </span></li><li class="alt"><span><span class="comment"> float* delta_s2 = &delta_neuron_S2[0] + addr2;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int y = 0; y < height_image_C3_CNN; y++) {</span> </span></li><li class=""><span><span class="comment"> for (int x = 0; x < width_image_C3_CNN; x++) {</span> </span></li><li class="alt"><span><span class="comment"> const float* weight_c3_ = weight_c3;</span> </span></li><li class=""><span><span class="comment"> const float delta_c3_ = delta_c3[y * width_image_C3_CNN + x];</span> </span></li><li class="alt"><span><span class="comment"> float* delta_s2_ = delta_s2 + y * width_kernel_conv_CNN + x;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int m = 0; m < height_kernel_conv_CNN; m++) {</span> </span></li><li class=""><span><span class="comment"> for (int n = 0; n < width_kernel_conv_CNN; n++) {</span> </span></li><li class="alt"><span><span class="comment"> delta_s2_[m * width_kernel_conv_CNN + n] += weight_c3_[m * width_kernel_conv_CNN + n] * delta_c3_;</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int i = 0; i < num_neuron_S2_CNN; i++) {</span> </span></li><li class=""><span><span class="comment"> delta_neuron_S2[i] *= activation_function_tanh_derivative(neuron_S2[i]);</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int i = 0; i < num_map_S2_CNN; i++) {//////////////////</span> </span></li><li class=""><span><span class="comment"> int addr1 = width_kernel_conv_CNN * height_kernel_conv_CNN * i;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int j = 0; j < num_map_C3_CNN; j++) {</span> </span></li><li class="alt"><span><span class="comment"> int addr2 = width_kernel_conv_CNN * height_kernel_conv_CNN * i * j;</span> </span></li><li class=""><span><span class="comment"> float* delta_weight_c3 = &delta_weight_C3[0] + addr2;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int y = 0; y < height_kernel_conv_CNN; y++) {</span> </span></li><li class="alt"><span><span class="comment"> for (int x = 0; x < width_kernel_conv_CNN; x++) {</span> </span></li><li class=""><span><span class="comment"> float dst = 0;</span> </span></li><li class="alt"><span><span class="comment"> const float* neuron_s2 = &neuron_S2[0] + addr1 + y * width_kernel_conv_CNN + x;</span> </span></li><li class=""><span><span class="comment"> const float* delta_c3 = &delta_neuron_C3[0] + width_image_C3_CNN * height_image_C3_CNN * j;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int m = 0; m < height_image_C3_CNN; m++) {</span> </span></li><li class="alt"><span><span class="comment"> dst += dot_product(neuron_s2 + m * width_kernel_conv_CNN, delta_c3 + y * width_image_C3_CNN, width_image_C3_CNN);</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> delta_weight_c3[y * width_kernel_conv_CNN + x] += dst;</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int i = 0; i < num_map_C3_CNN; i++) {</span> </span></li><li class="alt"><span><span class="comment"> const float* delta = &delta_neuron_C3[0] + width_image_C3_CNN * height_image_C3_CNN * i;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> //delta_bias_C3[i] += std::accumulate(delta, delta + width_image_C3_CNN * height_image_C3_CNN, (float)0.0);</span> </span></li><li class=""><span><span class="comment"> for (int y = 0; y < height_image_C3_CNN; y++) {</span> </span></li><li class="alt"><span><span class="comment"> for (int x = 0; x < width_image_C3_CNN; x++) {</span> </span></li><li class=""><span><span class="comment"> delta_bias_C3[i] += delta[y * width_image_C3_CNN + x];</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }*/</span><span> </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="comment">// propagate delta to previous layer</span><span> </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> inc = 0; inc < num_map_S2_CNN; inc++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> outc = 0; outc < num_map_C3_CNN; outc++) { </span></span></li><li class=""><span> <span class="datatypes">int</span><span> addr1 = get_index(0, 0, num_map_S2_CNN * outc + inc, width_kernel_conv_CNN, height_kernel_conv_CNN, num_map_S2_CNN * num_map_C3_CNN); </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> addr2 = get_index(0, 0, outc, width_image_C3_CNN, height_image_C3_CNN, num_map_C3_CNN); </span></span></li><li class=""><span> <span class="datatypes">int</span><span> addr3 = get_index(0, 0, inc, width_image_S2_CNN, height_image_S2_CNN, num_map_S2_CNN); </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span> *pw = &weight_C3[0] + addr1; </span></span></li><li class="alt"><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span> *pdelta_src = &delta_neuron_C3[0] + addr2;; </span></span></li><li class=""><span> <span class="datatypes">float</span><span>* pdelta_dst = &delta_neuron_S2[0] + addr3; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height_image_C3_CNN; y++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> x = 0; x < width_image_C3_CNN; x++) { </span></span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* ppw = pw; </span></span></li><li class="alt"><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span> ppdelta_src = pdelta_src[y * width_image_C3_CNN + x]; </span></span></li><li class=""><span> <span class="datatypes">float</span><span>* ppdelta_dst = pdelta_dst + y * width_image_S2_CNN + x; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> wy = 0; wy < height_kernel_conv_CNN; wy++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> wx = 0; wx < width_kernel_conv_CNN; wx++) { </span></span></li><li class=""><span> ppdelta_dst[wy * width_image_S2_CNN + wx] += *ppw++ * ppdelta_src; </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < num_neuron_S2_CNN; i++) { </span></span></li><li class="alt"><span> delta_neuron_S2[i] *= activation_function_tanh_derivative(neuron_S2[i]); </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="comment">// accumulate dw</span><span> </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> inc = 0; inc < num_map_S2_CNN; inc++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> outc = 0; outc < num_map_C3_CNN; outc++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> wy = 0; wy < height_kernel_conv_CNN; wy++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> wx = 0; wx < width_kernel_conv_CNN; wx++) { </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> addr1 = get_index(wx, wy, inc, width_image_S2_CNN, height_image_S2_CNN, num_map_S2_CNN); </span></span></li><li class=""><span> <span class="datatypes">int</span><span> addr2 = get_index(0, 0, outc, width_image_C3_CNN, height_image_C3_CNN, num_map_C3_CNN); </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> addr3 = get_index(wx, wy, num_map_S2_CNN * outc + inc, width_kernel_conv_CNN, height_kernel_conv_CNN, num_map_S2_CNN * num_map_C3_CNN); </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="datatypes">float</span><span> dst = 0.0; </span></span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* prevo = &neuron_S2[0] + addr1; </span></span></li><li class="alt"><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* delta = &delta_neuron_C3[0] + addr2; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height_image_C3_CNN; y++) { </span></span></li><li class=""><span> dst += dot_product(prevo + y * width_image_S2_CNN, delta + y * width_image_C3_CNN, width_image_C3_CNN); </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> delta_weight_C3[addr3] += dst; </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="comment">// accumulate db</span><span> </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> outc = 0; outc < len_bias_C3_CNN; outc++) { </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> addr1 = get_index(0, 0, outc, width_image_C3_CNN, height_image_C3_CNN, num_map_C3_CNN); </span></span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* delta = &delta_neuron_C3[0] + addr1; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height_image_C3_CNN; y++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> x = 0; x < width_image_C3_CNN; x++) { </span></span></li><li class=""><span> delta_bias_C3[outc] += delta[y * width_image_C3_CNN + x]; </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="datatypes">bool</span><span> CNN::Backward_C1() </span></span></li><li class="alt"><span>{ </span></li><li class=""><span> init_variable(delta_neuron_C1, 0.0, num_neuron_C1_CNN); </span></li><li class="alt"><span> init_variable(delta_weight_S2, 0.0, len_weight_S2_CNN); </span></li><li class=""><span> init_variable(delta_bias_S2, 0.0, len_bias_S2_CNN); </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="datatypes">float</span><span> scale_factor = 1.0 / (width_kernel_pooling_CNN * height_kernel_pooling_CNN); </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="comment">/*for (int i = 0; i < num_map_C1_CNN; i++) {</span> </span></li><li class="alt"><span><span class="comment"> int addr1 = width_image_S2_CNN * height_image_S2_CNN * i;</span> </span></li><li class=""><span><span class="comment"> int addr2 = width_image_C1_CNN * height_image_C1_CNN * i;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> const float* delta_s2 = &delta_neuron_S2[0] + addr1;</span> </span></li><li class="alt"><span><span class="comment"> float* delta_c1 = &delta_neuron_C1[0] + addr2;</span> </span></li><li class=""><span><span class="comment"> const float* neuron_c1 = &neuron_C1[0] + addr2;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int y = 0; y < height_image_C1_CNN; y++) {</span> </span></li><li class="alt"><span><span class="comment"> for (int x = 0; x < width_image_C1_CNN; x++) {</span> </span></li><li class=""><span><span class="comment"> float delta = 0.0;</span> </span></li><li class="alt"><span><span class="comment"> int index = width_image_S2_CNN * (y / height_kernel_pooling_CNN) + x / width_kernel_pooling_CNN;</span> </span></li><li class=""><span><span class="comment"> delta = weight_S2[i] * delta_s2[index];</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> delta_c1[y * width_image_C1_CNN + x] = delta * scale_factor * activation_function_tanh_derivative(neuron_c1[y * width_image_C1_CNN + x]);</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int i = 0; i < len_weight_S2_CNN; i++) {</span> </span></li><li class=""><span><span class="comment"> int addr1 = width_image_C1_CNN * height_image_C1_CNN * i;</span> </span></li><li class="alt"><span><span class="comment"> int addr2 = width_image_S2_CNN * height_image_S2_CNN * i;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> const float* neuron_c1 = &neuron_C1[0] + addr1;</span> </span></li><li class=""><span><span class="comment"> const float* delta_s2 = &delta_neuron_S2[0] + addr2;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> float diff = 0.0;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int y = 0; y < height_image_C1_CNN; y++) {</span> </span></li><li class="alt"><span><span class="comment"> for (int x = 0; x < width_image_C1_CNN; x++) {</span> </span></li><li class=""><span><span class="comment"> int index = y / height_kernel_pooling_CNN * height_image_S2_CNN + x / width_kernel_pooling_CNN;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> diff += neuron_c1[y * width_image_C1_CNN + x] * delta_s2[index];</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> delta_weight_S2[i] += diff * scale_factor;</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int i = 0; i < len_bias_S2_CNN; i++) {</span> </span></li><li class=""><span><span class="comment"> int addr1 = width_image_S2_CNN * height_image_S2_CNN * i;</span> </span></li><li class="alt"><span><span class="comment"> const float* delta_s2 = &delta_neuron_S2[0] + addr1;</span> </span></li><li class=""><span><span class="comment"> float diff = 0;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int y = 0; y < height_image_S2_CNN; y++) {</span> </span></li><li class="alt"><span><span class="comment"> for (int x = 0; x < width_image_S2_CNN; x++) {</span> </span></li><li class=""><span><span class="comment"> diff += delta_s2[y * width_image_S2_CNN + x];</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> delta_bias_S2[i] += diff;</span> </span></li><li class="alt"><span><span class="comment"> }*/</span><span> </span></span></li><li class=""><span> </span></li><li class="alt"><span> assert(in2wo_C1.size() == num_neuron_C1_CNN); </span></li><li class=""><span> assert(weight2io_C1.size() == len_weight_S2_CNN); </span></li><li class="alt"><span> assert(bias2out_C1.size() == len_bias_S2_CNN); </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < num_neuron_C1_CNN; i++) { </span></span></li><li class=""><span> <span class="keyword">const</span><span> wo_connections& connections = in2wo_C1[i]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> delta = 0.0; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> j = 0; j < connections.size(); j++) { </span></span></li><li class=""><span> delta += weight_S2[connections[j].first] * delta_neuron_S2[connections[j].second]; </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> delta_neuron_C1[i] = delta * scale_factor * activation_function_tanh_derivative(neuron_C1[i]); </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < len_weight_S2_CNN; i++) { </span></span></li><li class="alt"><span> <span class="keyword">const</span><span> io_connections& connections = weight2io_C1[i]; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> diff = 0.0; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> j = 0; j < connections.size(); j++) { </span></span></li><li class="alt"><span> diff += neuron_C1[connections[j].first] * delta_neuron_S2[connections[j].second]; </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> delta_weight_S2[i] += diff * scale_factor; </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < len_bias_S2_CNN; i++) { </span></span></li><li class=""><span> <span class="keyword">const</span><span> std::vector<</span><span class="datatypes">int</span><span>>& outs = bias2out_C1[i]; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> diff = 0; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> o = 0; o < outs.size(); o++) { </span></span></li><li class=""><span> diff += delta_neuron_S2[outs[o]]; </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> delta_bias_S2[i] += diff; </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">bool</span><span> CNN::Backward_input() </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> init_variable(delta_neuron_input, 0.0, num_neuron_input_CNN); </span></li><li class=""><span> init_variable(delta_weight_C1, 0.0, len_weight_C1_CNN); </span></li><li class="alt"><span> init_variable(delta_bias_C1, 0.0, len_bias_C1_CNN); </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="comment">/*for (int i = 0; i < num_map_input_CNN; i++) {///////////////////</span> </span></li><li class=""><span><span class="comment"> int addr1 = width_kernel_conv_CNN * height_kernel_conv_CNN * num_map_C1_CNN * i;</span> </span></li><li class="alt"><span><span class="comment"> int addr2 = width_image_input_CNN * height_image_input_CNN * i;</span> </span></li><li class=""><span><span class="comment"> for (int j = 0; j < num_map_C1_CNN; j++) {</span> </span></li><li class="alt"><span><span class="comment"> const float* weight_c1 = &weight_C1[0] + addr1 + j * width_kernel_conv_CNN * height_kernel_conv_CNN;</span> </span></li><li class=""><span><span class="comment"> const float* delta_c1 = &delta_neuron_C1[0] + width_image_C1_CNN * height_image_C1_CNN * j;</span> </span></li><li class="alt"><span><span class="comment"> float* delta_input_ = &delta_neuron_input[0] + addr2;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int y = 0; y < height_image_C1_CNN; y++) {</span> </span></li><li class=""><span><span class="comment"> for (int x = 0; x < width_image_C1_CNN; x++) {</span> </span></li><li class="alt"><span><span class="comment"> const float* weight_c1_ = weight_c1;</span> </span></li><li class=""><span><span class="comment"> const float delta_c1_ = delta_c1[y * width_image_C1_CNN + x];</span> </span></li><li class="alt"><span><span class="comment"> float* delta_input_0 = delta_input_ + y * width_image_C1_CNN + x;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int m = 0; m < height_kernel_conv_CNN; m++) {</span> </span></li><li class=""><span><span class="comment"> for (int n = 0; n < width_kernel_conv_CNN; n++) {</span> </span></li><li class="alt"><span><span class="comment"> delta_input_0[m * width_image_input_CNN + n] += weight_c1_[m * width_kernel_conv_CNN + n] * delta_c1_;</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int i = 0; i < num_neuron_input_CNN; i++) {</span> </span></li><li class=""><span><span class="comment"> delta_neuron_input[i] *= activation_function_identity_derivative(data_single_image[i]);</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> for (int i = 0; i < num_map_input_CNN; i++) {/////////////</span> </span></li><li class=""><span><span class="comment"> int addr1 = width_image_input_CNN * height_image_input_CNN * i;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int j = 0; j < num_map_C1_CNN; j++) {</span> </span></li><li class="alt"><span><span class="comment"> int addr2 = width_kernel_conv_CNN * height_kernel_conv_CNN * i * j;</span> </span></li><li class=""><span><span class="comment"> float* delta_weight_c1 = &delta_weight_C1[0] + addr2;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int y = 0; y < height_kernel_conv_CNN; y++) {</span> </span></li><li class="alt"><span><span class="comment"> for (int x = 0; x < width_kernel_conv_CNN; x++) {</span> </span></li><li class=""><span><span class="comment"> float dst = 0;</span> </span></li><li class="alt"><span><span class="comment"> const float* neuron_input_ = data_single_image + addr1 + y * width_image_input_CNN + x;</span> </span></li><li class=""><span><span class="comment"> const float* delta_c1 = &delta_neuron_C1[0] + width_image_C1_CNN * height_image_C1_CNN * j;</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int m = 0; m < height_image_C1_CNN; m++) {</span> </span></li><li class="alt"><span><span class="comment"> dst += dot_product(neuron_input_ + m * width_kernel_conv_CNN, delta_c1 + y * width_image_C1_CNN, width_image_C1_CNN);</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> delta_weight_c1[y * width_kernel_conv_CNN + x] += dst;</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"></span> </span></li><li class=""><span><span class="comment"> for (int i = 0; i < num_map_C1_CNN; i++) {</span> </span></li><li class="alt"><span><span class="comment"> const float* delta = &delta_neuron_C1[0] + width_image_C1_CNN * height_image_C1_CNN * i;</span> </span></li><li class=""><span><span class="comment"></span> </span></li><li class="alt"><span><span class="comment"> //delta_bias_C1[i] += std::accumulate(delta, delta + width_image_C1_CNN * height_image_C1_CNN, (float)0.0);</span> </span></li><li class=""><span><span class="comment"> for (int y = 0; y < height_image_C1_CNN; y++) {</span> </span></li><li class="alt"><span><span class="comment"> for (int x = 0; x < width_image_C1_CNN; x++) {</span> </span></li><li class=""><span><span class="comment"> delta_bias_C1[i] += delta[y * width_image_C1_CNN + x];</span> </span></li><li class="alt"><span><span class="comment"> }</span> </span></li><li class=""><span><span class="comment"> }</span> </span></li><li class="alt"><span><span class="comment"> }*/</span><span> </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="comment">// propagate delta to previous layer</span><span> </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> inc = 0; inc < num_map_input_CNN; inc++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> outc = 0; outc < num_map_C1_CNN; outc++) { </span></span></li><li class=""><span> <span class="datatypes">int</span><span> addr1 = get_index(0, 0, num_map_input_CNN * outc + inc, width_kernel_conv_CNN, height_kernel_conv_CNN, num_map_C1_CNN); </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> addr2 = get_index(0, 0, outc, width_image_C1_CNN, height_image_C1_CNN, num_map_C1_CNN); </span></span></li><li class=""><span> <span class="datatypes">int</span><span> addr3 = get_index(0, 0, inc, width_image_input_CNN, height_image_input_CNN, num_map_input_CNN); </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* pw = &weight_C1[0] + addr1; </span></span></li><li class="alt"><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* pdelta_src = &delta_neuron_C1[0] + addr2; </span></span></li><li class=""><span> <span class="datatypes">float</span><span>* pdelta_dst = &delta_neuron_input[0] + addr3; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height_image_C1_CNN; y++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> x = 0; x < width_image_C1_CNN; x++) { </span></span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* ppw = pw; </span></span></li><li class="alt"><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span> ppdelta_src = pdelta_src[y * width_image_C1_CNN + x]; </span></span></li><li class=""><span> <span class="datatypes">float</span><span>* ppdelta_dst = pdelta_dst + y * width_image_input_CNN + x; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> wy = 0; wy < height_kernel_conv_CNN; wy++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> wx = 0; wx < width_kernel_conv_CNN; wx++) { </span></span></li><li class=""><span> ppdelta_dst[wy * width_image_input_CNN + wx] += *ppw++ * ppdelta_src; </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < num_neuron_input_CNN; i++) { </span></span></li><li class="alt"><span> delta_neuron_input[i] *= activation_function_identity_derivative(data_single_image[i]<span class="comment">/*neuron_input[i]*/</span><span>); </span></span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="comment">// accumulate dw</span><span> </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> inc = 0; inc < num_map_input_CNN; inc++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> outc = 0; outc < num_map_C1_CNN; outc++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> wy = 0; wy < height_kernel_conv_CNN; wy++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> wx = 0; wx < width_kernel_conv_CNN; wx++) { </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> addr1 = get_index(wx, wy, inc, width_image_input_CNN, height_image_input_CNN, num_map_input_CNN); </span></span></li><li class=""><span> <span class="datatypes">int</span><span> addr2 = get_index(0, 0, outc, width_image_C1_CNN, height_image_C1_CNN, num_map_C1_CNN); </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> addr3 = get_index(wx, wy, num_map_input_CNN * outc + inc, width_kernel_conv_CNN, height_kernel_conv_CNN, num_map_C1_CNN); </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="datatypes">float</span><span> dst = 0.0; </span></span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* prevo = data_single_image + addr1;</span><span class="comment">//&neuron_input[0]</span><span> </span></span></li><li class="alt"><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* delta = &delta_neuron_C1[0] + addr2; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height_image_C1_CNN; y++) { </span></span></li><li class=""><span> dst += dot_product(prevo + y * width_image_input_CNN, delta + y * width_image_C1_CNN, width_image_C1_CNN); </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> delta_weight_C1[addr3] += dst; </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="comment">// accumulate db</span><span> </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> outc = 0; outc < len_bias_C1_CNN; outc++) { </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> addr1 = get_index(0, 0, outc, width_image_C1_CNN, height_image_C1_CNN, num_map_C1_CNN); </span></span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* delta = &delta_neuron_C1[0] + addr1; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height_image_C1_CNN; y++) { </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> x = 0; x < width_image_C1_CNN; x++) { </span></span></li><li class=""><span> delta_bias_C1[outc] += delta[y * width_image_C1_CNN + x]; </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="keyword">void</span><span> CNN::update_weights_bias(</span><span class="keyword">const</span><span> </span><span class="datatypes">float</span><span>* delta, </span><span class="datatypes">float</span><span>* weight, </span><span class="datatypes">int</span><span> len) </span></span></li><li class="alt"><span>{ </span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < len; i++) { </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> tmp = delta[i] * delta[i]; </span></span></li><li class=""><span> weight[i] -= learning_rate_CNN * delta[i] / (std::sqrt(tmp) + eps_CNN); </span></li><li class="alt"><span> } </span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span><span class="datatypes">bool</span><span> CNN::UpdateWeights() </span></span></li><li class="alt"><span>{ </span></li><li class=""><span> update_weights_bias(delta_weight_C1, weight_C1, len_weight_C1_CNN); </span></li><li class="alt"><span> update_weights_bias(delta_bias_C1, bias_C1, len_bias_C1_CNN); </span></li><li class=""><span> </span></li><li class="alt"><span> update_weights_bias(delta_weight_S2, weight_S2, len_weight_S2_CNN); </span></li><li class=""><span> update_weights_bias(delta_bias_S2, bias_S2, len_bias_S2_CNN); </span></li><li class="alt"><span> </span></li><li class=""><span> update_weights_bias(delta_weight_C3, weight_C3, len_weight_C3_CNN); </span></li><li class="alt"><span> update_weights_bias(delta_bias_C3, bias_C3, len_bias_C3_CNN); </span></li><li class=""><span> </span></li><li class="alt"><span> update_weights_bias(delta_weight_S4, weight_S4, len_weight_S4_CNN); </span></li><li class=""><span> update_weights_bias(delta_bias_S4, bias_S4, len_bias_S4_CNN); </span></li><li class="alt"><span> </span></li><li class=""><span> update_weights_bias(delta_weight_C5, weight_C5, len_weight_C5_CNN); </span></li><li class="alt"><span> update_weights_bias(delta_bias_C5, bias_C5, len_bias_C5_CNN); </span></li><li class=""><span> </span></li><li class="alt"><span> update_weights_bias(delta_weight_output, weight_output, len_weight_output_CNN); </span></li><li class=""><span> update_weights_bias(delta_bias_output, bias_output, len_bias_output_CNN); </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">int</span><span> CNN::predict(</span><span class="keyword">const</span><span> unsigned </span><span class="datatypes">char</span><span>* data, </span><span class="datatypes">int</span><span> width, </span><span class="datatypes">int</span><span> height) </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> assert(data && width == width_image_input_CNN && height == height_image_input_CNN); </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span> scale_min = -1; </span></span></li><li class=""><span> <span class="keyword">const</span><span> </span><span class="datatypes">float</span><span> scale_max = 1; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="datatypes">float</span><span> tmp[width_image_input_CNN * height_image_input_CNN]; </span></span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> y = 0; y < height; y++) { </span></span></li><li class=""><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> x = 0; x < width; x++) { </span></span></li><li class="alt"><span> tmp[y * width + x] = (data[y * width + x] / 255.0) * (scale_max - scale_min) + scale_min; </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> data_single_image = &tmp[0]; </span></li><li class=""><span> </span></li><li class="alt"><span> Forward_C1(); </span></li><li class=""><span> Forward_S2(); </span></li><li class="alt"><span> Forward_C3(); </span></li><li class=""><span> Forward_S4(); </span></li><li class="alt"><span> Forward_C5(); </span></li><li class=""><span> Forward_output(); </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="datatypes">int</span><span> pos = -1; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> max_value = -9999.0; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < num_neuron_output_CNN; i++) { </span></span></li><li class=""><span> <span class="keyword">if</span><span> (neuron_output[i] > max_value) { </span></span></li><li class="alt"><span> max_value = neuron_output[i]; </span></li><li class=""><span> pos = i; </span></li><li class="alt"><span> } </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">return</span><span> pos; </span></span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">bool</span><span> CNN::readModelFile(</span><span class="keyword">const</span><span> </span><span class="datatypes">char</span><span>* name) </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> <span class="datatypes">FILE</span><span>* fp = fopen(name, </span><span class="string">"rb"</span><span>); </span></span></li><li class=""><span> <span class="keyword">if</span><span> (fp == NULL) { </span></span></li><li class="alt"><span> <span class="keyword">return</span><span> </span><span class="keyword">false</span><span>; </span></span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="datatypes">int</span><span> width_image_input =0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> height_image_input = 0; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> width_image_C1 = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> height_image_C1 = 0; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> width_image_S2 = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> height_image_S2 = 0; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> width_image_C3 = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> height_image_C3 = 0; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> width_image_S4 = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> height_image_S4 = 0; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> width_image_C5 = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> height_image_C5 = 0; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> width_image_output = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> height_image_output = 0; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="datatypes">int</span><span> width_kernel_conv = 0; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> height_kernel_conv = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> width_kernel_pooling = 0; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> height_kernel_pooling = 0; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="datatypes">int</span><span> num_map_input = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> num_map_C1 = 0; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> num_map_S2 = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> num_map_C3 = 0; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> num_map_S4 = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> num_map_C5 = 0; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> num_map_output = 0; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="datatypes">int</span><span> len_weight_C1 = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> len_bias_C1 = 0; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> len_weight_S2 = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> len_bias_S2 = 0; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> len_weight_C3 = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> len_bias_C3 = 0; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> len_weight_S4 = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> len_bias_S4 = 0; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> len_weight_C5 = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> len_bias_C5 = 0; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> len_weight_output = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> len_bias_output = 0; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="datatypes">int</span><span> num_neuron_input = 0; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> num_neuron_C1 = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> num_neuron_S2 = 0; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> num_neuron_C3 = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> num_neuron_S4 = 0; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> num_neuron_C5 = 0; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> num_neuron_output = 0; </span></span></li><li class=""><span> </span></li><li class="alt"><span> fread(&width_image_input, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fread(&height_image_input, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fread(&width_image_C1, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fread(&height_image_C1, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fread(&width_image_S2, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fread(&height_image_S2, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fread(&width_image_C3, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fread(&height_image_C3, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fread(&width_image_S4, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fread(&height_image_S4, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fread(&width_image_C5, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fread(&height_image_C5, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fread(&width_image_output, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fread(&height_image_output, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> </span></li><li class=""><span> fread(&width_kernel_conv, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fread(&height_kernel_conv, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fread(&width_kernel_pooling, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fread(&height_kernel_pooling, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> </span></li><li class="alt"><span> fread(&num_map_input, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fread(&num_map_C1, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fread(&num_map_S2, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fread(&num_map_C3, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fread(&num_map_S4, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fread(&num_map_C5, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fread(&num_map_output, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> </span></li><li class="alt"><span> fread(&len_weight_C1, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fread(&len_bias_C1, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fread(&len_weight_S2, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fread(&len_bias_S2, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fread(&len_weight_C3, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fread(&len_bias_C3, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fread(&len_weight_S4, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fread(&len_bias_S4, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fread(&len_weight_C5, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fread(&len_bias_C5, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fread(&len_weight_output, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fread(&len_bias_output, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> </span></li><li class=""><span> fread(&num_neuron_input, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fread(&num_neuron_C1, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fread(&num_neuron_S2, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fread(&num_neuron_C3, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fread(&num_neuron_S4, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fread(&num_neuron_C5, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fread(&num_neuron_output, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> </span></li><li class=""><span> fread(weight_C1, <span class="keyword">sizeof</span><span>(weight_C1), 1, fp); </span></span></li><li class="alt"><span> fread(bias_C1, <span class="keyword">sizeof</span><span>(bias_C1), 1, fp); </span></span></li><li class=""><span> fread(weight_S2, <span class="keyword">sizeof</span><span>(weight_S2), 1, fp); </span></span></li><li class="alt"><span> fread(bias_S2, <span class="keyword">sizeof</span><span>(bias_S2), 1, fp); </span></span></li><li class=""><span> fread(weight_C3, <span class="keyword">sizeof</span><span>(weight_C3), 1, fp); </span></span></li><li class="alt"><span> fread(bias_C3, <span class="keyword">sizeof</span><span>(bias_C3), 1, fp); </span></span></li><li class=""><span> fread(weight_S4, <span class="keyword">sizeof</span><span>(weight_S4), 1, fp); </span></span></li><li class="alt"><span> fread(bias_S4, <span class="keyword">sizeof</span><span>(bias_S4), 1, fp); </span></span></li><li class=""><span> fread(weight_C5, <span class="keyword">sizeof</span><span>(weight_C5), 1, fp); </span></span></li><li class="alt"><span> fread(bias_C5, <span class="keyword">sizeof</span><span>(bias_C5), 1, fp); </span></span></li><li class=""><span> fread(weight_output, <span class="keyword">sizeof</span><span>(weight_output), 1, fp); </span></span></li><li class="alt"><span> fread(bias_output, <span class="keyword">sizeof</span><span>(bias_output), 1, fp); </span></span></li><li class=""><span> </span></li><li class="alt"><span> fflush(fp); </span></li><li class=""><span> fclose(fp); </span></li><li class="alt"><span> </span></li><li class=""><span> out2wi_S2.clear(); </span></li><li class="alt"><span> out2bias_S2.clear(); </span></li><li class=""><span> out2wi_S4.clear(); </span></li><li class="alt"><span> out2bias_S4.clear(); </span></li><li class=""><span> </span></li><li class="alt"><span> calc_out2wi(width_image_C1_CNN, height_image_C1_CNN, width_image_S2_CNN, height_image_S2_CNN, num_map_S2_CNN, out2wi_S2); </span></li><li class=""><span> calc_out2bias(width_image_S2_CNN, height_image_S2_CNN, num_map_S2_CNN, out2bias_S2); </span></li><li class="alt"><span> calc_out2wi(width_image_C3_CNN, height_image_C3_CNN, width_image_S4_CNN, height_image_S4_CNN, num_map_S4_CNN, out2wi_S4); </span></li><li class=""><span> calc_out2bias(width_image_S4_CNN, height_image_S4_CNN, num_map_S4_CNN, out2bias_S4); </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">bool</span><span> CNN::saveModelFile(</span><span class="keyword">const</span><span> </span><span class="datatypes">char</span><span>* name) </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> <span class="datatypes">FILE</span><span>* fp = fopen(name, </span><span class="string">"wb"</span><span>); </span></span></li><li class=""><span> <span class="keyword">if</span><span> (fp == NULL) { </span></span></li><li class="alt"><span> <span class="keyword">return</span><span> </span><span class="keyword">false</span><span>; </span></span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="datatypes">int</span><span> width_image_input = width_image_input_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> height_image_input = height_image_input_CNN; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> width_image_C1 = width_image_C1_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> height_image_C1 = height_image_C1_CNN; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> width_image_S2 = width_image_S2_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> height_image_S2 = height_image_S2_CNN; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> width_image_C3 = width_image_C3_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> height_image_C3 = height_image_C3_CNN; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> width_image_S4 = width_image_S4_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> height_image_S4 = height_image_S4_CNN; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> width_image_C5 = width_image_C5_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> height_image_C5 = height_image_C5_CNN; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> width_image_output = width_image_output_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> height_image_output = height_image_output_CNN; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="datatypes">int</span><span> width_kernel_conv = width_kernel_conv_CNN; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> height_kernel_conv = height_kernel_conv_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> width_kernel_pooling = width_kernel_pooling_CNN; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> height_kernel_pooling = height_kernel_pooling_CNN; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="datatypes">int</span><span> num_map_input = num_map_input_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> num_map_C1 = num_map_C1_CNN; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> num_map_S2 = num_map_S2_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> num_map_C3 = num_map_C3_CNN; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> num_map_S4 = num_map_S4_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> num_map_C5 = num_map_C5_CNN; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> num_map_output = num_map_output_CNN; </span></span></li><li class="alt"><span> </span></li><li class=""><span> <span class="datatypes">int</span><span> len_weight_C1 = len_weight_C1_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> len_bias_C1 = len_bias_C1_CNN; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> len_weight_S2 = len_weight_S2_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> len_bias_S2 = len_bias_S2_CNN; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> len_weight_C3 = len_weight_C3_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> len_bias_C3 = len_bias_C3_CNN; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> len_weight_S4 = len_weight_S4_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> len_bias_S4 = len_bias_S4_CNN; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> len_weight_C5 = len_weight_C5_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> len_bias_C5 = len_bias_C5_CNN; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> len_weight_output = len_weight_output_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> len_bias_output = len_bias_output_CNN; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="datatypes">int</span><span> num_neuron_input = num_neuron_input_CNN; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> num_neuron_C1 = num_neuron_C1_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> num_neuron_S2 = num_neuron_S2_CNN; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> num_neuron_C3 = num_neuron_C3_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> num_neuron_S4 = num_neuron_S4_CNN; </span></span></li><li class=""><span> <span class="datatypes">int</span><span> num_neuron_C5 = num_neuron_C5_CNN; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> num_neuron_output = num_neuron_output_CNN; </span></span></li><li class=""><span> </span></li><li class="alt"><span> fwrite(&width_image_input, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fwrite(&height_image_input, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fwrite(&width_image_C1, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fwrite(&height_image_C1, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fwrite(&width_image_S2, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fwrite(&height_image_S2, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fwrite(&width_image_C3, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fwrite(&height_image_C3, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fwrite(&width_image_S4, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fwrite(&height_image_S4, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fwrite(&width_image_C5, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fwrite(&height_image_C5, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fwrite(&width_image_output, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fwrite(&height_image_output, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> </span></li><li class=""><span> fwrite(&width_kernel_conv, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fwrite(&height_kernel_conv, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fwrite(&width_kernel_pooling, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fwrite(&height_kernel_pooling, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> </span></li><li class="alt"><span> fwrite(&num_map_input, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fwrite(&num_map_C1, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fwrite(&num_map_S2, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fwrite(&num_map_C3, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fwrite(&num_map_S4, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fwrite(&num_map_C5, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fwrite(&num_map_output, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> </span></li><li class="alt"><span> fwrite(&len_weight_C1, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fwrite(&len_bias_C1, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fwrite(&len_weight_S2, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fwrite(&len_bias_S2, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fwrite(&len_weight_C3, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fwrite(&len_bias_C3, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fwrite(&len_weight_S4, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fwrite(&len_bias_S4, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fwrite(&len_weight_C5, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fwrite(&len_bias_C5, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fwrite(&len_weight_output, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fwrite(&len_bias_output, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> </span></li><li class=""><span> fwrite(&num_neuron_input, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fwrite(&num_neuron_C1, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fwrite(&num_neuron_S2, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fwrite(&num_neuron_C3, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fwrite(&num_neuron_S4, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> fwrite(&num_neuron_C5, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class=""><span> fwrite(&num_neuron_output, <span class="keyword">sizeof</span><span>(</span><span class="datatypes">int</span><span>), 1, fp); </span></span></li><li class="alt"><span> </span></li><li class=""><span> fwrite(weight_C1, <span class="keyword">sizeof</span><span>(weight_C1), 1, fp); </span></span></li><li class="alt"><span> fwrite(bias_C1, <span class="keyword">sizeof</span><span>(bias_C1), 1, fp); </span></span></li><li class=""><span> fwrite(weight_S2, <span class="keyword">sizeof</span><span>(weight_S2), 1, fp); </span></span></li><li class="alt"><span> fwrite(bias_S2, <span class="keyword">sizeof</span><span>(bias_S2), 1, fp); </span></span></li><li class=""><span> fwrite(weight_C3, <span class="keyword">sizeof</span><span>(weight_C3), 1, fp); </span></span></li><li class="alt"><span> fwrite(bias_C3, <span class="keyword">sizeof</span><span>(bias_C3), 1, fp); </span></span></li><li class=""><span> fwrite(weight_S4, <span class="keyword">sizeof</span><span>(weight_S4), 1, fp); </span></span></li><li class="alt"><span> fwrite(bias_S4, <span class="keyword">sizeof</span><span>(bias_S4), 1, fp); </span></span></li><li class=""><span> fwrite(weight_C5, <span class="keyword">sizeof</span><span>(weight_C5), 1, fp); </span></span></li><li class="alt"><span> fwrite(bias_C5, <span class="keyword">sizeof</span><span>(bias_C5), 1, fp); </span></span></li><li class=""><span> fwrite(weight_output, <span class="keyword">sizeof</span><span>(weight_output), 1, fp); </span></span></li><li class="alt"><span> fwrite(bias_output, <span class="keyword">sizeof</span><span>(bias_output), 1, fp); </span></span></li><li class=""><span> </span></li><li class="alt"><span> fflush(fp); </span></li><li class=""><span> fclose(fp); </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="keyword">return</span><span> </span><span class="keyword">true</span><span>; </span></span></li><li class="alt"><span>} </span></li><li class=""><span> </span></li><li class="alt"><span><span class="datatypes">float</span><span> CNN::test() </span></span></li><li class=""><span>{ </span></li><li class="alt"><span> <span class="datatypes">int</span><span> count_accuracy = 0; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> num = 0; num < num_patterns_test_CNN; num++) { </span></span></li><li class=""><span> data_single_image = data_input_test + num * num_neuron_input_CNN; </span></li><li class="alt"><span> data_single_label = data_output_test + num * num_neuron_output_CNN; </span></li><li class=""><span> </span></li><li class="alt"><span> Forward_C1(); </span></li><li class=""><span> Forward_S2(); </span></li><li class="alt"><span> Forward_C3(); </span></li><li class=""><span> Forward_S4(); </span></li><li class="alt"><span> Forward_C5(); </span></li><li class=""><span> Forward_output(); </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="datatypes">int</span><span> pos_t = -1; </span></span></li><li class="alt"><span> <span class="datatypes">int</span><span> pos_y = -2; </span></span></li><li class=""><span> <span class="datatypes">float</span><span> max_value_t = -9999.0; </span></span></li><li class="alt"><span> <span class="datatypes">float</span><span> max_value_y = -9999.0; </span></span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">for</span><span> (</span><span class="datatypes">int</span><span> i = 0; i < num_neuron_output_CNN; i++) { </span></span></li><li class=""><span> <span class="keyword">if</span><span> (neuron_output[i] > max_value_y) { </span></span></li><li class="alt"><span> max_value_y = neuron_output[i]; </span></li><li class=""><span> pos_y = i; </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">if</span><span> (data_single_label[i] > max_value_t) { </span></span></li><li class=""><span> max_value_t = data_single_label[i]; </span></li><li class="alt"><span> pos_t = i; </span></li><li class=""><span> } </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> <span class="keyword">if</span><span> (pos_y == pos_t) { </span></span></li><li class=""><span> ++count_accuracy; </span></li><li class="alt"><span> } </span></li><li class=""><span> </span></li><li class="alt"><span> Sleep(1); </span></li><li class=""><span> } </span></li><li class="alt"><span> </span></li><li class=""><span> <span class="comment">//std::cout << "count_accuracy: " << count_accuracy << std::endl;</span><span> </span></span></li><li class="alt"><span> <span class="keyword">return</span><span> (count_accuracy * 1.0 / num_patterns_test_CNN); </span></span></li><li class=""><span>} </span></li><li class="alt"><span> </span></li><li class=""><span>} </span></li>
内容来自用户分享和网络整理，不保证内容的准确性，如有侵权内容，可联系管理员处理 
相关文章推荐
- bp神经网络及matlab实现
- CUDA搭建
- 如何用70行代码实现深度神经网络算法
- 基于神经网络的预测模型
- 神经网络初步学习手记
- convolutional neural network
- RBF的一点个人理解
- 利用自收敛深度人工神经网络构建(DNN)构建多语种大词汇量连续语音识别系统
- UFLDL Exercise: Convolutional Neural Network
- 基于遗传算法(GA)的神经网络训练算法
- 神经网络 caffe 的 vs2013 版本代码
- 传统BP神经网络完整例子（电力负荷预测）
- Matlab 神经网络工具箱
- 使用深度卷积网络和支撑向量机实现的商标检测与分类的例子
- 对Pedestrian Detection aided by Deep Learning Semantic Tasks的小结
- Google最新人工智能算法RankBrain的实现--写在后面的话
- 备份：创建VS工程使用神经网络库——FANN
- 卷积神经网络知识要点
- Deep learning: autoencoders and sparsity