Scala实现神经网络 ANN , 偏函数式风格

为了加深理解,用Scala实现全链接的二分类神经网络, 风格偏函数式, 没有使用循环, 不使用var变量, 需要使用Breeze线性代数库.

用Python制造了一组相对难分的训练数据并写入文件,如图.



代码如下

先导入一些库

package ann
import java.io.File
import breeze.linalg.{*, max, min, sum, DenseMatrix => BDM, DenseVector => BDV} //那个*号居然是广播运算用的
import breeze.linalg.InjectNumericOps
import breeze.{numerics => BN} // 导入函数库
import breeze.plot._ //作图库
import scala.annotation.tailrec
import scala.math
import scala.util.Random

定义网络, 神经网络的本质就是一个函数, 给个输入,有个输出

//封装一下网络参数, 主要是权重,偏置和激活函数
case class LayerPara(weight:BDM[Double] , bias:BDV[Double],activationFunc: ActivationFunc)

trait Layer{
def apply(input:BDM[Double] ):BDM[Double]
//定一个类似List的操作,表示网络链接
def <=:(nextLayerPara:LayerPara):HLayer = HLayer(nextLayerPara,this) //这是个具体方法, 会被直接装配到子类里面
}
//输入层只有一个, 弄个单例
object Input extends Layer{
def apply(input:BDM[Double] ):BDM[Double]  = input
}
//隐层,
case class HLayer( layerPara:LayerPara,next:Layer ) extends Layer{
val w:BDM[Double] = layerPara.weight
val b:BDV[Double] = layerPara.bias
def apply(input: BDM[Double]  ):BDM[Double] = {
val net = next(input) * w
layerPara.activationFunc( net(*,::)+b )
}
}

定义激活函数

trait ActivationFunc{
def apply(x: BDM[Double]  ): BDM[Double]
def derivative(x:BDM[Double]  ): BDM[Double]
}

object ActivationFunc {
object sigmoid extends ActivationFunc {
@inline
def apply(x: BDM[Double]  ): BDM[Double]  = 1.0 /:/ (1.0 +:+ BN.exp(-x))
def derivative(x: BDM[Double]  ): BDM[Double]   = sigmoid(x) *:* (1.0 -:- sigmoid(x)) //把导数也定义了
}

object relu extends ActivationFunc{
@inline
def apply(x: BDM[Double]):BDM[Double] = x.map(v=>if(v>0.0) v else 0.0)
def derivative(x: BDM[Double]): BDM[Double] = x.map(v=> if (v>0.0) 1.0 else 0.0)
}
}

读文件

def fileLoader(file: String): (BDM[Double] , BDM[Double] ) = {
val dataRaw = breeze.linalg.csvread(new File(file))
val dataMat = dataRaw(::, 0 to -2) //取到倒数第二列
val labels = dataRaw(::,-1) //取最后一列
(dataMat, labels.toDenseMatrix.t)
}

更新网络, 输入一批数据, 更新产生一个新的网络,而不是修改原处

def update(batch:BDM[Double],net:HLayer, error:BDM[Double],learnRate:Double):HLayer={
val size = batch.rows.toDouble
val nextOut = net.next(batch) //下一层的输出
val errorTerm = net.layerPara.activationFunc.derivative({val netValue = nextOut * net.w; netValue(*,::) + net.b}) *:* error //计算误差项
val del_w = ( nextOut.t * errorTerm ) *:* learnRate /:/ size //w的改变量
val del_b =  sum(errorTerm(::,*)).t  *:* learnRate /:/ size //b的改变量
net.next match {
case Input => HLayer(LayerPara(net.w -:- del_w ,net.b -:- del_b, net.layerPara.activationFunc),Input)
case nextLayer:HLayer => HLayer(LayerPara(net.w -:- del_w ,net.b -:- del_b, net.layerPara.activationFunc),
update(batch,nextLayer, errorTerm * net.w.t, learnRate )) //递归地将误差传下去
}
}

用尾递归代替循环进行训练

@tailrec
def train(trainData:BDM[Double],labels:BDM[Double],dataLength:Int,
net:HLayer,
learnRate:Double,
batchSize:Int,
maxIter:Int, ii:Int=0,
trainErrorLog:Array[Double]= new Array[Double](0)): (HLayer,Array[Double]) ={

val batchStart = Random.nextInt(dataLength - batchSize)
val slice = batchStart until (batchStart+batchSize)
val batch = dataMat(slice,::)
val netUpdated = update(batch,net, net(batch) -:- labels(slice,::),learnRate)

val newLog = {
if (ii % 20 == 0) {
val trainError = 0.5 * sum( BN.pow(netUpdated(trainData) -:- labels,2) )
println("训练误差:" + trainError)
trainErrorLog :+ trainError
}
else {
trainErrorLog
}
}

if (ii>=maxIter) (netUpdated, trainErrorLog)
else train(trainData,labels,dataLength,netUpdated,learnRate,batchSize,maxIter,ii+1,newLog)
}

主程序, 需要把上面的函数定义加入进来的

object ANN extends App{
import ActivationFunc.{sigmoid, relu}
val (dataMat, label) = fileLoader("./ANNTestData.csv")
//初始化各层参数
val l1=LayerPara(BDM.zeros[Double](2,10).map(_+Random.nextGaussian()/math.sqrt(1
b8f8
0)),BDV.zeros(10),relu)
val l2=LayerPara(BDM.zeros[Double](10,8).map(_+Random.nextGaussian()/math.sqrt(8)),BDV.zeros(8),sigmoid)
val l3=LayerPara(BDM.zeros[Double](8,6).map(_+Random.nextGaussian()/math.sqrt(6)),BDV.zeros(6),relu)
val l4=LayerPara(BDM.zeros[Double](6,1).map(_+Random.nextGaussian()/math.sqrt(1)),BDV.zeros(1),sigmoid)

val initNet = l4<=:l3<=:l2<=:l1<=:Input // 串接网络
//训练 , 传入数据, 标签, 数据大小, 初始网络, 学习率0.28, 大小为50的批次, 训练55000步.
val (trainedNet, errorLog) = train(dataMat,label,dataMat.rows,initNet,0.28,50,55000)

//下面是作图的
//作出误差
val f = Figure()
f.width = 600
f.height = 800
val sub = f.subplot(2,1,0)
val ranges =BDV.range(0,errorLog.length).map(_.toDouble)
sub += plot(ranges,BDV(errorLog))

//作出分类结果
val sub2 = f.subplot(2,1,1)
val x = dataMat(::,0)
val y = dataMat(::,1)
val newLabels = trainedNet(dataMat)
val colorPalette = GradientPaintScale(0.0, 1.0, PaintScale.Rainbow)
sub2 += scatter(x,y,_ =>0.02, i=> if (newLabels(i,0)<0.5) colorPalette(0.86) else colorPalette(0.26))
f.saveas("ann.pdf")

结果如图



结果表明神经网络作出了基本正确的分类