第4门课程-卷积神经网络-第二周作业1-基于Keras的人脸表情分类

0- 背景：

从人脸图像中的表情判断一个人是否快乐。本文将基于Keras实现该功能。Keras是一个更高级的API，其底层框架可以是TensorFlow或者CNTK。

1-数据加载：

导入依赖的库：

import numpy as np
#import tensorflow as tf
from keras import layers
from keras.layers import Input, Dense, Activation, ZeroPadding2D, BatchNormalization, Flatten, Conv2D
from keras.layers import AveragePooling2D, MaxPooling2D, Dropout, GlobalMaxPooling2D, GlobalAveragePooling2D
from keras.models import Model
from keras.preprocessing import image
from keras.utils import layer_utils
from keras.utils.data_utils import get_file
from keras.applications.imagenet_utils import preprocess_input
import pydot
from IPython.display import SVG
from keras.utils.vis_utils import model_to_dot
from keras.utils import plot_model
from kt_utils import *

import keras.backend as K
K.set_image_data_format('channels_last')
import matplotlib.pyplot as plt
from matplotlib.pyplot import imshow

%matplotlib inline

导入数据：

X_train_orig, Y_train_orig, X_test_orig, Y_test_orig, classes = load_dataset()

# Normalize image vectors
X_train = X_train_orig/255.
X_test = X_test_orig/255.

# Reshape
Y_train = Y_train_orig.T
Y_test = Y_test_orig.T

print ("number of training examples = " + str(X_train.shape[0]))
print ("number of test examples = " + str(X_test.shape[0]))
print ("X_train shape: " + str(X_train.shape))
print ("Y_train shape: " + str(Y_train.shape))
print ("X_test shape: " + str(X_test.shape))
print ("Y_test shape: " + str(Y_test.shape))

运行结果：

number of training examples = 600
number of test examples = 150
X_train shape: (600, 64, 64, 3)
Y_train shape: (600, 1)
X_test shape: (150, 64, 64, 3)
Y_test shape: (150, 1)

由此可知：

图像的尺寸是 (64,64,3)

训练集600张

测试集150张

2- 创建模型

基于Keras我们可以快速实现我们的模型，以下是一个模型案例：

def model(input_shape):
# Define the input placeholder as a tensor with shape input_shape. Think of this as your input image!
X_input = Input(input_shape)

# Zero-Padding: pads the border of X_input with zeroes
X = ZeroPadding2D((3, 3))(X_input)

# CONV -> BN -> RELU Block applied to X
X = Conv2D(32, (7, 7), strides = (1, 1), name = 'conv0')(X)
X = BatchNormalization(axis = 3, name = 'bn0')(X)
X = Activation('relu')(X)

# MAXPOOL
X = MaxPooling2D((2, 2), name='max_pool')(X)

# FLATTEN X (means convert it to a vector) + FULLYCONNECTED
X = Flatten()(X)
X = Dense(1, activation='sigmoid', name='fc')(X)

# Create model. This creates your Keras model instance, you'll use this instance to train/test the model.
model = Model(inputs = X_input, outputs = X, name='HappyModel')

return model

从上可以看出前向传播过程都用了一个相同的变量X，并在前向传播过程不断覆盖到旧值，这与此前我们基于TensorFlow构建模型时候使用到的 X, Z1, A1, Z2, A2等变量不同。

创建HappyModel模型如下：

# GRADED FUNCTION: HappyModel

def HappyModel(input_shape):
"""
Implementation of the HappyModel.

Arguments:
input_shape -- shape of the images of the dataset

Returns:
model -- a Model() instance in Keras
"""

### START CODE HERE ###
# Feel free to use the suggested outline in the text above to get started, and run through the whole
# exercise (including the later portions of this notebook) once. The come back also try out other
# network architectures as well.
X_input = Input(shape=input_shape)
X = ZeroPadding2D(padding=(1, 1))(X_input)
X = Conv2D(8, kernel_size=(3,3), strides=(1,1))(X)#8个filter
X = BatchNormalization(axis=3)(X)#??
X = Activation('relu')(X)
X = MaxPooling2D(pool_size=(2,2), strides=(2,2), padding='valid')(X)

X = ZeroPadding2D(padding=(1, 1))(X)
X = Conv2D(16, kernel_size=(3,3), strides=(1,1))(X)
X = BatchNormalization(axis=3)(X)
X = Activation('relu')(X)
X = MaxPooling2D(pool_size=(2,2), strides=(2,2), padding='valid')(X)

X = ZeroPadding2D(padding=(1, 1))(X)
X = Conv2D(32, kernel_size=(3,3), strides=(1,1))(X)
X = BatchNormalization(axis=3)(X)
X = Activation('relu')(X)
X = MaxPooling2D(pool_size=(2,2), strides=(2,2), padding='valid')(X)

# FC
X = Flatten()(X)
Y = Dense(1, activation='sigmoid')(X)#就是全连接层

model = Model(inputs = X_input, outputs = Y, name='HappyModel')
### END CODE HERE ###

return model

在模型建立之后，可以根据以下4步进行模型的训练和测试：

调用模型创建函数，创建模型

编译模型：

model.compile(optimizer = "...", loss = "...", metrics = ["accuracy"])

在训练集上进行模型训练：

model.fit(x = ..., y = ..., epochs = ..., batch_size = ...)

在测试数据集上进行模型测试：

model.evaluate(x = ..., y = ...)

模型相关资料 Keras documentation.

具体代码如下：

Step1：

### START CODE HERE ### (1 line)
happyModel = HappyModel((64, 64, 3))
### END CODE HERE ###

Step2：

在编译模型的同时设置学习率等超参数。

### START CODE HERE ### (1 line)
import time
import keras
start=time.time()
happyModel.compile(optimizer=keras.optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.0), loss='binary_crossentropy', metrics=['accuracy'])
### END CODE HERE ###
stop=time.time()
print("compile step cost=",str(stop-start))

运行结果：

compile step cost= 0.3502788543701172

Step3：

在训练模型的时候，选择合适的 epochs次数和batch size。

### START CODE HERE ### (1 line)
start=time.time()
happyModel.fit(x=X_train, y=Y_train, batch_size=16, epochs=20)
### END CODE HERE ###
stop=time.time()
print("fit step cost=",str(stop-start))

运行结果如下：

Epoch 1/20
600/600 [==============================] - 10s 16ms/step - loss: 0.5246 - acc: 0.7733
Epoch 2/20
600/600 [==============================] - 7s 12ms/step - loss: 0.2175 - acc: 0.9300
Epoch 3/20
600/600 [==============================] - 7s 11ms/step - loss: 0.1363 - acc: 0.9667
Epoch 4/20
600/600 [==============================] - 7s 11ms/step - loss: 0.0904 - acc: 0.9833
Epoch 5/20
600/600 [==============================] - 6s 10ms/step - loss: 0.0780 - acc: 0.9817
Epoch 6/20
600/600 [==============================] - 6s 9ms/step - loss: 0.0557 - acc: 0.9917
Epoch 7/20
600/600 [==============================] - 5s 9ms/step - loss: 0.0433 - acc: 0.9933
Epoch 8/20
600/600 [==============================] - 6s 9ms/step - loss: 0.0593 - acc: 0.9850
Epoch 9/20
600/600 [==============================] - 5s 9ms/step - loss: 0.0305 - acc: 0.9967
Epoch 10/20
600/600 [==============================] - 5s 9ms/step - loss: 0.0281 - acc: 0.9967
Epoch 11/20
600/600 [==============================] - 5s 9ms/step - loss: 0.0238 - acc: 0.9967
Epoch 12/20
600/600 [==============================] - 5s 9ms/step - loss: 0.0219 - acc: 0.9967
Epoch 13/20
600/600 [==============================] - 5s 9ms/step - loss: 0.0209 - acc: 0.9967
Epoch 14/20
600/600 [==============================] - 5s 9ms/step - loss: 0.0157 - acc: 0.9983
Epoch 15/20
600/600 [==============================] - 5s 9ms/step - loss: 0.0124 - acc: 0.9983
Epoch 16/20
600/600 [==============================] - 5s 9ms/step - loss: 0.0100 - acc: 1.0000
Epoch 17/20
600/600 [==============================] - 5s 9ms/step - loss: 0.0098 - acc: 0.9983
Epoch 18/20
600/600 [==============================] - 5s 9ms/step - loss: 0.0091 - acc: 0.9983
Epoch 19/20
600/600 [==============================] - 5s 9ms/step - loss: 0.0059 - acc: 1.0000
Epoch 20/20
600/600 [==============================] - 5s 9ms/step - loss: 0.0063 - acc: 0.9983
fit step cost= 122.55410814285278

注意：如果在运行一次

fit()

，该模型会接着已经学习到的参数继续训练，而不会重新初始化参数后再训练。

Step4：

### START CODE HERE ### (1 line)
start=time.time()
preds = happyModel.evaluate(x=X_test, y=Y_test)
### END CODE HERE ###
print ("Loss = " + str(preds[0]))
print ("Test Accuracy = " + str(preds[1]))
stop=time.time()
print("test step cost=",str(stop-start))

运用测试集对模型估计的结果：

150/150 [==============================] - 1s 6ms/step
Loss = 0.095218008558
Test Accuracy = 0.959999997616
test step cost= 0.921884298324585

我们可以设置不同的 mini batch size或者epochs 来对照结果。比如epochs 次数选择少些，再或者mini batch size选择大些。

当测试集的正确率过低，我们可以通过以下的方法进行尝试提升：

尝试以下模块： CONV->BATCHNORM->RELU :

X = Conv2D(32, (3, 3), strides = (1, 1), name = 'conv0')(X)
X = BatchNormalization(axis = 3, name = 'bn0')(X)
X = Activation('relu')(X)

直至长和宽的维度都相对较低，且通道数较高（如32）。此时，有用的信息很多被编码存于这多个通道上，然后将其拉平，接入全连接层。

可以在这些模块之后接池化层MAXPOOL，以降低长宽上的维度。.

或者换一个optimizer.，一般Adam优化器足够了。

当运行时候内存不够时，降低batch_size (比如12)

增加epochs次数，直到train accuracy变高

3- 作用于真实数据：

### START CODE HERE ###
img_path = 'images/my_image.jpg'
### END CODE HERE ###
img = image.load_img(img_path, target_size=(64, 64))
imshow(img)

x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)

print(happyModel.predict(x))

运行结果：

[[ 1.]]

4- 其他函数：

model.summary()

:

以表格形式打印各层信息。

plot_model()

:

绘制模型的流程图，另外，我们可以用

SVG()

将模型保存为”.png” 格式。

happyModel.summary()

打印结果如下：

_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_1 (InputLayer)         (None, 64, 64, 3)         0
_________________________________________________________________
zero_padding2d_1 (ZeroPaddin (None, 66, 66, 3)         0
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 64, 64, 8)         224
_________________________________________________________________
batch_normalization_1 (Batch (None, 64, 64, 8)         32
_________________________________________________________________
activation_1 (Activation)    (None, 64, 64, 8)         0
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 32, 32, 8)         0
_________________________________________________________________
zero_padding2d_2 (ZeroPaddin (None, 34, 34, 8)         0
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 32, 32, 16)        1168
_________________________________________________________________
batch_normalization_2 (Batch (None, 32, 32, 16)        64
_________________________________________________________________
activation_2 (Activation)    (None, 32, 32, 16)        0
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 16, 16, 16)        0
_________________________________________________________________
zero_padding2d_3 (ZeroPaddin (None, 18, 18, 16)        0
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 16, 16, 32)        4640
_________________________________________________________________
batch_normalization_3 (Batch (None, 16, 16, 32)        128
_________________________________________________________________
activation_3 (Activation)    (None, 16, 16, 32)        0
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 8, 8, 32)          0
_________________________________________________________________
flatten_1 (Flatten)          (None, 2048)              0
_________________________________________________________________
dense_1 (Dense)              (None, 1)                 2049
=================================================================
Total params: 8,305
Trainable params: 8,193
Non-trainable params: 112
________________________________

模型流程图绘制：

plot_model(happyModel, to_file='HappyModel.png')
SVG(model_to_dot(happyModel).create(prog='dot', format='svg'))
#这里需要注意一点,否则会报错：
#ImportError: Failed to import pydot. You must install pydot and graphviz for `pydotprint` to work.

注意，可能出现以下的错误：

ImportError: Failed to import pydot. You must install pydot and graphviz for `pydotprint` to work.

解决方案如下： 报错的原因其实不在于pydot，跟python包没有关系，而是因为graphviz需要安装二进制执行文件（跟imagick类似），所以还需要去官网下一个graphviz安装包安装： https://graphviz.gitlab.io/_pages/Download/Download_windows.html

然后在环境变量的path里面添加可执行文件的环境变量即可。Windows系统下，可能需要重启。

如果是linux系统的话，安装对应的lib apt-get install -y graphviz libgraphviz-dev