DeepLearning.ai作业:(4-2)-- 深度卷积网络实例探究(Deep convolutional models:case studies)
title: ‘DeepLearning.ai作业:(4-2)-- 深度卷积网络实例探究(Deep convolutional models:case studies)’
id: dl-ai-4-2h
tags:
- dl.ai
- homework
categories: - AI
- Deep Learning
date: 2018-10-09 19:20:57
首发于个人博客:fangzh.top,欢迎来访
本周作业分为两部分,一部分是keras的基本使用,另一部分是ResNet的构建。
Part1: Keras – Tutorial
Keras是TensorFlow的高层封装,可以更高效的实现神经网络的搭建。
先导入库
import numpy as np from keras import layers from keras.layers import Input, Dense, Activation, ZeroPadding2D, BatchNormalization, Flatten, Conv2D from keras.layers import AveragePooling2D, MaxPooling2D, Dropout, GlobalMaxPooling2D, GlobalAveragePooling2D from keras.models import Model from keras.preprocessing import image from keras.utils import layer_utils from keras.utils.data_utils import get_file from keras.applications.imagenet_utils import preprocess_input import pydot from IPython.display import SVG from keras.utils.vis_utils import model_to_dot from keras.utils import plot_model from kt_utils import * import keras.backend as K K.set_image_data_format('channels_last') import matplotlib.pyplot as plt from matplotlib.pyplot import imshow %matplotlib inline[/code]
构建模型
def HappyModel(input_shape): """ Implementation of the HappyModel. Arguments: input_shape -- shape of the images of the dataset Returns: model -- a Model() instance in Keras """ ### START CODE HERE ### # Feel free to use the suggested outline in the text above to get started, and run through the whole # exercise (including the later portions of this notebook) once. The come back also try out other # network architectures as well. X_input = Input(input_shape) X = ZeroPadding2D((3, 3))(X_input) X = Conv2D(32,(7,7),strides=(1,1),name="Conv0")(X) X = BatchNormalization(axis = 3, name = 'bn0')(X) X = Activation('relu')(X) X = MaxPooling2D((2, 2), name='max_pool')(X) X = Flatten()(X) X = Dense(1, activation='sigmoid', name='fc')(X) model = Model(inputs = X_input, outputs = X, name='HappyModel') ### END CODE HERE ### return model[/code]
然后实例化这个模型
### START CODE HERE ### (1 line) happyModel = HappyModel(X_train.shape[1:]) ### END CODE HERE ###[/code]
进行优化器和loss的选择
### START CODE HERE ### (1 line) happyModel.compile(optimizer='Adam',loss='binary_crossentropy',metrics=['accuracy']) ### END CODE HERE ###[/code]
训练
### START CODE HERE ### (1 line) happyModel.fit(x=X_train,y = Y_train,epochs=10,batch_size=32) ### END CODE HERE ###[/code]
预测:
### START CODE HERE ### (1 line) preds = happyModel.evaluate(X_test,Y_test) ### END CODE HERE ### print() print ("Loss = " + str(preds[0])) print ("Test Accuracy = " + str(preds[1]))[/code]
可以用summary()来看看详细信息:
happyModel.summary()[/code]
_________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_1 (InputLayer) (None, 64, 64, 3) 0 _________________________________________________________________ zero_padding2d_1 (ZeroPaddin (None, 70, 70, 3) 0 _________________________________________________________________ Conv0 (Conv2D) (None, 64, 64, 32) 4736 _________________________________________________________________ bn0 (BatchNormalization) (None, 64, 64, 32) 128 _________________________________________________________________ activation_1 (Activation) (None, 64, 64, 32) 0 _________________________________________________________________ max_pool (MaxPooling2D) (None, 32, 32, 32) 0 _________________________________________________________________ flatten_1 (Flatten) (None, 32768) 0 _________________________________________________________________ fc (Dense) (None, 1) 32769 ================================================================= Total params: 37,633 Trainable params: 37,569 Non-trainable params: 64 _________________________________________________________________
用plot_model()来得到详细的graph
plot_model(happyModel, to_file='HappyModel.png') SVG(model_to_dot(happyModel).create(prog='dot', format='svg'))[/code]
Part2: Residual Networks
主要有两个步骤:
- 构建基本的ResNet的块
- 将块放到一起,变成一个网络,来做图像分类
1 - The problem of very deep neural networks
这一部分非常深的神经网络的一些问题,主要是参数会变得很小或者爆炸,这样子训练的时候就会收敛的很慢,因此,用残差网络可以有效的改善这个问题。
2 - Building a Residual Network
根据输入输入的维度不同,分为两种块:
1. identity block(一致块)
可以看到,identity block的前后两端维度是一致的,可以直接相加。
在这里我们实现了一个跳跃三层的块。
基本结构是:
First component of main path:
- The first CONV2D has F1F1 filters of shape (1,1) and a stride of (1,1). Its padding is “valid” and its name should be
conv_name_base + '2a'
. Use 0 as the seed for the random initialization. - The first BatchNorm is normalizing the channels axis. Its name should be
bn_name_base + '2a'
. - Then apply the ReLU activation function. This has no name and no hyperparameters.
Second component of main path:
- The second CONV2D has F2F2 filters of shape (f,f)(f,f) and a stride of (1,1). Its padding is “same” and its name should be
conv_name_base + '2b'
. Use 0 as the seed for the random initialization. - The second BatchNorm is normalizing the channels axis. Its name should be
bn_name_base + '2b'
. - Then apply the ReLU activation function. This has no name and no hyperparameters.
Third component of main path:
- The third CONV2D has F3F3 filters of shape (1,1) and a stride of (1,1). Its padding is “valid” and its name should be
conv_name_base + '2c'
. Use 0 as the seed for the random initialization. - The third BatchNorm is normalizing the channels axis. Its name should be
bn_name_base + '2c'
. Note that there is no ReLU activation function in this component.
Final step:
- The shortcut and the input are added together.
- Then apply the ReLU activation function. This has no name and no hyperparameters.
注意在跳跃相加部分要用函数keras的函授Add(),不能用加号,不然会出错。
这里f是卷积核的大小,filters是这三层卷积层的深度的list,stage指的是哪一大层的网络,用来取名字的,后面有用,block是在stage下的某一层的网络,用a,b,c,d等字母表示。
# GRADED FUNCTION: identity_block def identity_block(X, f, filters, stage, block): """ Implementation of the identity block as defined in Figure 3 Arguments: X -- input tensor of shape (m, n_H_prev, n_W_prev, n_C_prev) f -- integer, specifying the shape of the middle CONV's window for the main path filters -- python list of integers, defining the number of filters in the CONV layers of the main path stage -- integer, used to name the layers, depending on their position in the network block -- string/character, used to name the layers, depending on their position in the network Returns: X -- output of the identity block, tensor of shape (n_H, n_W, n_C) """ # defining name basis conv_name_base = 'res' + str(stage) + block + '_branch' bn_name_base = 'bn' + str(stage) + block + '_branch' # Retrieve Filters F1, F2, F3 = filters # Save the input value. You'll need this later to add back to the main path. X_shortcut = X # First component of main path X = Conv2D(filters = F1, kernel_size = (1, 1), strides = (1,1), padding = 'valid', name = conv_name_base + '2a', kernel_initializer = glorot_uniform(seed=0))(X) X = BatchNormalization(axis = 3, name = bn_name_base + '2a')(X) X = Activation('relu')(X) ### START CODE HERE ### # Second component of main path (≈3 lines) X = Conv2D(filters = F2, kernel_size = (f, f), strides= (1,1), padding = 1d80f 'same', name = conv_name_base + '2b', kernel_initializer = glorot_uniform(seed=0))(X) X = BatchNormalization(axis = 3, name = bn_name_base + '2b')(X) X = Activation('relu')(X) # Third component of main path (≈2 lines) X = Conv2D(filters = F3, kernel_size = (1, 1), strides= (1,1), padding = 'valid', name = conv_name_base + '2c', kernel_initializer = glorot_uniform(seed=0))(X) X = BatchNormalization(axis = 3, name = bn_name_base + '2c')(X) # Final step: Add shortcut value to main path, and pass it through a RELU activation (≈2 lines) X = Add()([X,X_shortcut]) X = Activation('relu')(X) ### END CODE HERE ### return X[/code]
2. The convolutional block(卷积块)
当两端的维度不一致时,可以加一个卷积核来转化维度,这时候没有激活函数。
First component of main path:
- The first CONV2D has F1F1 filters of shape (1,1) and a stride of (s,s). Its padding is “valid” and its name should be
conv_name_base + '2a'
. - The first BatchNorm is normalizing the channels axis. Its name should be
bn_name_base + '2a'
. - Then apply the ReLU activation function. This has no name and no hyperparameters.
Second component of main path:
- The second CONV2D has F2F2 filters of (f,f) and a stride of (1,1). Its padding is “same” and it’s name should be
conv_name_base + '2b'
. - The second BatchNorm is normalizing the channels axis. Its name should be
bn_name_base + '2b'
. - Then apply the ReLU activation function. This has no name and no hyperparameters.
Third component of main path:
- The third CONV2D has F3F3 filters of (1,1) and a stride of (1,1). Its padding is “valid” and it’s name should be
conv_name_base + '2c'
. - The third BatchNorm is normalizing the channels axis. Its name should be
bn_name_base + '2c'
. Note that there is no ReLU activation function in this component.
Shortcut path:
- The CONV2D has F3F3 filters of shape (1,1) and a stride of (s,s). Its padding is “valid” and its name should be
conv_name_base + '1'
. - The BatchNorm is normalizing the channels axis. Its name should be
bn_name_base + '1'
.
Final step:
- The shortcut and the main path values are added together.
- Then apply the ReLU activation function. This has no name and no hyperparameters.
这里参数新增了s是stride每一步数
def convolutional_block(X, f, filters, stage, block, s = 2): """ Implementation of the convolutional block as defined in Figure 4 Arguments: X -- input tensor of shape (m, n_H_prev, n_W_prev, n_C_prev) f -- integer, specifying the shape of the middle CONV's window for the main path filters -- python list of integers, defining the number of filters in the CONV layers of the main path stage -- integer, used to name the layers, depending on their position in the network block -- string/character, used to name the layers, depending on their position in the network s -- Integer, specifying the stride to be used Returns: X -- output of the convolutional block, tensor of shape (n_H, n_W, n_C) """ # defining name basis conv_name_base = 'res' + str(stage) + block + '_branch' bn_name_base = 'bn' + str(stage) + block + '_branch' # Retrieve Filters F1, F2, F3 = filters # Save the input value X_shortcut = X ##### MAIN PATH ##### # First component of main path X = Conv2D(F1, (1, 1), strides = (s,s), name = conv_name_base + '2a', kernel_initializer = glorot_uniform(seed=0))(X) X = BatchNormalization(axis = 3, name = bn_name_base + '2a')(X) X = Activation('relu')(X) ### START CODE HERE ### # Second component of main path (≈3 lines) X = Conv2D(F2, (f, f), strides = (1,1), name = conv_name_base + '2b', padding = 'same', kernel_initializer = glorot_uniform(seed=0))(X) X = BatchNormalization(axis = 3, name = bn_name_base + '2b')(X) X = Activation('relu')(X) # Third component of main path (≈2 lines) X = Conv2D(F3, (1, 1), strides = (1,1), name = conv_name_base + '2c', padding = 'valid', kernel_initializer = glorot_uniform(seed=0))(X) X = BatchNormalization(axis = 3, name = bn_name_base + '2c')(X) ##### SHORTCUT PATH #### (≈2 lines) X_shortcut = Conv2D(F3, (1, 1), strides = (s,s), name = conv_name_base + '1', padding = 'valid', kernel_initializer = glorot_uniform(seed=0))(X_shortcut) X_shortcut = BatchNormalization(axis = 3, name = bn_name_base + '1')(X_shortcut) # Final step: Add shortcut value to main path, and pass it through a RELU activation (≈2 lines) X = Add()([X,X_shortcut]) X = Activation('relu')(X) ### END CODE HERE ### return X[/code]
3 - Building your first ResNet model (50 layers)
构建一个50层的网络,分为5块,结构如下:
The details of this ResNet-50 model are:
- Zero-padding pads the input with a pad of (3,3)
- Stage 1: The 2D Convolution has 64 filters of shape (7,7) and uses a stride of (2,2). Its name is “conv1”.
- BatchNorm is applied to the channels axis of the input.
- MaxPooling uses a (3,3) window and a (2,2) stride.
-
The convolutional block uses three set of filters of size [64,64,256], “f” is 3, “s” is 1 and the block is “a”.
-
The convolutional block uses three set of filters of size [128,128,512], “f” is 3, “s” is 2 and the block is “a”.
-
The convolutional block uses three set of filters of size [256, 256, 1024], “f” is 3, “s” is 2 and the block is “a”.
-
The convolutional block uses three set of filters of size [512, 512, 2048], “f” is 3, “s” is 2 and the block is “a”.
'fc' + str(classes).
Exercise: Implement the ResNet with 50 layers described in the figure above. We have implemented Stages 1 and 2. Please implement the rest. (The syntax for implementing Stages 3-5 should be quite similar to that of Stage 2.) Make sure you follow the naming convention in the text above.
You’ll need to use this function:
- Average pooling see reference
# GRADED FUNCTION: ResNet50 def ResNet50(input_shape = (64, 64, 3), classes = 6): """ Implementation of the popular ResNet50 the following architecture: CONV2D -> BATCHNORM -> RELU -> MAXPOOL -> CONVBLOCK -> IDBLOCK*2 -> CONVBLOCK -> IDBLOCK*3 -> CONVBLOCK -> IDBLOCK*5 -> CONVBLOCK -> IDBLOCK*2 -> AVGPOOL -> TOPLAYER Arguments: input_shape -- shape of the images of the dataset classes -- integer, number of classes Returns: model -- a Model() instance in Keras """ # Define the input as a tensor with shape input_shape X_input = Input(input_shape) # Zero-Padding X = ZeroPadding2D((3, 3))(X_input) # Stage 1 X = Conv2D(64, (7, 7), strides = (2, 2), name = 'conv1', kernel_initializer = glorot_uniform(seed=0))(X) X = BatchNormalization(axis = 3, name = 'bn_conv1')(X) X = Activation('relu')(X) X = MaxPooling2D((3, 3), strides=(2, 2))(X) # Stage 2 X = convolutional_block(X, f = 3, filters = [64, 64, 256], stage = 2, block='a', s = 1) X = identity_block(X, 3, [64, 64, 256], stage=2, block='b') X = identity_block(X, 3, [64, 64, 256], stage=2, block='c') ### START CODE HERE ### # Stage 3 (≈4 lines) X = convolutional_block(X, f = 3, filters = [128, 128, 512], stage = 3, block='a', s = 2) X = identity_block(X, 3, [128, 128, 512], stage=3, block='b') X = identity_block(X, 3, [128, 128, 512], stage=3, block='c') X = identity_block(X, 3, [128, 128, 512], stage=3, block='d') # Stage 4 (≈6 lines) X = convolutional_block(X, f = 3, filters = [256, 256, 1024], stage = 4, block='a', s = 2) X = identity_block(X, 3, [256, 256, 1024], stage=4, block='b') X = identity_block(X, 3, [256, 256, 1024], stage=4, block='c') X = identity_block(X, 3, [256, 256, 1024], stage=4, block='d') X = identity_block(X, 3, [256, 256, 1024], stage=4, block='e') X = identity_block(X, 3, [256, 256, 1024], stage=4, block='f') # Stage 5 (≈3 lines) X = convolutional_block(X, f = 3, filters = [512, 512, 2048], stage = 5, block='a', s = 2) X = identity_block(X, 3, [512, 512, 2048], stage=5, block='b') X = identity_block(X, 3, [512, 512, 2048], stage=5, block='c') # AVGPOOL (≈1 line). Use "X = AveragePooling2D(...)(X)" X = AveragePooling2D(pool_size=(2,2),strides=(1,1),padding='valid')(X) ### END CODE HERE ### # output layer X = Flatten()(X) X = Dense(classes, activation='softmax', name='fc' + str(classes), kernel_initializer = glorot_uniform(seed=0))(X) # Create model model = Model(inputs = X_input, outputs = X, name='ResNet50') return model[/code] 阅读更多
- deeplearning.ai - 深度卷积网络:实例探究
- Andrew Ng's deeplearning Course4Week2 Deep convolutional models(深度卷积网络:实例探究)
- 吴恩达 deeplearning.ai课程-卷积神经网络 (2)深度卷积模型-实例探究
- 4.卷积神经网络-第二周 深度卷积网络:实例探究
- Andrew Ng 深度学习课程Deeplearning.ai 编程作业——deep Neural network for image classification(1-4.2)
- 吴恩达深度学习笔记 course4 week2 深度卷积网络 实例探究
- Coursera deeplearning.ai 深度学习笔记1-4-Deep Neural Networks-深度神经网络原理推导与代码实现
- 吴恩达Coursera深度学习课程 DeepLearning.ai 提炼笔记(5-1)-- 循环神经网络
- 吴恩达深度学习课程deeplearning.ai课程作业:Class 1 Week 2 assignment2_1
- 吴恩达深度学习课程deeplearning.ai课程作业:Class 2 Week 1 3.Gradient Checking
- DeepLearning.ai学习笔记(一)神经网络和深度学习--Week3浅层神经网络
- [DeeplearningAI笔记]神经网络与深度学习人工智能行业大师访谈
- 吴恩达深度学习课程deeplearning.ai课程作业:Class 4 Week 4 Art Generation with Neural Style Transfer
- [DeeplearningAI笔记]改善深层神经网络_深度学习的实用层面1.10_1.12/梯度消失/梯度爆炸/权重初始化
- 深度学习DeepLearning.ai系列课程学习总结:3. Python矢量化实现神经网络
- 【deeplearning.ai-神经网络和深度学习】第一周答案
- 吴恩达Coursera深度学习课程 DeepLearning.ai 提炼笔记(1-2)-- 神经网络基础
- [DeeplearningAI笔记]神经网络与深度学习2.1-2.4神经网络基础
- [DeeplearningAI笔记]神经网络与深度学习2.11_2.16神经网络基础(向量化)
- 吴恩达深度学习课程deeplearning.ai课程作业:Class 4 Week 3 Car detection