Convolutional neural networks(CNN) (十一) Convolution and Pooling Exercise

{作为CNN学习入门的一部分，笔者在这里逐步给出UFLDL的各章节Exercise的个人代码实现，供大家参考指正}

理论部分可以在线参阅(页面最下方有中文选项)Feature extraction
using convolution 以及
Pooling 部分内容

Note:

UFLDL中对Conv的计算方法与CS231n中的表述稍有不同，这里是对图像RGB三通道分别使用不同的Kernel(Kernel_row
x Kernel_col)进行卷积，然后再进行三通道的加和。

而CS231n中对三通道相同位置(Kernel_row
x Kernel_col)的一个 volume
(Channels x Kernel_row x Kernel_col)展成一维向量，

与某一个卷积核(Channels x Kernel_row x Kernel_col)展成一维向量做dot操作，得到该卷积核在FeatureMap对应位置上的卷积特征。

{ 每一个卷积核都对应于输出FeatureMap的某一层
: size(FeatureMap) = Kernel_num x Channels x Kernel_row x Kernel_col }

cnnExercise.m

%% CS294A/CS294W Convolutional Neural Networks Exercise

% Instructions
% ------------
%
% This file contains code that helps you get started on the
% convolutional neural networks exercise. In this exercise, you will only
% need to modify cnnConvolve.m and cnnPool.m. You will not need to modify
% this file.

%%======================================================================
%% STEP 0: Initialization
% Here we initialize some parameters used for the exercise.

imageDim = 64; % image dimension
imageChannels = 3; % number of channels (rgb, so 3)

patchDim = 8; % patch dimension
numPatches = 50000; % number of patches

visibleSize = patchDim * patchDim * imageChannels; % number of input units
outputSize = visibleSize; % number of output units
hiddenSize = 400; % number of hidden units

epsilon = 0.1; % epsilon for ZCA whitening

poolDim = 19; % dimension of pooling region

%%======================================================================
%% STEP 1: Train a sparse autoencoder (with a linear decoder) to learn
% features from color patches. If you have completed the linear decoder
% execise, use the features that you have obtained from that exercise,
% loading them into optTheta. Recall that we have to keep around the
% parameters used in whitening (i.e., the ZCA whitening matrix and the
% meanPatch)

% --------------------------- YOUR CODE HERE --------------------------
% Train the sparse autoencoder and fill the following variables with
% the optimal parameters:

% optTheta = zeros(2*hiddenSize*visibleSize+hiddenSize+visibleSize, 1);
% ZCAWhite = zeros(visibleSize, visibleSize);
% meanPatch = zeros(visibleSize, 1);

% load STL10Features.mat;

% --------------------------------------------------------------------

% Display and check to see that the features look good
% W = reshape(optTheta(1:visibleSize * hiddenSize), hiddenSize, visibleSize);
% b = optTheta(2*hiddenSize*visibleSize+1:2*hiddenSize*visibleSize+hiddenSize);

% displayColorNetwork( (W*ZCAWhite)');

%%======================================================================
%% STEP 2: Implement and test convolution and pooling
% In this step, you will implement convolution and pooling, and test them
% on a small part of the data set to ensure that you have implemented
% these two functions correctly. In the next step, you will actually
% convolve and pool the features with the STL10 images.

%% STEP 2a: Implement convolution
% Implement convolution in the function cnnConvolve in cnnConvolve.m

% Note that we have to preprocess the images in the exact same way
% we preprocessed the patches before we can obtain the feature activations.

% load stlTrainSubset.mat % loads numTrainImages, trainImages, trainLabels

%% Use only the first 8 images for testing
% convImages = trainImages(:, :, :, 1:8);

% NOTE: Implement cnnConvolve in cnnConvolve.m first!
% convolvedFeatures = cnnConvolve(patchDim, hiddenSize, convImages, W, b, ZCAWhite, meanPatch);

%% STEP 2b: Checking your convolution
% To ensure that you have convolved the features correctly, we have
% provided some code to compare the results of your convolution with
% activations from the sparse autoencoder

% For 1000 random points
% for i = 1:1000
% featureNum = randi([1, hiddenSize]);
% imageNum = randi([1, 8]);
% imageRow = randi([1, imageDim - patchDim + 1]);
% imageCol = randi([1, imageDim - patchDim + 1]);
%
% patch = convImages(imageRow:imageRow + patchDim - 1, imageCol:imageCol + patchDim - 1, :, imageNum);
% patch = patch(:);
% patch = patch - meanPatch;
% pa
fffc
tch = ZCAWhite * patch;
%
% features = feedForwardAutoencoder(optTheta, hiddenSize, visibleSize, patch);
%
% if abs(features(featureNum, 1) - convolvedFeatures(featureNum, imageNum, imageRow, imageCol)) > 1e-9
% fprintf('Convolved feature does not match activation from autoencoder\n');
% fprintf('Feature Number : %d\n', featureNum);
% fprintf('Image Number : %d\n', imageNum);
% fprintf('Image Row : %d\n', imageRow);
% fprintf('Image Column : %d\n', imageCol);
% fprintf('Convolved feature : %0.5f\n', convolvedFeatures(featureNum, imageNum, imageRow, imageCol));
% fprintf('Sparse AE feature : %0.5f\n', features(featureNum, 1));
% error('Convolved feature does not match activation from autoencoder');
% end
% end
%
% disp('Congratulations! Your convolution code passed the test.');

%% STEP 2c: Implement pooling
% Implement pooling in the function cnnPool in cnnPool.m

% NOTE: Implement cnnPool in cnnPool.m first!
% pooledFeatures = cnnPool(poolDim, convolvedFeatures);

%% STEP 2d: Checking your pooling
% To ensure that you have implemented pooling, we will use your pooling
% function to pool over a test matrix and check the results.

% testMatrix = reshape(1:64, 8, 8);
% expectedMatrix = [mean(mean(testMatrix(1:4, 1:4))) mean(mean(testMatrix(1:4, 5:8))); ...
% mean(mean(testMatrix(5:8, 1:4))) mean(mean(testMatrix(5:8, 5:8))); ];
%
% testMatrix = reshape(testMatrix, 1, 1, 8, 8);
%
% pooledFeatures = squeeze(cnnPool(4, testMatrix));
%
% if ~isequal(pooledFeatures, expectedMatrix)
% disp('Pooling incorrect');
% disp('Expected');
% disp(expectedMatrix);
% disp('Got');
% disp(pooledFeatures);
% else
% disp('Congratulations! Your pooling code passed the test.');
% end

%%======================================================================
%% STEP 3: Convolve and pool with the dataset
% In this step, you will convolve each of the features you learned with
% the full large images to obtain the convolved features. You will then
% pool the convolved features to obtain the pooled features for
% classification.
%
% Because the convolved features matrix is very large, we will do the
% convolution and pooling 50 features at a time to avoid running out of
% memory. Reduce this number if necessary

stepSize = 50;
assert(mod(hiddenSize, stepSize) == 0, 'stepSize should divide hiddenSize');

load stlTrainSubset.mat % loads numTrainImages, trainImages, trainLabels
load stlTestSubset.mat % loads numTestImages, testImages, testLabels
%{
pooledFeaturesTrain = zeros(hiddenSize, numTrainImages, ...
floor((imageDim - patchDim + 1) / poolDim), ...
floor((imageDim - patchDim + 1) / poolDim) );
pooledFeaturesTest = zeros(hiddenSize, numTestImages, ...
floor((imageDim - patchDim + 1) / poolDim), ...
floor((imageDim - patchDim + 1) / poolDim) );

tic();

for convPart = 1:(hiddenSize / stepSize)

featureStart = (convPart - 1) * stepSize + 1;
featureEnd = convPart * stepSize;

fprintf('Step %d: features %d to %d\n', convPart, featureStart, featureEnd);
Wt = W(featureStart:featureEnd, :);
bt = b(featureStart:featureEnd);

fprintf('Convolving and pooling train images\n');
convolvedFeaturesThis = cnnConvolve(patchDim, stepSize, ...
trainImages, Wt, bt, ZCAWhite, meanPatch);
pooledFeaturesThis = cnnPool(poolDim, convolvedFeaturesThis);
pooledFeaturesTrain(featureStart:featureEnd, :, :, :) = pooledFeaturesThis;
toc();
clear convolvedFeaturesThis pooledFeaturesThis;

fprintf('Convolving and pooling test images\n');
convolvedFeaturesThis = cnnConvolve(patchDim, stepSize, ...
testImages, Wt, bt, ZCAWhite, meanPatch);
pooledFeaturesThis = cnnPool(poolDim, convolvedFeaturesThis);
pooledFeaturesTest(featureStart:featureEnd, :, :, :) = pooledFeaturesThis;
toc();

clear convolvedFeaturesThis pooledFeaturesThis;

end

% You might want to save the pooled features since convolution and pooling takes a long time
save('cnnPooledFeatures.mat', 'pooledFeaturesTrain', 'pooledFeaturesTest');
toc();
%}
load cnnPooledFeatures.mat;
%%======================================================================
%% STEP 4: Use pooled features for classification
% Now, you will use your pooled features to train a softmax classifier,
% using softmaxTrain from the softmax exercise.
% Training the softmax classifer for 1000 iterations should take less than
% 10 minutes.

% Add the path to your softmax solution, if necessary
% addpath /path/to/solution/

% Setup parameters for softmax
softmaxLambda = 1e-4;
numClasses = 4;
% Reshape the pooledFeatures to form an input vector for softmax
softmaxX = permute(pooledFeaturesTrain, [1 3 4 2]);
softmaxX = reshape(softmaxX, numel(pooledFeaturesTrain) / numTrainImages,...
numTrainImages);
softmaxY = trainLabels;

options = struct;
options.maxIter = 1000;
softmaxModel = softmaxTrain(numel(pooledFeaturesTrain) / numTrainImages,...
numClasses, softmaxLambda, softmaxX, softmaxY, options);

%%======================================================================
%% STEP 5: Test classifer
% Now you will test your trained classifer against the test images

softmaxX = permute(pooledFeaturesTest, [1 3 4 2]);
softmaxX = reshape(softmaxX, numel(pooledFeaturesTest) / numTestImages, numTestImages);
softmaxY = testLabels;

[pred] = softmaxPredict(softmaxModel, softmaxX);
acc = (pred(:) == softmaxY(:));
acc = sum(acc) / size(acc, 1);
fprintf('Accuracy: %2.3f%%\n', acc * 100);

% You should expect to get an accuracy of around 80% on the test images.
cnnConvolve.m

function convolvedFeatures = cnnConvolve(patchDim, numFeatures, images, W, b, ZCAWhite, meanPatch)
% cnnConvolve Returns the convolution of the features given by W and b with
% the given images
%
% Parameters:
% patchDim - patch (feature) dimension
% numFeatures - number of features
% images - large images to convolve with, matrix in the form
% images(r, c, channel, image number)
% W, b - W, b for features from the sparse autoencoder
% ZCAWhite, meanPatch - ZCAWhitening and meanPatch matrices used for
% preprocessing
%
% Returns:
% convolvedFeatures - matrix of convolved features in the form
% convolvedFeatures(featureNum, imageNum, imageRow, imageCol)

numImages = size(images, 4);
imageDim = size(images, 1);
imageChannels = size(images, 3);

% convolvedFeatures = zeros(numFeatures, numImages, imageDim - patchDim + 1, imageDim - patchDim + 1);

% Instructions:
% Convolve every feature with every large image here to produce the
% numFeatures x numImages x (imageDim - patchDim + 1) x (imageDim - patchDim + 1)
% matrix convolvedFeatures, such that
% convolvedFeatures(featureNum, imageNum, imageRow, imageCol) is the
% value of the convolved featureNum feature for the imageNum image over
% the region (imageRow, imageCol) to (imageRow + patchDim - 1, imageCol + patchDim - 1)
%
% Expected running times:
% Convolving with 100 images should take less than 3 minutes
% Convolving with 5000 images should take around an hour
% (So to save time when testing, you should convolve with less images, as
% described earlier)

% -------------------- YOUR CODE HERE --------------------
% Precompute the matrices that will be used during the convolution. Recall
% that you need to take into account the whitening and mean subtraction steps

WT = W * ZCAWhite;
mean_b = b - WT * meanPatch;

% --------------------------------------------------------
patchSize = patchDim * patchDim;
convolvedFeatures = zeros(numFeatures, numImages, imageDim - patchDim + 1, imageDim - patchDim + 1);
for imageNum = 1:numImages
for featureNum = 1:numFeatures

% convolution of image with feature matrix for each channel
convolvedImage = zeros(imageDim - patchDim + 1, imageDim - patchDim + 1);
for channel = 1:imageChannels

% Obtain the feature (patchDim x patchDim) needed during the convolution
% ---- YOUR CODE HERE ----
% feature = zeros(8,8); % You should replace this
feature = reshape(WT(featureNum,(channel-1)*patchSize+1:channel*patchSize), patchDim, patchDim);

% ------------------------

% Flip the feature matrix because of the definition of convolution, as explained later
feature = rot90(squeeze(feature),2);

% Obtain the image
im = squeeze(images(:, :, channel, imageNum));

% Convolve "feature" with "im", adding the result to convolvedImage
% be sure to do a 'valid' convolution
% ---- YOUR CODE HERE ----

convolvedImage = convolvedImage + conv2(im, feature, 'valid');

% ------------------------

end

% Subtract the bias unit (correcting for the mean subtraction as well)
% Then, apply the sigmoid function to get the hidden activation
% ---- YOUR CODE HERE ----
convolvedImage = sigmoid(convolvedImage + mean_b(featureNum));

% ------------------------

% The convolved feature is the sum of the convolved values for all channels
convolvedFeatures(featureNum, imageNum, :, :) = convolvedImage;
end
end

end

function sigm = sigmoid(x)

sigm = 1 ./ (1 + exp(-x));
end

cnnPool.m

function pooledFeatures = cnnPool(poolDim, convolvedFeatures)
%cnnPool Pools the given convolved features
%
% Parameters:
% poolDim - dimension of pooling region
% convolvedFeatures - convolved features to pool (as given by cnnConvolve)
% convolvedFeatures(featureNum, imageNum, imageRow, imageCol)
%
% Returns:
% pooledFeatures - matrix of pooled features in the form
% pooledFeatures(featureNum, imageNum, poolRow, poolCol)
%

numImages = size(convolvedFeatures, 2);
numFeatures = size(convolvedFeatures, 1);
convolvedDim = size(convolvedFeatures, 3);

pooledFeatures = zeros(numFeatures, numImages, floor(convolvedDim / poolDim), floor(convolvedDim / poolDim));

% -------------------- YOUR CODE HERE --------------------
% Instructions:
% Now pool the convolved features in regions of poolDim x poolDim,
% to obtain the
% numFeatures x numImages x (convolvedDim/poolDim) x (convolvedDim/poolDim)
% matrix pooledFeatures, such that
% pooledFeatures(featureNum, imageNum, poolRow, poolCol) is the
% value of the featureNum feature for the imageNum image pooled over the
% corresponding (poolRow, poolCol) pooling region
% (see http://ufldl/wiki/index.php/Pooling )
%
% Use mean pooling here.
% -------------------- YOUR CODE HERE --------------------

for iterFeature = 1:numFeatures
for iterImage = 1:numImages
for iterDim_col = 1:floor(convolvedDim / poolDim)
for iterDim_row = 1:floor(convolvedDim / poolDim)
pooledFeatures(iterFeature, iterImage, iterDim_col, iterDim_row) = ...
mean(mean(convolvedFeatures(iterFeature, iterImage, ...
1+(iterDim_col-1)*poolDim:iterDim_col*poolDim, ...
1+(iterDim_row-1)*poolDim:iterDim_row*poolDim)));
end
end
end
end
end

实验结果：

Step 8: features 351 to 400
Convolving and pooling train images
时间已过 2249.400697 秒。
Convolving and pooling test images
时间已过 2436.324619 秒。

特征训练时长：
2436.324619 / 60 = 40.6054 mins 

Softmax训练结果：

93         96     1.00000e+00     4.66655e-01     1.82286e-04

Function Value changing by less than TolX

Accuracy: 80.469%