Stanford UFLDL教程 Exercise:Convolution and Pooling
2015-12-07 09:25
531 查看
Exercise:Convolution and Pooling
Contents[hide]1Convolution and Pooling 1.1Dependencies 1.2Step 1: Load learned features 1.3Step 2: Implement and test convolution and pooling 1.3.1Step 2a: Implement convolution 1.3.2Step 2b: Check your convolution 1.3.3Step 2c: Pooling 1.3.4Step 2d: Check your pooling 1.4Step 3: Convolve and pool with the dataset 1.5Step 4: Use pooled features for classification 1.6Step 5: Test classifier |
Convolution and Pooling
In this exercise you will use the features you learned on 8x8 patches sampled from images from the STL-10 dataset intheearlier exercise on linear decoders for classifying images from a reduced STL-10 dataset applyingconvolution
and
pooling. The reduced STL-10 dataset comprises 64x64 images from 4 classes (airplane, car, cat, dog).
In the file cnn_exercise.zip we have provided some starter code. You should write your code at the places
indicated "YOUR CODE HERE" in the files.
For this exercise, you will need to modify cnnConvolve.m andcnnPool.m.
Dependencies
The following additional files are required for this exercise:A subset of the STL10 Dataset (stlSubset.zip)
Starter Code (cnn_exercise.zip)
You will also need:
sparseAutoencoderLinear.m or your saved features from
Exercise:Learning color features with Sparse Autoencoders
feedForwardAutoencoder.m (and related functions) from
Exercise:Self-Taught Learning
softmaxTrain.m (and related functions) from
Exercise:Softmax Regression
If you have not completed the exercises listed above, we strongly suggest you complete them first.
Step 1: Load learned features
In this step, you will use the features fromExercise:Learning color features with Sparse Autoencoders. If you have completed that exercise, you can load the color features that were previously saved. To verify that the features are good, the visualized features should look like the following:
Step 2: Implement and test convolution and pooling
In this step, you will implement convolution and pooling, and test them on a small part of the data set to ensure that you have implemented these two functions correctly. In the next step, you will actually convolve and pool the features with the STL-10images.
Step 2a: Implement convolution
Implement convolution, as described infeature extraction using convolution, in the function cnnConvolve incnnConvolve.m. Implementing convolution is somewhat involved, so we will guide you through the process below.
First, we want to compute σ(Wx(r,c) +b) for all
valid (r,c) (valid meaning that the entire 8x8 patch is contained within the image; this is as opposed to afull convolution, which allows the patch to extend outside the image, with the
area outside the image assumed to be 0), whereW and
b are the learned weights and biases from the input layer to the hidden layer, andx(r,c) is the 8x8 patch with the upper left corner at(r,c).
To accomplish this, one naive method is to loop over all such patches and computeσ(Wx(r,c) +
b) for each of them; while this is fine in theory, it can very slow. Hence, we usually use Matlab's built in convolution functions, which are well optimized.
Observe that the convolution above can be broken down into the following three small steps. First, computeWx(r,c) for all(r,c). Next, add b
to all the computed values. Finally, apply the sigmoid function to the resulting values. This doesn't seem to buy you anything, since the first step still requires a loop. However, you can replace the loop in the first step with one of MATLAB's optimized convolution
functions,conv2, speeding up the process significantly.
However, there are two important points to note in using conv2.
First, conv2 performs a 2-D convolution, but you have 5 "dimensions" - image number, feature number, row of image, column of image, and (color) channel of image - that you want to convolve over. Because of this, you will have to convolve each feature
and image channel separately for each image, using the row and column of the image as the 2 dimensions you convolve over. This means that you will need three outer loops over the image numberimageNum, feature number
featureNum, and the channel number of the imagechannel. Inside the three nested for-loops, you will perform a
conv2 2-D convolution, using the weight matrix for thefeatureNum-th feature and
channel-th channel, and the image matrix for theimageNum-th image.
Second, because of the mathematical definition of convolution, the feature matrix must be "flipped" before passing it toconv2. The following implementation tip explains the "flipping" of feature matrices when using MATLAB's convolution functions:
Implementation tip: Using conv2 and convn
Because the mathematical definition of convolution involves "flipping" the matrix to convolve with (reversing its rows and its columns), to use MATLAB's convolution functions, you must first "flip" the weight matrix so that when MATLAB "flips" it according
to the mathematical definition the entries will be at the correct place. For example, suppose you wanted to convolve two matricesimage (a large image) and
W (the feature) using conv2(image, W), and W is a 3x3 matrix as below:
If you use conv2(image, W), MATLAB will first "flip" W, reversing its rows and columns, before convolvingW with
image, as below:
If the original layout of W was correct, after flipping, it would be incorrect. For the layout to be correct after flipping, you will have to flipW before passing it into
conv2, so that after MATLAB flips W in conv2, the layout will be correct. For
conv2, this means reversing the rows and columns, which can be done withflipud and
fliplr, as shown below:
% Flip W for use in conv2 W = flipud(fliplr(W));
Next, to each of the convolvedFeatures, you should then add b, the corresponding bias for thefeatureNum-th feature.
However, there is one additional complication. If we had not done any preprocessing of the input patches, you could just follow the procedure as described above, and apply the sigmoid function to obtain the convolved features, and we'd be done. However,
because you preprocessed the patches before learning features on them, you must also apply the same preprocessing steps to the convolved patches to get the correct feature activations.
In particular, you did the following to the patches:
subtract the mean patch, meanPatch to zero the mean of the patches
ZCA whiten using the whitening matrix ZCAWhite.
These same three steps must also be applied to the input image patches.
Taking the preprocessing steps into account, the feature activations that you should compute is
, whereT
is the whitening matrix and
is the mean patch. Expanding this, you obtain
,
which suggests that you should convolve the images withWT rather than
W as earlier, and you should add
, rather than justb
to convolvedFeatures, before finally applying the sigmoid function.
Step 2b: Check your convolution
We have provided some code for you to check that you have done the convolution correctly. The code randomly checks the convolved values for a number of (feature, row, column) tuples by computing the feature activations usingfeedForwardAutoencoderfor the selected features and patches directly using the sparse autoencoder.
Step 2c: Pooling
Implementpooling in the function cnnPool in cnnPool.m. You should implementmean pooling (i.e., averaging over feature responses) for this part.
Step 2d: Check your pooling
We have provided some code for you to check that you have done the pooling correctly. The code runscnnPool against a test matrix to see if it produces the expected result.Step 3: Convolve and pool with the dataset
In this step, you will convolve each of the features you learned with the full 64x64 images from the STL-10 dataset to obtain the convolved features for both the training and test sets. You will then pool the convolved features to obtain the pooled featuresfor both training and test sets. The pooled features for the training set will be used to train your classifier, which you can then test on the test set.
Because the convolved features matrix is very large, the code provided does the convolution and pooling 50 features at a time to avoid running out of memory.
Step 4: Use pooled features for classification
In this step, you will use the pooled features to train a softmax classifier to map the pooled features to the class labels. The code in this section usessoftmaxTrain from the softmax exercise to train a softmax classifier on the pooled featuresfor 500 iterations, which should take around a few minutes.
Step 5: Test classifier
Now that you have a trained softmax classifier, you can see how well it performs on the test set. These pooled features for the test set will be run through the softmax classifier, and the accuracy of the predictions will be computed. You should expect toget an accuracy of around 80%.
from: http://ufldl.stanford.edu/wiki/index.php/Exercise:Convolution_and_Pooling
相关文章推荐
- 斯坦福cs193p作业解析之计算器1(Calculator)
- 斯坦福cs193p作业解析之计算器2(Calculator)
- 斯坦福cs193p作业解析之图形计算器(Graph Calculator)
- 斯坦福cs193p作业解析之Smashtag(Twitter)
- Pooling
- Stanford机器学习---第一讲. Linear Regression with one variable
- Stanford机器学习---第二讲. 多变量线性回归 Linear Regression with multiple variable
- Stanford机器学习---第四讲. 神经网络的表示 Neural Networks representation
- Stanford机器学习---第五讲. 神经网络的学习 Neural Networks learning
- Stanford机器学习---第六讲. 怎样选择机器学习方法、系统
- Stanford机器学习---第七讲. 机器学习系统设计
- Stanford机器学习---第八讲. 支持向量机SVM
- Stanford机器学习---第九讲. 聚类
- Stanford机器学习---第十讲. 数据降维
- corenlp-python的README.md翻译
- Stanford Parser 使用方法
- 信息检索:对搜索引擎性能的评价指标的小作业---pooling方法以及MAP value的计算
- cs229 斯坦福机器学习笔记(一)-- 入门与LR模型
- pooling
- Stanford UFLDL教程 池化Pooling