neural-networks-and-deep-learning mnist-loader
2016-07-14 23:08
337 查看
最近发现笔记本里还有这么个文件夹中的代码没看,感觉需要看一看的。
这是写的第二个代码分析了。第一个是gibbsLDA++,感觉还挺有意思的。
那就来分析看看吧,先从最简单的开始吧。
cPickle和gzip是很方便的压缩存储的工具。cPickle是进行序列化的工具,而gzip是压缩的工具。
其实最原始的load_data就是将序列化的东西摘取出来。
其实这个函数load_data_wrapper主要的工作其实是,对于train_data的数据进行zip操作,而且将标签进行向量化。
这是写的第二个代码分析了。第一个是gibbsLDA++,感觉还挺有意思的。
那就来分析看看吧,先从最简单的开始吧。
cPickle和gzip是很方便的压缩存储的工具。cPickle是进行序列化的工具,而gzip是压缩的工具。
def load_data(): """Return the MNIST data as a tuple containing the training data, the validation data, and the test data. The ``training_data`` is returned as a tuple with two entries. The first entry contains the actual training images. This is a numpy ndarray with 50,000 entries. Each entry is, in turn, a numpy ndarray with 784 values, representing the 28 * 28 = 784 pixels in a single MNIST image. The second entry in the ``training_data`` tuple is a numpy ndarray containing 50,000 entries. Those entries are just the digit values (0...9) for the corresponding images contained in the first entry of the tuple. The ``validation_data`` and ``test_data`` are similar, except each contains only 10,000 images. This is a nice data format, but for use in neural networks it's helpful to modify the format of the ``training_data`` a little. That's done in the wrapper function ``load_data_wrapper()``, see below. """ f = gzip.open('../data/mnist.pkl.gz', 'rb') training_data, validation_data, test_data = cPickle.load(f) f.close() return (training_data, validation_data, test_data)
其实最原始的load_data就是将序列化的东西摘取出来。
def load_data_wrapper(): """Return a tuple containing ``(training_data, validation_data, test_data)``. Based on ``load_data``, but the format is more convenient for use in our implementation of neural networks. In particular, ``training_data`` is a list containing 50,000 2-tuples ``(x, y)``. ``x`` is a 784-dimensional numpy.ndarray containing the input image. ``y`` is a 10-dimensional numpy.ndarray representing the unit vector corresponding to the correct digit for ``x``. ``validation_data`` and ``test_data`` are lists containing 10,000 2-tuples ``(x, y)``. In each case, ``x`` is a 784-dimensional numpy.ndarry containing the input image, and ``y`` is the corresponding classification, i.e., the digit values (integers) corresponding to ``x``. Obviously, this means we're using slightly different formats for the training data and the validation / test data. These formats turn out to be the most convenient for use in our neural network code.""" tr_d, va_d, te_d = load_data() training_inputs = [np.reshape(x, (784, 1)) for x in tr_d[0]] training_results = [vectorized_result(y) for y in tr_d[1]] training_data = zip(training_inputs, training_results) validation_inputs = [np.reshape(x, (784, 1)) for x in va_d[0]] validation_data = zip(validation_inputs, va_d[1]) test_inputs = [np.reshape(x, (784, 1)) for x in te_d[0]] test_data = zip(test_inputs, te_d[1]) return (training_data, validation_data, test_data)
其实这个函数load_data_wrapper主要的工作其实是,对于train_data的数据进行zip操作,而且将标签进行向量化。
def vectorized_result(j): """Return a 10-dimensional unit vector with a 1.0 in the jth position and zeroes elsewhere. This is used to convert a digit (0...9) into a corresponding desired output from the neural network.""" e = np.zeros((10, 1)) e[j] = 1.0 return e
相关文章推荐
- NN学习笔记
- exact nn search in hamming space
- 同或门(XNOR)电路的网络学习
- the most right digit of N^N.
- zoj 2277 (数论)
- 基于Tensorflow的机器学习(3) -- KMeans and NN(Nearest Neighbors)
- Hadoop远程调试
- 人工神经网络
- [EMNLP2017]Global Normalization of Convolutional Neural Networks for Joint Entity and Relation
- 读书笔记:Deep Learning [Ada-Computation&ML series]--chapter8.Optimization
- neural-networks-and-deep-learning mnist_average_darkness.py
- neural-networks-and-deep-learning expand_mnist.py
- Tensorflow笔记(四)——Nearest Neighbor Example
- 神经网络浅讲:从神经元到深度学习
- DL学习笔记【20】nn包中的各位Simple layers
- DL学习笔记【17】nn包中的各位Convolutional layers
- 【Scikit-Learn 中文文档】最近邻 - 监督学习 - 用户指南 | ApacheCN
- from keras.layers import Input 介绍
- [深度学习系列之一] C++ 实现神经网络
- Torch在使用中常见的问题hdf5安装,nn增加类