Notes on tensorflow(八)read tfrecords with slim
2017-04-30 16:21
639 查看
http://blog.csdn.net/weixin_35653315/article/details/71015845 中描述了将pascal voc数据集转换tfrecord的过程。本文则通过slim来读取生成的tfrecord。读取和解码操作由
到目前为止,返回的image是一个tensor,而且是一个3维的,一次只有一张。需要组成一个batch。组成batch之前图片要预处理,一是为了将图片size变成固定大小, 二是数据增长。上面的示例代码来自https://github.com/balancap/SSD-Tensorflow/blob/master/datasets/pascalvoc_common.py#L49
。下面的示例代码来自 https://github.com/balancap/SSD-Tensorflow/blob/master/train_ssd_network.py#L203
tf.TFRecordReader完成
import tensorflow as tf slim = tf.contrib.slim file_pattern = './pascal_train_*.tfrecord' #文件名格式 # 适配器1:将example反序列化成存储之前的格式。由tf完成 keys_to_features = { 'image/encoded': tf.FixedLenFeature((), tf.string, default_value=''), 'image/format': tf.FixedLenFeature((), tf.string, default_value='jpeg'), 'image/height': tf.FixedLenFeature([1], tf.int64), 'image/width': tf.FixedLenFeature([1], tf.int64), 'image/channels': tf.FixedLenFeature([1], tf.int64), 'image/shape': tf.FixedLenFeature([3], tf.int64), 'image/object/bbox/xmin': tf.VarLenFeature(dtype=tf.float32), 'image/object/bbox/ymin': tf.VarLenFeature(dtype=tf.float32), 'image/object/bbox/xmax': tf.VarLenFeature(dtype=tf.float32), 'image/object/bbox/ymax': tf.VarLenFeature(dtype=tf.float32), 'image/object/bbox/label': tf.VarLenFeature(dtype=tf.int64), 'image/object/bbox/difficult': tf.VarLenFeature(dtype=tf.int64), 'image/object/bbox/truncated': tf.VarLenFeature(dtype=tf.int64), } #适配器2:将反序列化的数据组装成更高级的格式。由slim完成 items_to_handlers = { 'image': slim.tfexample_decoder.Image('image/encoded', 'image/format'), 'shape': slim.tfexample_decoder.Tensor('image/shape'), 'object/bbox': slim.tfexample_decoder.BoundingBox( ['ymin', 'xmin', 'ymax', 'xmax'], 'image/object/bbox/'), 'object/label': slim.tfexample_decoder.Tensor('image/object/bbox/label'), 'object/difficult': slim.tfexample_decoder.Tensor('image/object/bbox/difficult'), 'object/truncated': slim.tfexample_decoder.Tensor('image/object/bbox/truncated'), } # 解码器 decoder = slim.tfexample_decoder.TFExampleDecoder(keys_to_features, items_to_handlers) # dataset对象定义了数据集的文件位置,解码方式等元信息 dataset = slim.dataset.Dataset( data_sources=file_pattern, reader=tf.TFRecordReader, num_samples = 3, # 手动生成了三个文件, 每个文件里只包含一个example decoder=decoder, items_to_descriptions = {}, num_classes=21) #provider对象根据dataset信息读取数据 provider = slim.dataset_data_provider.DatasetDataProvider( dataset, num_readers=3, shuffle=False) [image, shape, glabels, gbboxes] = provider.get(['image', 'shape', 'object/label', 'object/bbox']) print type(image) print image.shape
<class 'tensorflow.python.framework.ops.Tensor'> (?, ?, 3)
到目前为止,返回的image是一个tensor,而且是一个3维的,一次只有一张。需要组成一个batch。组成batch之前图片要预处理,一是为了将图片size变成固定大小, 二是数据增长。上面的示例代码来自https://github.com/balancap/SSD-Tensorflow/blob/master/datasets/pascalvoc_common.py#L49
。下面的示例代码来自 https://github.com/balancap/SSD-Tensorflow/blob/master/train_ssd_network.py#L203
# Pre-processing image, labels and bboxes. image, glabels, gbboxes = \ image_preprocessing_fn(image, glabels, gbboxes, out_shape=ssd_shape, data_format=DATA_FORMAT) # Encode groundtruth labels and bboxes. gclasses, glocalisations, gscores = \ ssd_net.bboxes_encode(glabels, gbboxes, ssd_anchors) batch_shape = [1] + [len(ssd_anchors)] * 3 # Training batches and queue. r = tf.train.batch( tf_utils.reshape_list([image, gclasses, glocalisations, gscores]), batch_size=FLAGS.batch_size, num_threads=FLAGS.num_preprocessing_threads, capacity=5 * FLAGS.batch_size) b_image, b_gclasses, b_glocalisations, b_gscores = \ tf_utils.reshape_list(r, batch_shape) # Intermediate queueing: unique batch computation pipeline for all # GPUs running the training. batch_queue = slim.prefetch_queue.prefetch_queue( tf_utils.reshape_list([b_image, b_gclasses, b_glocalisations, b_gscores]), capacity=2 * deploy_config.num_clones)
相关文章推荐
- the basic approach to read dataset(TFRecord) with iterator in Tensorflow
- Getting started with TensorFlow on iOS
- Softmax on Digits Data with TensorFlow
- tensorflow slim(TF-Slim)介绍
- Setup Tensorflow with GPU on Mac OSX 10.11
- chapter2 of OReilly.Hands-On.Machine.Learning.with.Scikit-Learn.and.TensorFlow
- 【Tensorflow】辅助工具篇——tensorflow slim(TF-Slim)介绍
- Notes on Tensorflow
- Notes on tensorflow(二)Get started
- Notes on tensorflow(七)将数据集转换为TFRecord
- 学习笔记TF065:TensorFlowOnSpark
- Notes on Tensorflow(四)Variables
- Tensorflow TFRecords及多线程训练详解
- 【Tensorflow】辅助工具篇——tensorflow slim(TF-Slim)介绍
- tensorflowxun训练自己的数据集之从tfrecords读取数据
- TensorFlow Machine Learning with Financial Data on Google Cloud Platform
- OReilly.Hands-On.Machine.Learning.with.Scikit-Learn.and.TensorFlow.翻译以及读书心得--p41-53
- tensorflow slim【TF-Slim】
- 170620 How to install tensorflow with GPU on linux
- Tensorflow辅助工具篇——tensorflow slim(TF-Slim)介绍