Notes on Tensorflow（四）Variables

https://www.tensorflow.org/programmers_guide/variables

Variable

变量，通常用于表示和存储模型的参数，使用接口

tf.Variable

类来定义。 它有以下几个特点：

1. 使用前必须显示初始化

2. 值可以变， 但shape一般不能变。除非

validate_shape = False

3. 从物理结构上讲， 它在内存中就是一段缓存，里面存储着一个tensor

4. 它可以被保存到硬盘上

创建Variable

创建Variable时必须提供初始值，这个初始值可以是list， numpy array或另一个tf tensor

import tensorflow as tf
import numpy as np
a = tf.Variable([1, 1, 1], name = 'a')
b = tf.Variable(np.ones((1, 3)), name = 'b')
c = tf.Variable(tf.ones(1, 3), name = 'c')

指定device

在创建变量时可以指定变量存储的设备

Operations that mutate a variable, such as tf.Variable.assign and the parameter update operations in a tf.train.Optimizer must run on the same device as the variable. Incompatible device placement directives will be ignored when creating these operations

TODO

with tf.device('/cpu:0'):
d = tf.Variable([1] * 3, name = 'd')

with tf.device('/gpu:0'):
e = tf.Variable([1] * 3, name = 'e')

初始化

创建Variable时需要指定它的初始值， 但这个操作并没有真正执行初始操作。 初始化操作由初始化操作子(initialization operator)完成.

在run graph之前必须执行初始化操作。

init_op = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init_op)
print e.eval()

如果在执行上面的代码时报

Could not satisfy explicit device specification '/device:GPU:0' because no supported kernel for GPU devices is available

， 则使用下面的代码。原因还没仔细去看， 这个问题在github上有人提到过https://github.com/tensorflow/tensorflow/issues/2292 。

init_op = tf.global_variables_initializer()
config = tf.ConfigProto(allow_soft_placement = True)
with tf.Session(config = config) as sess:
sess.run(init_op)
print e.eval()

[1 1 1]

一个Variable可以使用另一个Variable作为它的初始值

w = tf.Variable(tf.random_normal([100, 100], stddev = 1))
w2 = tf.Variable(w.initialized_value() * 2)

保存和加载

使用

tf.train.Saver

保存和加载Variables。

保存

import tensorflow as tf
weights = tf.Variable(tf.random_normal([2, 2], stddev = 1), name = 'weights')
bias = tf.Variable(tf.zeros([2, 1]), name = 'bias')
init_op = tf.global_variables_initializer()
saver = tf.train.Saver()
config = tf.ConfigProto(allow_soft_placement = True)
with tf.Session(config = config) as sess:
sess.run(init_op)
saver.save(sess, '/tmp/model.ckpt')

加载

import tensorflow as tf
weights = tf.Variable(tf.random_normal([2, 2], stddev = 1), name = 'weights')
bias = tf.Variable(tf.zeros([2, 1]), name = 'bias')

saver = tf.train.Saver()
config = tf.ConfigProto(allow_soft_placement = True)
with tf.Session(config = config) as sess:
saver.restore(sess, '/tmp/model.ckpt')
print bias.eval()

[[ 0.]
[ 0.]]

定制保存和加载

创建

Saver

时， 可以指定保存/加载哪些参数， 及以什么名字保存/从什么变量加载

import tensorflow as tf
weights = tf.Variable(tf.random_normal([2, 2], stddev = 1), name = 'weights')
bias = tf.Variable(tf.zeros([2, 1]), name = 'bias')
init_op = tf.global_variables_initializer()
saver = tf.train.Saver({'w':weights})
config = tf.ConfigProto(allow_soft_placement = True)
with tf.Session(config = config) as sess:
sess.run(init_op)
saver.save(sess, '/tmp/model.ckpt')

import tensorflow as tf
weights = tf.Variable(tf.random_normal([2, 2], stddev = 1), name = 'weights')
bias = tf.Variable(tf.zeros([2, 1]), name = 'bias')

saver = tf.train.Saver({'w': weights})
config = tf.ConfigProto(allow_soft_placement = True)
init_op = tf.global_variables_initializer()
with tf.Session(config = config) as sess:
sess.run(init_op)
saver.restore(sess, '/tmp/model.ckpt')
print bias.eval()
print weights.eval().shape

[[ 0.]
[ 0.]](2, 2)

若是所有参数都被restored， 就不用

sess.run(init_op)

了。但这里

bias

没有restore, 所以还需要

sess.run(init_op)