I'm trying to build an object detector with CNN using tensorflow with python framework. I would like to train my model to do just object recognition (classification) at first and then using several convolutional layers of the pretarined model train it to predict bounding boxes. I will need to replace fully connected layers and probably some last convolutional layers. So, for this reason, I would like to know if it is possible to import only weights from tensorflow graph that was used to train object classifier to a newly defined graph that I will train to do object detection. So basically I would like to do something like this:
# here I initialize the new graph
conv_1=tf.nn.conv2d(in, weights_from_old_graph)
conv_2=tf.nn.conv2d(conv_1, weights_from_old_graph)
...
conv_n=tf.nn.nnconv2d(conv_n-1,randomly_initialized_weights)
fc_1=tf.matmul(conv_n, randomly_initalized_weights)
Use saver with no arguments to save the entire model.
tf.reset_default_graph()
v1 = tf.get_variable("v1", [3], initializer = tf.initializers.random_normal)
v2 = tf.get_variable("v2", [5], initializer = tf.initializers.random_normal)
saver = tf.train.Saver()
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
saver.save(sess, save_path='./test-case.ckpt')
print(v1.eval())
print(v2.eval())
saver = None
v1 = [ 2.1882825 1.159807 -0.26564872]
v2 = [0.11437789 0.5742971 ]
Then in the model you want to restore to certain values, pass a list of variable names you want to restore or a dictionary of {"variable name": variable} to the Saver.
tf.reset_default_graph()
b1 = tf.get_variable("b1", [3], initializer= tf.initializers.random_normal)
b2 = tf.get_variable("b2", [3], initializer= tf.initializers.random_normal)
saver = tf.train.Saver(var_list={'v1': b1})
with tf.Session() as sess:
saver.restore(sess, "./test-case.ckpt")
print(b1.eval())
print(b2.eval())
INFO:tensorflow:Restoring parameters from ./test-case.ckpt
b1 = [ 2.1882825 1.159807 -0.26564872]
b2 = FailedPreconditionError: Attempting to use uninitialized value b2
Although I agree with Aechlys to restore variables. The problem is harder when we want to fix these variables. For example, we trained these variables and we want to use them in another model, but this time without training them (training new variables like in transfer-learning). You can see the answer I posted here.
Quick example:
with tf.session() as sess:
new_saver = tf.train.import_meta_graph(pathToMeta)
new_saver.restore(sess, pathToNonMeta)
weight1 = sess.run(sess.graph.get_tensor_by_name("w1:0"))
tf.reset_default_graph() #this will eliminate the variables we restored
with tf.session() as sess:
weights =
{
'1': tf.Variable(weight1 , name='w1-bis', trainable=False)
}
...
We are now sure the restored variables are not a part of the graph.
Related
Problem:
I am very new to Tensorflow. My specific question is what particular arguments should I put inside sess.run(fetches, feed_dict) function. For instance, how could find out what the values of the arguments?
Steps:
Here is my understanding of the steps after looking at other posts.
Save tranied tensorflow model, it should consists of 4 files, below are my outputs:
checkpoint
Inception_resnet_v2.ckpt.data-00000-of-00001
Inception_resnet_v2.ckpt.index
Inception_resnet_v2.ckpt.meta
Resize the input image to whatever format required by the neural network.
Start tensorflow session.
Retrive the Graph and associated parameters, tensors...
Predict the input image.
Code：
Traning code:
https://github.com/taki0112/SENet-Tensorflow/blob/master/SE_Inception_resnet_v2.py
[Solved] Test code:
import tensorflow as tf
import numpy as np
import cv2
labels = ["airplane","automobile","bird","cat","deer","dog","frog","horse","ship","truck"]
# Load graph and parameters, etc.
sess=tf.Session()
saver = tf.train.import_meta_graph('./model/Inception_resnet_v2.ckpt.meta')
saver.restore(sess, tf.train.latest_checkpoint("./model/"))
graph = tf.get_default_graph()
# Get tensor names
x = graph.get_tensor_by_name("Placeholder:0")
training_flag = graph.get_tensor_by_name("Placeholder_2:0")
op_to_restore = graph.get_tensor_by_name("final_fully_connected/dense/BiasAdd:0")
# Preprocess imgae imput
src = cv2.imread("./input/car3.jpg")
dst = cv2.resize(src, (32, 32), interpolation=cv2.INTER_CUBIC)
b,g,r = cv2.split(dst)
b = (b - np.mean(b)) / np.std(b) * .1
g = (g - np.mean(g)) / np.std(g) * .1
r = (r - np.mean(r)) / np.std(r) * .1
src = cv2.merge((b,g,r))
picture = dst.reshape(1, 32, 32, 3)
feed_dict ={x: picture, training_flag:False}
result_index = sess.run(op_to_restore,feed_dict)
print(result_index)
print (labels[np.argmax(result_index)])
the arguments actually depend on what you're doing, but mostly the first argument is the weights and placeholders. Whenever you are working with Tensorflow, you define a graph which is fed examples(training data) and some hyperparameters like learning rate, global step etc. It’s a standard practice to feed all the training data and hyperparameters using placeholders. when you build a network using placeholders and save it the network is saved, however, values of the placeholders are not saved.
Let's see a toy example:
import tensorflow as tf
#Prepare to feed input, i.e. feed_dict and placeholders
w1 = tf.placeholder("float", name="w1")
w2 = tf.placeholder("float", name="w2")
b1= tf.Variable(2.0,name="bias")
feed_dict ={w1:4,w2:8}
#Define a test operation that we will restore
w3 = tf.add(w1,w2)
w4 = tf.multiply(w3,b1,name="op_to_restore")
sess = tf.Session()
sess.run(tf.global_variables_initializer())
#Create a saver object which will save all the variables
saver = tf.train.Saver()
#Run the operation by feeding input
print sess.run(w4,feed_dict)
#Prints 24 which is sum of (w1+w2)*b1
#Now, save the graph
saver.save(sess, 'my_test_model',global_step=1000)
Now, when we want to restore it, we not only have to restore the graph and weights, but also prepare a new feed_dict that will feed the new training data to the network. We can get reference to these saved operations and placeholder variables via graph.get_tensor_by_name() method. So if you want to train the same model with further new data, then you would have to utilize those weigtages, if however you just want to get the prediction from the model you trained, you could utilize the op_to_restore and the feed_dict as new data. Something like this, if you follow the above example:
import tensorflow as tf
sess=tf.Session()
#First let's load meta graph and restore weights
saver = tf.train.import_meta_graph('my_test_model-1000.meta')
saver.restore(sess,tf.train.latest_checkpoint('./'))
# Now, let's access and create placeholders variables and
# create feed-dict to feed new data
graph = tf.get_default_graph()
w1 = graph.get_tensor_by_name("w1:0")
w2 = graph.get_tensor_by_name("w2:0")
feed_dict ={w1:13.0,w2:17.0}
#Now, access the op that you want to run.
op_to_restore = graph.get_tensor_by_name("op_to_restore:0")
print sess.run(op_to_restore,feed_dict)
#This will print 60 which is calculated
#using new values of w1 and w2 and saved value of b1.
So, this is how it works, in your case, since you're trying to load the Inception model, your op_to_restore should depend on what you're trying to restore if you could tell us what you're trying to do, then only it's possible to suggest something. However in the other parameter feed_dict , it's just the numpy array of image pixel, of you, you're trying to classify/predict or whatever you're doing.
I took the code from the following article. This will help you as well. http://cv-tricks.com/tensorflow-tutorial/save-restore-tensorflow-models-quick-complete-tutorial/
Update: For your particular case, you may like to try the following code to predict the classes in the new images.
import tensorflow as tf
slim = tf.contrib.slim
from inception_resnet_v2 import *
#Well, since you're using resnet_v2, this may be equivalent to you.
checkpoint_file = 'inception_resnet_v2_2016_08_30.ckpt'
sample_images = ['dog.jpg', 'panda.jpg']
#Load the model
sess = tf.Session()
arg_scope = inception_resnet_v2_arg_scope()
with slim.arg_scope(arg_scope):
logits, end_points = inception_resnet_v2(input_tensor, is_training=False)
#With this, you could consider the op_variable with the following
predict_values, logit_values = sess.run([end_points['Predictions'], logits], feed_dict={input_tensor: im})
#Here im is the normalized numpy array of the image pixels.
Furthermore, the following resources may help you even more:
Using pre-trained inception_resnet_v2 with Tensorflow
https://github.com/tensorflow/tensorflow/issues/7172
I want to restore a tensorflow model after it's trained. I know that I can use tf.train.Saver but the problem is with the restoring because I get confused with the names for get_tensor_by_name. Can anybody help me?
This is my graph:
x_hat = tf.placeholder(tf.float32, shape=[None, dim_img], name='input_img')
x = tf.placeholder(tf.float32, shape=[None, dim_img], name='target_img')
# dropout
keep_prob = tf.placeholder(tf.float32, name='keep_prob')
# input for PMLR
z_in = tf.placeholder(tf.float32, shape=[None, dim_z], name='latent_variable')
# network architecture
y, z, loss, neg_marginal_likelihood, KL_divergence = vae.autoencoder(x_hat, x, dim_img, dim_z, n_hidden,
keep_prob)
When you save a model you save 2 things: 1) the meta graph, that is a representation of the graph (all the TF symbols you've defined; and 2) the checkpoint which contains the actual Variable values (which are saved and restored by name).
When you restore you can restore one or both of those components. What you are describing is restoring both the meta graph AND the checkpoint data. In this case you need to look up the various operations and tensors you're interested by name, which can be confusing (especially if you didn't name your variables well, which you should always do).
# In this method you import the meta graph then restore
saver = tf.train.import_meta_graph('my-save-dir/my-model-10000.meta')
saver.restore(sess, 'my-save-dir/my-model-10000')
The other option to restoring (which I prefer mysefl) is to not load the meta graph at all. Instead just re-run the same code you originally used to create the graph (if you've done things well this will all be organized in one place). Then you only restore the checkpoint. This approach has the benefit that you can easily keep a reference to all the operations you'll need (such as cost, train_op, placeholders, etc).
# This method only performs the restor operation
# assuming the graph is already constructure
saver.restore(sess, 'my-save-dir/my-model-10000')
I fine-tuned InceptionNet v3 model given in the slim-tensorflow library. I trained the model on my own dataset. Now, I have .ckpt and .meta file for the model.
Now, as far as from I understand, I have two ways to restore the model and the weights. First, from the .meta file like this
checkpoint = './fine_tuned_model/model.ckpt-233700.meta'
with tf.Session() as sess:
new_saver = tf.train.import_meta_graph(checkpoint)
print(new_saver)
new_saver.restore(sess, tf.train.latest_checkpoint('./fine_tuned_model/'))
The second way is to call the model and restore the checkpoint. Like this
with slim.arg_scope(slim.nets.inception.inception_v3_arg_scope()):
logits, inceptionv3 = nets.inception.inception_v3(inputs=img, num_classes=5980, is_training=True,
dropout_keep_prob=.6)
# Restore convolutional layers:
variables_to_restore = slim.get_variables_to_restore(exclude=['InceptionV3/Logits', 'InceptionV3/AuxLogits'])
init_fn = slim.assign_from_checkpoint_fn(model_path, variables_to_restore)
Now, I think for my purpose second way is easier.
Now, my issue is that after restoring the model, how can I extract features from the last to last layer given an Image? I have included a screenshot of the model here
One solution I found was like this and here as far as I know I have to extract features from PreLogits layer but I am not sure how to set the values here
with tf.Session() as sess:
feature_tensor = sess.graph.get_tensor_by_name('mul:0')
features_last_layer = sess.run(feature_tensor,{inputs: img})
print features_last_layer
Here, I am unable to find out what should I pass to get_tenor_by_name(??) and also, how pass sess.run here?
Thank you. Please let me know if there are other ways to restore the model and get the features.
I have figured out the solution for this.
# Placeholder for the image,
image_tensor = tf.placeholder(tf.float32, shape=(None, 128, 128, 3))
# load the model
with slim.arg_scope(slim.nets.inception.inception_v3_arg_scope()):
logits, inceptionv3 = nets.inception.inception_v3(image_tensor,is_training=False)
# Restore convolutional layers:
variables_to_restore = slim.get_variables_to_restore(exclude=['InceptionV3/Logits', 'InceptionV3/AuxLogits'])
# get the latest checkpoint you want to use after training
init_fn = slim.assign_from_checkpoint_fn(model_path, variables_to_restore)
checkpoint = './fine_tuned_model/model.ckpt-233700.data-00000-of-00001'
saver = tf.train.Saver(variables_to_restore)
saver.restore(sess, tf.train.latest_checkpoint('./fine_tuned_model/'))
# the required image
img_car = cv2.imread('car.jpeg')
img_car = cv2.resize(img_car,(128, 128))
imgnumpy = np.ndarray((1,128,128,3))
imgnumpy[0] = img_car
# get output from any layer you want.
output = sess.run(inceptionv3["PreLogits"], feed_dict={image_tensor: imgnumpy})
I have loaded a pretrained model (Model 1) using the following code:
def load_seq2seq_model(sess):
with open(os.path.join(seq2seq_config_dir_path, 'config.pkl'), 'rb') as f:
saved_args = pickle.load(f)
# Initialize the model with saved args
model = Model1(saved_args)
#Inititalize Tensorflow saver
saver = tf.train.Saver()
# Checkpoint
ckpt = tf.train.get_checkpoint_state(seq2seq_config_dir_path)
print('Loading model: ', ckpt.model_checkpoint_path)
# Restore the model at the checkpoint
saver.restore(sess, ckpt.model_checkpoint_path)
return model
Now, I want to train another model (Model 2) from scratch which will take the output of the Model 1. But for that I need to define a session and load the pre-trained model and initialize the model tf.initialize_all_variables(). So, the pre-trained model will also get initialized.
Can anyone please tell me how to train the Model 2 after getting the output from the pre-trained model Model 1 properly?
What I am trying is given below -
with tf.Session() as sess:
# Initialize all the variables of the graph
seq2seq_model = load_seq2seq_model(sess)
sess.run(tf.initialize_all_variables())
.... Rest of the training code goes here....
All variables that are restored using a saver don't need to be initialized. Therefore, instead of using tf.initialize_all_variables() you can use tf.variables_initializer(var_list) to only initialize the weights of the second network.
To get a list of all the weights of the second network you can create the Model 2 network in a variable scope:
with tf.variable_scope("model2"):
model2 = Model2(...)
Then use
model_2_variables_list = tf.get_collection(
tf.GraphKeys.GLOBAL_VARIABLES,
scope="model2"
)
to get the variable list of the Model 2 network. Finally you can create the initialisier for the second network:
init2 = tf.variables_initializer(model_2_variables_list)
with tf.Session() as sess:
# Initialize all the variables of the graph
seq2seq_model = load_seq2seq_model(sess)
sess.run(init2)
.... Rest of the training code goes here....
I have trained a Tensorflow Model, and now I want to export the "function" to use it in my python program. Is that possible, and if yes, how? Any help would be nice, could not find much in the documentation. (I dont want to save a session!)
I have now stored the session as you suggested. I am loading it now like this:
f = open('batches/batch_9.pkl', 'rb')
input = pickle.load(f)
f.close()
sess = tf.Session()
saver = tf.train.Saver()
saver.restore(sess, 'trained_network.ckpt')
y_pred = []
sess.run(y_pred, feed_dict={x: input})
print(y_pred)
However, I get the error "no Variables to save" when I try to initialize the saver.
What I want to do is this: I am writing a bot for a board game, and the input is the situation on the board formatted into a tensor. Now I want to return a tensor which gives me the best position to play next, i.e. a tensor with 0 everywhere and a 1 at one position.
I don't know if there is any other way to do it, but you can use your model in another Python program by saving your session:
Your training code:
# build your model
sess = tf.Session()
# train your model
saver = tf.train.Saver()
saver.save(sess, 'model/model.ckpt')
In your application:
# build your model (same as training)
sess = tf.Session()
saver = tf.train.Saver()
saver.restore(sess, 'model/model.ckpt')
You can then evaluate any tensor in your model using a feed_dict. This obviously depends on your model. For example:
#evaluate tensor
sess.run(y_pred, feed_dict={x: input_data})