I am trying to freeze a pre-trained model to then convert it in TF Lite and deploy it into an Android device.
By inspecting the resulting .pb file with xxd I see that it only contains the placeholder output variable. The size of the .pb is a few Bytes.
Why all the graph and variables are not included in the model?
I used the code below derived from https://github.com/sankit1/cv-tricks.com/tree/master/Tensorflow-tutorials/freeze_model_and_deploy. It works fine with other models but not with mine.
import tensorflow as tf
from tensorflow.python.framework import graph_util
import os,sys
path = './model-gaze/'
output_node_names = "pos"
model_name = 'model-23'
saver = tf.train.import_meta_graph(path+model_name+'.meta', clear_devices=True)
graph = tf.get_default_graph()
input_graph_def = graph.as_graph_def()
sess = tf.Session()
saver.restore(sess, path+model_name)
output_graph_def = graph_util.convert_variables_to_constants(
sess, # The session is used to retrieve the weights
input_graph_def, # The graph_def is used to retrieve the nodes
output_node_names.split(",") # The output node names are used to select the usefull nodes
)
output_graph=path+model_name+".pb"
with tf.gfile.GFile(output_graph, "wb") as f:
f.write(output_graph_def.SerializeToString())
sess.close()
I would expect that all the weights and graph data are included inside the .pb but cannot manage to get them there.
The link which you are following is the right procedure to freeze a tensorflow model.
Freezing a model reduces the size of the model as it only saves the ""output_node_names"" which you give it to store.
Please refer to the below link on the entire process.
https://cv-tricks.com/how-to/freeze-tensorflow-models/
Here, can you please elaborate on what ""pos"" is ?
Also, here if you pass the prediction op as that is the required final op for predictions, it should work fine.
And if this does not help, please share your model and the code from where you have saved the model, to further debug the issue.
Related
I have .meta and .ckpt files of the tensorflow model. I wanted to know exact input and output node name but I am getting a list of node names by following this.
When I have a frozen protobuf model, I get the input node name and output node name as the starting and end of the list using this code:
import tensorflow as tf
from tensorflow.python.platform import gfile
GRAPH_PB_PATH = 'frozen_model.pb'
with tf.Session() as sess:
print("load graph")
with gfile.FastGFile(GRAPH_PB_PATH,'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
sess.graph.as_default()
tf.import_graph_def(graph_def, name='')
graph_nodes=[n for n in graph_def.node]
names = []
for t in graph_nodes:
names.append(t.name)
print(names)
Can I do something similar for .ckpt or .meta file ?
The .meta file contains information about the different node in the tensorflow graph. This has been better explained here.
The values of the different variables in the graph at that moment are stored separately in the checkpoint folder in checkpoint.data-xxxx-of-xxxx file.
There is no concept of an input or output node in the normal checkpoint process, as opposed to the case of a frozen model. Freezing a model outputs a subset of the whole tensorflow graph. This subset of the main graph has only those nodes present on which the output node is dependent on. Because freezing a model is done for serving purposes, it converts the tensorflow variables to constants, eliminating the need for storing additional information like gradients of the different variables at each step.
If you still want to identify the nodes you would be interested in, you can restore your graph from the .meta file and visualize it in tensorboard.
import tensorflow as tf
from tensorflow.summary import FileWriter
sess = tf.Session()
tf.train.import_meta_graph("your-meta-graph-file.meta")
FileWriter("__tb", sess.graph)
This will create a __tb folder in your current directory and you can then view the graph by issuing the following command.
tensorboard --logdir __tb
Here is a link to the screenshot of some model with a node selected. You can get the name of the node from the top right corner.
Context:
I have a simple classifier based on tf.estimator.DNNClassifier that takes text and output probabilities over an intent tags. I am able to train an export the model to a serveable as well as serve the serveable using tensorflow serving. The problem is this servable is too big (around 1GB) and so I wanted to try some tensorflow graph transforms to try to reduce the size of the files being served.
Problem:
I understand how to take the saved_model.pb and use freeze_model.py to create a new .pb file that can be used to call transforms on. The result of these transforms (a .pb file as well) is not a servable and cannot be used with tensorflow serving.
How can a developer go from:
saved model -> graph transforms -> back to a servable
There's documentation that suggests that this is certainly possible, but its not at all intuitive from the docs as to how to do this.
What I've Tried:
import tensorflow as tf
from tensorflow.saved_model import simple_save
from tensorflow.saved_model import signature_constants
from tensorflow.saved_model import tag_constants
from tensorflow.tools.graph_transforms import TransformGraph
with tf.Session(graph=tf.Graph()) as sess_meta:
meta_graph_def = tf.saved_model.loader.load(
sess_meta,
[tag_constants.SERVING],
"/model/path")
graph_def = meta_graph_def.graph_def
other_graph_def = TransformGraph(
graph_def,
["Placeholder"],
["dnn/head/predictions/probabilities"],
["quantize_weights"])
with tf.Graph().as_default():
graph = tf.get_default_graph()
tf.import_graph_def(other_graph_def)
in_tensor = graph.get_tensor_by_name(
"import/Placeholder:0")
out_tensor = graph.get_tensor_by_name(
"import/dnn/head/predictions/probabilities:0")
inputs = {"inputs": in_tensor}
outputs = {"outputs": out_tensor}
simple_save(sess_meta, "./new", inputs, outputs)
My idea was to load the servable, extract the graph_def from the meta_graph_def, transform the graph_def and then try to recreate the servable. This seems to be the incorrect approach.
Is there a way to successfully perform transforms (to reduce file size at inference) on a graph from an exported servable, and then recreate a servable with the transformed graph?
Thanks.
Update (2018-08-28):
Found contrib.meta_graph_transform() which looks promising.
Update (2018-12-03):
A related github issue I opened that seems to be resolved in a detailed blog post which is listed at the end of the ticket.
I trained my model with estimator of TensorFlow. It seems that export_savedmodel should be used to make .pb file, but I don't really know how to construct the serving_input_receiver_fn. Anybody any ideas?
Example code is welcomed.
Extra questions:
Is .pb the only file I need when I want to reload the model? Variable unnecessary?
How much will .pb reduced the model file size compared with .ckpt with adam optimizer?
You can use freeze_graph.py to produce a .pb from .ckpt + .pbtxt
if you're using tf.estimator.Estimator, then you'll find these two files in the model_dir
python freeze_graph.py \
--input_graph=graph.pbtxt \
--input_checkpoint=model.ckpt-308 \
--output_graph=output_graph.pb
--output_node_names=<output_node>
Is .pb the only file I need when I want to reload the model? Variable unnecessary?
Yes, You'll have to know you're model's input nodes and output node names too. Then use import_graph_def to load the .pb file and get the input and output operations using get_operation_by_name
How much will .pb reduced the model file size compared with .ckpt with adam optimizer?
A .pb file is not a compressed .ckpt file, so there is no "compression rate".
However, there is a way to optimize your .pb file for inference, and this optimization may reduce the file size as it removes parts of the graph that are training only operations (see the complete description here).
[comment] how can I get the input and output node names?
You can set the input and output node names using the op name parameter.
To list the node names in your .pbtxt file, use the following script.
import tensorflow as tf
from google.protobuf import text_format
with open('graph.pbtxt') as f:
graph_def = text_format.Parse(f.read(), tf.GraphDef())
print [n.name for n in graph_def.node]
[comment] I found that there is a tf.estimator.Estimator.export_savedmodel(), is that the function to store model in .pb directly? And I'm struggling in it's parameter serving_input_receiver_fn. Any ideas?
export_savedmodel() generates a SavedModel which is a universal serialization format for TensorFlow models. It should contain everything's needed to fit with TensorFlow Serving APIs
serving_input_receiver_fn() is a part of those needed things you have to provide in order to generate a SavedModel, it determines the input signature of your model by adding placeholders to the graph.
From the doc
This function has the following purposes:
To add placeholders to the graph that the serving system will feed
with inference requests.
To add any additional ops needed to convert
data from the input format into the feature Tensors expected by the
model.
If you're receiving your inference requests in the form of serialized tf.Examples (which is a typical pattern) then you can use the example provided in the doc.
feature_spec = {'foo': tf.FixedLenFeature(...),
'bar': tf.VarLenFeature(...)}
def serving_input_receiver_fn():
"""An input receiver that expects a serialized tf.Example."""
serialized_tf_example = tf.placeholder(dtype=tf.string,
shape=[default_batch_size],
name='input_example_tensor')
receiver_tensors = {'examples': serialized_tf_example}
features = tf.parse_example(serialized_tf_example, feature_spec)
return tf.estimator.export.ServingInputReceiver(features, receiver_tensors)
[comment] Any idea to list the node names in '.pb'?
It depends on how it was generated.
if it's a SavedModel the use:
import tensorflow as tf
with tf.Session() as sess:
meta_graph_def = tf.saved_model.loader.load(
sess,
[tf.saved_model.tag_constants.SERVING],
'./saved_models/1519232535')
print [n.name for n in meta_graph_def.graph_def.node]
if it's a MetaGraph then use:
import tensorflow as tf
from tensorflow.python.platform import gfile
with tf.Session() as sess:
with gfile.FastGFile('model.pb', 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
sess.graph.as_default()
tf.import_graph_def(graph_def, name='')
print [n.name for n in graph_def.node]
New checkpoint format generates three files: model.ckpt-1000.data-00000-of-00001,model.ckpt-1000.meta,model.ckpt-1000.index. Old checkpoint format only generates two files: model.ckpt-1000 and model.ckpt-1000.meta.
When I restore model wrote with new checkpoint format using code saver.restore(sess, './model.ckpt-1000'), it can restore model successfully. Now current directory has three files model.ckpt-1000.data-00000-of-00001, model.ckpt-1000.meta, model.ckpt-1000.index. If I delete model.ckpt-1000.index, it cannot restore model. Error information:DataLossError (see above for traceback): Unable to open table file ./model.ckpt-1000: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?.
But I notice tensorflow pretrained model that only has one file: *.ckpt. When I use pretrained model provided by tensorflow, it can restore sucessfully. It does not need *.index file.
My tensorflow version is 1.1.0.
How did it save model? Did it use old checkpoint format?
I believe it did use the old checkpoint format. Here's a simple example I used to verify:
import tensorflow as tf
slim = tf.contrib.slim
x = tf.placeholder(tf.float32, [None, 16])
y = slim.fully_connected(x, 4)
saver_v1 = tf.train.Saver(write_version=tf.train.SaverDef.V1)
saver_v2 = tf.train.Saver()
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
saver_v1.save(sess, 'model_v1.ckpt',
write_meta_graph=False,
write_state=False)
saver_v2.save(sess, 'model_v2.ckpt',
write_meta_graph=False,
write_state=False)
The results of ls model* after running the script:
model_v1.ckpt model_v2.ckpt.data-00000-of-00001 model_v2.ckpt.index
System information
OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
Mac os Sierra (10.12.5)
TensorFlow installed from:
Using pip
TensorFlow version (use command below):
1.2.1
The Problem:
I'm trying to save and restore a model trained from Python to Python.
I've the model saved in three .chkpt files (meta, index and data-000000-of-00001) and I'm trying to read it into my session, save the model using add_meta_graph_and_variables and then read it again using the loader: loader.load(session,[tf.saved_model.tag_constants.TRAINING], pathToSaveModel).
This is my code:
First, I restore the weights from the three files containing "data", "index" and "meta" (the metagraph and the weights") into my session using saver restore:
with tf.Session(graph=tf.Graph()) as session:
##HERE IS THE CODE OF MY NETWORK (Very long)
session.run(tf.global_variables_initializer())
#Load
saver = tf.train.Saver()
saver.restore(session, "newModel.chkpt")
features = loadFeatures(["cat2.jpg"])
res = predictions.eval(
feed_dict={
x: features,
keep_prob: 1.0, })
print('Image {} has a prob {} '.format(image, res))
b = saved_model_builder.SavedModelBuilder(pathToSaveModel)
b.add_meta_graph_and_variables(session, [tf.saved_model.tag_constants.TRAINING])
b.save()
With this code, I've a good classification and finally a new folder containing the model saved with add_meta_graph_and_variables:
Now, I want to use the saved model to classify, again, the same image. This time I used the loader instead the restore:
with tf.Session(graph=tf.Graph()) as session:
##HERE IS THE CODE OF MY NETWORK (Very long)
#session.run(tf.global_variables_initializer())
#Load
from tensorflow.python.saved_model import loader
loader.load(session, [tf.saved_model.tag_constants.TRAINING], pathToSaveModel)
features = loadFeatures(["cat2.jpg"])
res = predictions.eval(
feed_dict={
x: features,
keep_prob: 1.0, })
print('Image {} has a prob {} '.format(image, res))
And here comes the problem:
FailedPreconditionError (see above for traceback): Attempting to use uninitialized value b_fcO
[[Node: b_fcO/read = Identity[T=DT_FLOAT, _class=["loc:#b_fcO"], _device="/job:localhost/replica:0/task:0/cpu:0"](b_fcO)]]
If I've tried to use: session.run(tf.global_variables_initializer()) then it works but the classification is not valid, I think that the weights are not being exported / imported from the very beginning and after test many things I'm stuck here.
Any clues about what I'm doing wrong?.
Thanks in advance.
Update:
This is how the model is in three files in the beginning:
Just a few things you should check are:
What is pathToSaveModel?
Where is the checkpoint file?
open the checkpoint file with a text editor: to what folder does it point?
is the path to the weights correct?
By going over these questions I was always able to find the mistake I made. Hope it helps!