(I'm using Tensorflow 1.8.0 ...)
The documentation from Keras on how to save a model mentions no difference between saving a sequential model vs. one created from the functional API. But, all of the following blocks of code fail:
import tensorflow as tf
net = tf.keras.models.Model()
net.save('file')
or
import tensorflow as tf
net = tf.keras.models.Model()
print(net.to_json())
or
import tensorflow as tf
net = tf.keras.models.Model()
print(net.to_yaml())
or
import tensorflow as tf
net = tf.keras.models.Model()
print(net.get_config())
They raise a NotImplementedError. In the Keras module, the relevant lines are
if not self._is_graph_network:
raise NotImplementedError
which shows up in .save and get_config (the latter is also called by to_json and to_yaml.
The only thing that DOES work is the following
import tensorflow as tf
net = tf.keras.models.Model()
net.save_weights('file')
in which case the weights are saved successfully and can be successfully loaded with net.load_weights.
However, replacing the second line of the above blocks of code, net = tf.keras.models.Model(), with net = tf.keras.models.Sequential(), making net a sequential model, allows everything above to work.
Is it really not possible to save the structure of a Keras model made with the functional API (using Model rather than Sequential)? Right now, can we only save weights?
Of course its possible to save Model, all your examples have an empty Model, whcih makes no sense to save. Keras' author simply did not implement that case.
If you test with a non-empty Model you will see that saving works perfectly. We use it every day.
Related
I have a keras sequential model. I have saved that model using the command.
tf.keras.models.save_model(model, 'model')
Now it has the following folder structure,
Now I am loading the model using
model = tf.saved_model.load('model')
I also tried with
model = tf.keras.models.load_model('model')
then I am trying to predict using
model.predict(padded_seq, verbose=0)
it is giving me error
AttributeError: '_UserObject' object has no attribute 'predict'
how to use the predict on the model loaded. I have tried with h5 model, it worked fine. But my main use is with this kind of model which is throwing error.
You are using the incorrect function to load your model (tf.saved_model.load); It does not return a Keras object (from the docs):
The object returned by tf.saved_model.load is not a Keras object (i.e. doesn't have .fit, .predict, etc. methods).
You should be using tf.keras.models.load_model to load a Keras model.
I have encountered the same problem with SavedModel models downloaded from TFHUB (example: InceptionV3), even loading it with tf.keras.models.load_model() returns a plain model (a sort of a basic generic model to allow back-compatibility) that does not have keras API (predict, fit, summary, build, etc) on top of it, the object type is: <tensorflow.python.saved_model.load.Loader._recreate_base_user_object.<locals>._UserObject object at 0x14a42ac2bcf8>
If you want to use just the inference call (predict), you can call your model directly on data (__call__ method is defined) as follow:
model(padded_seq) # or model.__call__(padded_seq)
One workaround I have found to get the Keras API again is wrapping it inside a KerasLayer in a Sequential model as follow:
import tensorflow as tf
import tensorflow_hub as hub
model = tf.keras.Sequential([
hub.KerasLayer("saved/model/path")
])
model.build(<input_shape>)
Now the model supports all Keras API like predict, summary, etc, and this now should work:
model.predict(padded_seq, verbose=0)
I have been studying about GradCam and I noticed most cases are used on a Keras/Tensorflow model. However I have a tensorflow lite model that has been compiled to .tflite format. I am not sure if it's even possible to access my CNN layers after it's been compiled, given that I tried using keras library to load the model and it only accepts specific file types, not exactly .tflite since it threw errors:
from tensorflow.keras.models import load_model
model = load_model("/content/drive/My Drive/tensorflow_lite_model.tflite")
It gives the error:
OSError: SavedModel file does not exist
What I was trying to do was to print the .tflite models using model.summary as a way to confirm If I could perform any operation with the model layers. If that is so, then I don't think it's possible to use Grad-Cam with a tensorflow lite model.
Therefore, I would like to know If that is true, or did I just try to validate it, the wrong way?
TFLite model file is a different serialization format with the TensorFlow model formats, keras and saved model.
Since you already have a TFLite model, you need to use the TensorFlow Lite Interpreter API, instead of using the TensorFlow API.
interpreter = tf.lite.Interpreter(model_path="converted_model.tflite")
interpreter.allocate_tensors()
Please refer to this link for the details.
The TF GradCam model can be converted into the TFLite model. It is technically possible to convert any TF models to the corresponding TFLite model. If you have any issues with the conversion, please file bug at the tensorflow github.
I have in Keras a RNN with 5 layers of LSTM cells. This model is already trained, and now I am testing it.
On one hand, I use the function "load_model" from tensorflow.keras.models. Something like
model = load_model("mymodel.hdf5")
On the other hand, I first create an empty model of my RNN, and then I load the weights, something like
loadmodeltemp = load_model("mymodel.hdf5")
model = functionThatWillCreateMyRNN()
model.set_weights(loadmodeltemp.get_weights())
The second option is on general 2xfaster than the first option. I mean when the model is already loaded and you are training or testing it. I am not speaking about the load_model itself, but about what happens afterwards.
I am using archlinux, tensorflow 2.0.0, and keras is imported like "from tensorflow import keras", and the version is 2.2.4-tf.
The GPU is a 1080TI. When forcing CPU it is around 4x slower than the second option.
What's load_model doing that it is causing everything to go slower? It's just that it'd be neater to have a load_model function that does everything, instead of having to load the model and then load the weights.
I would like to train an InceptionV3 Neural Network from scratch. I already have an implementation running which utilizes this TensorFlow Hub module: https://tfhub.dev/google/imagenet/inception_v3/feature_vector/1 and performs fine-tuning utilizing the included pre-trained weights.
I now would like to use the same TensorFlow Hub module but discard the provided weights and use my own kernel initializer (e.g. tf.initializers.truncated_normal, tf.initializers.he_normal, etc...).
How can I modify the trainable variables in the TFHub module to use a custom initializer? Just to be clear, I want to replace the pre-trained weights at runtime, and just keep the model architecture. Please let me know if I really should be using TFSlim, or the model zoo.
Here is what I have so far:
import tensorflow as tf
import tensorflow_hub as hub
tfhub_module_url = 'https://tfhub.dev/google/imagenet/inception_v3/feature_vector/1'
initializer = tf.truncated_normal
def main(_):
_graph = tf.Graph()
with _graph.as_default():
module_spec = hub.load_module_spec(tfhub_module_url)
height, width = hub.get_expected_image_size(module_spec)
resized_input_tensor = tf.placeholder(tf.float32, [None, height, width, 3], name='resized_input_tensor')
m = hub.Module(module_spec, trainable=True)
bottleneck_tensor = m(resized_input_tensor)
trainable_vars = tf.trainable_variables()
# TODO: This fails, because this probably isn't how this is supposed to be done:
for trainable_var in trainable_vars:
trainable_var.initializer = tf.initializers.he_normal
with tf.Session(graph=_graph) as sess:
print(trainable_vars)
tf.logging.set_verbosity(tf.logging.INFO)
tf.app.run()
What is the correct way of doing this?
There is no straightforward way to do what you want to do, because TF Hub modules are really built to represent pre-trained model pieces. If you only want the graph, you could use tensorflow_models/slim code directly. (Or you could patch the tensorflow_hub library code to not rewire variable initializers with restore ops in the first place.)
EDIT 2019-04-15: See also tensorflow_hub issue #267: In TF2, the notion of initializers is going away, so the TF Hub Authors don't want to start depending on it for the TF1 API.
lately I've been playing around with CNNs written in Keras, TF. Unfortunately I've come across one problem there:
In these magnificent tutorials
( https://github.com/Hvass-Labs/TensorFlow-Tutorials/blob/master/vgg16.py ; rest of the code is here),
which I highly recommend, maestro loads pre-trained vgg16.tfmodel in very, very ugly way.
def __init__(self):
# Now load the model from file. The way TensorFlow
# does this is confusing and requires several steps.
# Create a new TensorFlow computational graph.
self.graph = tf.Graph()
# Set the new graph as the default.
with self.graph.as_default():
# TensorFlow graphs are saved to disk as so-called Protocol Buffers
# aka. proto-bufs which is a file-format that works on multiple
# platforms. In this case it is saved as a binary file.
# Open the graph-def file for binary reading.
path = os.path.join(data_dir, path_graph_def)
with tf.gfile.FastGFile(path, 'rb') as file:
# The graph-def is a saved copy of a TensorFlow graph.
# First we need to create an empty graph-def.
graph_def = tf.GraphDef()
# Then we load the proto-buf file into the graph-def.
graph_def.ParseFromString(file.read())
# Finally we import the graph-def to the default TensorFlow graph.
tf.import_graph_def(graph_def, name='')
# Now self.graph holds the VGG16 model from the proto-buf file.
# Get a reference to the tensor for inputting images to the graph.
self.input = self.graph.get_tensor_by_name(self.tensor_name_input_image)
# Get references to the tensors for the commonly used layers.
self.layer_tensors = [self.graph.get_tensor_by_name(name + ":0") for name in self.layer_names]
And the problem is- I want my own pre-trained model load in the same/similiar way, so I can put the model into the graph of the class I am calling later and if possible, get the last lines of the code here working.(Meaning getting the tensors of wanted layers from the graph.)
All my tries based on load_model imported from keras and the comp. graph failed me. Also I didn't want to load it in a completely different way, because I would have to change A LOT of code afterwards- for a newbie kind of a big problem.
Ok, I hope the question will reach the right person and also that it is not too trivial for you :D.
BTW: The complex problem I'm solving, for you to make picture is the style transfer also in the same github repository. (https://github.com/Hvass-Labs/TensorFlow-Tutorials/blob/master/15_Style_Transfer.ipynb)
So you want to load in a keras model into tensorflow basically? Can easily be done with the following code:
import keras.backend as k
from keras.models import load_model
import tensorflow as tf
model = load_model("your model.h5") # now it's in the memory of keras
with k.get_session() as sess:
# here you have a tensorflow computational graph, view it by:
tf.summary.FileWriter("folder name", sess.graph)
# if you need a certain tensor do:
sess.graph.get_tensor_by_name("tensor name")
To have a little read about the get_session function, click here
to view the graph, you need to load in the folder from the FileWriter with tensorboard like this:
tensorboard --logdir path/to/folder
Hope this provided some help, good luck!