I've trained models in Keras and I need to load this models in code made in Tflearn.
I can save the Keras trained models with the line:
model.save('my_model.hdf5') #keras
It saves the architecture and weights of the trained model.
or the line:
model.save_weights('my_model.hdf5') #keras
It saves only the weights of the trained model.
The purpose is to load the model into tflearn using for example this line:
model.load ('example.tflearn') #tflearn
This line just load the weights, requires that the architecture be passed before.
I tried loading the files generated by .save and .save_weights from Keras with .load of tflearn but it did not work.
Related
I have a keras sequential model. I have saved that model using the command.
tf.keras.models.save_model(model, 'model')
Now it has the following folder structure,
Now I am loading the model using
model = tf.saved_model.load('model')
I also tried with
model = tf.keras.models.load_model('model')
then I am trying to predict using
model.predict(padded_seq, verbose=0)
it is giving me error
AttributeError: '_UserObject' object has no attribute 'predict'
how to use the predict on the model loaded. I have tried with h5 model, it worked fine. But my main use is with this kind of model which is throwing error.
You are using the incorrect function to load your model (tf.saved_model.load); It does not return a Keras object (from the docs):
The object returned by tf.saved_model.load is not a Keras object (i.e. doesn't have .fit, .predict, etc. methods).
You should be using tf.keras.models.load_model to load a Keras model.
I have encountered the same problem with SavedModel models downloaded from TFHUB (example: InceptionV3), even loading it with tf.keras.models.load_model() returns a plain model (a sort of a basic generic model to allow back-compatibility) that does not have keras API (predict, fit, summary, build, etc) on top of it, the object type is: <tensorflow.python.saved_model.load.Loader._recreate_base_user_object.<locals>._UserObject object at 0x14a42ac2bcf8>
If you want to use just the inference call (predict), you can call your model directly on data (__call__ method is defined) as follow:
model(padded_seq) # or model.__call__(padded_seq)
One workaround I have found to get the Keras API again is wrapping it inside a KerasLayer in a Sequential model as follow:
import tensorflow as tf
import tensorflow_hub as hub
model = tf.keras.Sequential([
hub.KerasLayer("saved/model/path")
])
model.build(<input_shape>)
Now the model supports all Keras API like predict, summary, etc, and this now should work:
model.predict(padded_seq, verbose=0)
I would like to fine-tune a pre-trained transformers model on Question Answering. The model was pre-trained on large engineering & science related corpora.
I have been provided a "checkpoint.pt" file containing the weights of the model. They have also provided me with a "bert_config.json" file but I am not sure if this is the correct configuration file.
from transformers import AutoModel, AutoTokenizer, AutoConfig
MODEL_PATH = "./checkpoint.pt"
config = AutoConfig.from_pretrained("./bert_config.json")
model = AutoModel.from_pretrained(MODEL_PATH, config=config)
The reason I believe that bert_config.json doesn't match "./checkpoint.pt" file is that, when I load the model with the code above, I get the error that goes as below.
Some weights of the model checkpoint at ./aerobert/phase2_ckpt_4302592.pt were not used when initializing BertModel: ['files', 'optimizer', 'model', 'master params']
This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of BertModel were not initialized from the model checkpoint at ./aerobert/phase2_ckpt_4302592.pt and are newly initialized: ['encoder.layer.2.attention.output.LayerNorm.weight', 'encoder.layer.6.output.LayerNorm.bias', 'encoder.layer.7.intermediate.dense.bias', 'encoder.layer.2.output.LayerNorm.bias', 'encoder.layer.21.attention.self.value.bias', 'encoder.layer.11.attention.self.value.bias', ............
If I am correct in assuming that "bert_config.json" is not the correct one, is there a way to load this model correctly without the config.json file?
Is there a way to see the model architecture from the saved weights of checkpoint.pt file?
This is a warning message instead of a error.
It means that the pretrained model is pretrained in some task (such as Question Answering, MLM etc), if your own fine tune task is the same as those pretrained task, then this IS NOT expected; unless this IS expected because some pooler of pretrained model will not be used in fine tune.
But this message doesn't mean that the bert_config.json isn't the right one. You can test it on huggingface's official colab notebook
You can find more information in this issue.
Im trying to save and load weights from the model i have trained.
the code im using to save the model is.
TensorBoard(log_dir='/output')
model.fit_generator(image_a_b_gen(batch_size), steps_per_epoch=1, epochs=1)
model.save_weights('model.hdf5')
model.save_weights('myModel.h5')
Let me know if this an incorrect way to do it,or if there is a better way to do it.
but when i try to load them,using this,
from keras.models import load_model
model = load_model('myModel.h5')
but i get this error:
ValueError Traceback (most recent call
last)
<ipython-input-7-27d58dc8bb48> in <module>()
1 from keras.models import load_model
----> 2 model = load_model('myModel.h5')
/home/decentmakeover2/anaconda3/lib/python3.5/site-
packages/keras/models.py in load_model(filepath, custom_objects, compile)
235 model_config = f.attrs.get('model_config')
236 if model_config is None:
--> 237 raise ValueError('No model found in config file.')
238 model_config = json.loads(model_config.decode('utf-8'))
239 model = model_from_config(model_config,
custom_objects=custom_objects)
ValueError: No model found in config file.
Any suggestions on what i may be doing wrong?
Thank you in advance.
Here is a YouTube video that explains exactly what you're wanting to do: Save and load a Keras model
There are three different saving methods that Keras makes available. These are described in the video link above (with examples), as well as below.
First, the reason you're receiving the error is because you're calling load_model incorrectly.
To save and load the weights of the model, you would first use
model.save_weights('my_model_weights.h5')
to save the weights, as you've displayed. To load the weights, you would first need to build your model, and then call load_weights on the model, as in
model.load_weights('my_model_weights.h5')
Another saving technique is model.save(filepath). This save function saves:
The architecture of the model, allowing to re-create the model.
The weights of the model.
The training configuration (loss, optimizer).
The state of the optimizer, allowing to resume training exactly where you left off.
To load this saved model, you would use the following:
from keras.models import load_model
new_model = load_model(filepath)'
Lastly, model.to_json(), saves only the architecture of the model. To load the architecture, you would use
from keras.models import model_from_json
model = model_from_json(json_string)
For loading weights, you need to have a model first. It must be:
existingModel.save_weights('weightsfile.h5')
existingModel.load_weights('weightsfile.h5')
If you want to save and load the entire model (this includes the model's configuration, it's weights and the optimizer states for further training):
model.save_model('filename')
model = load_model('filename')
Since this question is quite old, but still comes up in google searches, I thought it would be good to point out the newer (and recommended) way to save Keras models.
Instead of saving them using the older h5 format like has been shown before, it is now advised to use the SavedModel format, which is actually a dictionary that contains both the model configuration and the weights.
More information can be found here: https://www.tensorflow.org/guide/keras/save_and_serialize
The snippets to save & load can be found below:
model.fit(test_input, test_target)
# Calling save('my_model') creates a SavedModel folder 'my_model'.
model.save('my_model')
# It can be used to reconstruct the model identically.
reconstructed_model = keras.models.load_model('my_model')
A sample output of this :
Loading model from scratch requires you to build model from scratch,
so you can try saving your model architecture first using model.to_json()
model_architecture = model.to_json()
Save model weighs using
model.save_weights('model_weights.h5')
For loading the weights you need to reconstruct your model using the saved json file
first.
from tensorflow.keras.models import model_from_json
model = model_from_json(model_architecture)
Then load the weights using
model.load_weights('model_weights.h5')
You can now Compile and test the model , No need to retrain
eg
model.compile(loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=keras.optimizers.Adam(lr=0.001), metrics=["accuracy"])
model.evaluate(x_test, y_test, batch_size=32, verbose=2)
I am exploring the c API for tensorflow 2.0.
Problem:
When loading model into python,weights are not restored and thus model seems to be untrained.
Workflow:
I am using TF 2.0 C api to handle the training of my model.
The general setup I follow is:
1. Define model in python using TF keras api.
import tensorflow as tf
from tensorflow import keras
model = keras.Sequential([keras.layers.Dense(128,
input_shape=(784,),
activation='relu'),
keras.layers.Dense(10, activation='softmax')
])
model.compile(optimizer='adam',
loss="categorical_crossentropy",
metrics=['accuracy'])
keras.experimental.export_saved_model(model,"keras_model")
I am using keras.experimental.export_saved_model() because I need the "signature_def['train']" which is not saved when using keras.Model.save().
2. Train model in C using TF 2.0 C api
Saved model is then loaded into my C program via:
TF_LoadSessionFromSavedModel()
Subsequently it is trained and checkpoints are saved with:
TF_SessionRun()
Saving the model creates new checkpoints files ("checkpoint.index" and "checkpoint.data-00000-of-00001") in the "variables" folder of where my model is stored.
3. PROBLEM Reload model in python
After training I reload my model in python. This is where I find out that the loaded model has wights corresponding to the untrained model. I know this because predictions are giberish when my trained model in C was predicting accurately.
I load my model via:
import tensorflow as tf
from tensorflow import keras
model = keras.experimental.load_from_saved_model("keras_model")
Again, I am using keras.experimental.load_from_saved_model() because when I use keras.models.load_model() I get the following ValueError:
ValueError: Importing a SavedModel with tf.saved_model.load requires a 'tags=' argument if there is more than one MetaGraph. Got 'tags=None', but there are 3 MetaGraphs in the SavedModel with tag sets [['train'], ['eval'], ['serve']]. Pass a 'tags=' argument to load this SavedModel.
If I pass a "tags=serve" argument to keras.model.load_model() I get the following TypeError:
TypeError: load_model() got an unexpected keyword argument 'tags'
Trying to load the saved checkpoints file into my model via keras.Model.load_weights() results in an OSErro due to the file format:
h5py/_objects.pyx in h5py._objects.with_phil.wrapper()
h5py/_objects.pyx in h5py._objects.with_phil.wrapper()
h5py/h5f.pyx in h5py.h5f.open()
OSError: Unable to open file (file signature not found)
Questions
What is the correct way to load a trained model via it's checkpoints file?
How do I save a model not using keras.experimental.export_saved_model and still be able to access the signature_def['train']?
I recently found this Project which runs inference of keras model in a browser with GPU support using webgl. I have a few tensorflow project that I would like to run inference on a browser, is there a way to export tensorflow models into hdf5 file so it can be run using keras-js
If you are using Keras, you can do something like this.
model.save_weights('my_model.hdf5')
The only way I can see this working is if you use a Keras model as an interface to your TensorFlow workflow. If you do that, you can do this to save the model and its weights:
# save model
with open(model_save_filename, "w") as model_save_file:
model_json = model.to_json()
model_save_file.write(model_json)
# save model weights
model.save_weights(model_weights_save_filename)
More information on using Keras as an interface to Tensorflow workflows here: https://blog.keras.io/keras-as-a-simplified-interface-to-tensorflow-tutorial.html#using-keras-models-with-tensorflow