Please help me. I am using Tensorflow 2.0 GPU.
I train the model and save in .h5 format
model = keras.Sequential()
model.add(layers.Bidirectional(layers.CuDNNLSTM(self._window_size, return_sequences=True),
input_shape=(self._window_size, x_train.shape[-1])))
model.add(layers.Dropout(rate=self._dropout, seed=self._seed))
model.add(layers.Bidirectional(layers.CuDNNLSTM((self._window_size * 2), return_sequences=True)))
model.add(layers.Dropout(rate=self._dropout, seed=self._seed))
model.add(layers.Bidirectional(layers.CuDNNLSTM(self._window_size, return_sequences=False)))
model.add(layers.Dense(units=1))
model.add(layers.Activation('linear'))
model.summary()
model.compile(
loss='mean_squared_error',
optimizer='adam'
)
# обучаем модель
history = model.fit(
x_train,
y_train,
epochs=self._epochs,
batch_size=self._batch_size,
shuffle=False,
validation_split=0.1
)
model.save('rts.h5')
Then I load this model and use it for forecasting and everything works.
model = keras.models.load_model('rts.h5')
y_hat = model.predict(x_test)
But the question arose of using a trained model in Tensorflow Serving. And the model in .h5 format is not accepted.
I run:
sudo docker run --gpus 1 -p 8501:8501 --mount type=bind,source=/home/alex/PycharmProjects/TensorflowServingTestData/RtsModel,target=/models/rts_model -e MODEL_NAME=rts_model -t tensorflow/serving:latest-gpu
But the question arose of using a trained model in Tensorflow Serving. And the model in .h5 format is not accepted.
I run:
And I get the error:
tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:267] No versions of servable rts_model found under base path /models/rts_model
I try to save the trained model as described here, https://www.tensorflow.org/guide/saved_model#using_savedmodel_with_estimators:
And I get the error:
ValueError: Layer has 2 states but was passed 0 initial states.
I tried to save the model as follows, https://www.tensorflow.org/api_docs/python/tf/keras/models/save_model:
And have the same error:
ValueError: Layer has 2 states but was passed 0 initial states.
The only thing that works to save the model in the format for Tensorflow Serving is:
keras.experimental.export_saved_model(model, 'saved_model/1/')
Saved model work in Serving. But I get a warning that this method is deprecated and will be removed in a future version.
Instructions for updating:
Please use `model.save(..., save_format="tf")` or `tf.keras.models.save_model(..., save_format="tf")`.
And it closed me.
When I try to use these methods, it gives an error.
When I use what works, writes that it is deprecated.
Please, help.
How to save a trained model in Tensorflow 2.0. so that it can be used for Tensorflow Serving.
I was trying to fix this too!
According to the answer here the normal LSTM (i.e. tf.keras.layers.LSTM) will use GPU, and should be used in general over the cuDNNLSTM class unless you specifically need the original implementation (not sure why you would).
According to docs the normal LSTM will use cuDNN implementation if some requirements are met (see below).
When using this LSTM layer, I could successfully save to the tf output type, just using model.save_model('output_path', save_format='tf')
Requirements for LSTM using cuDNN are as follows (note that all the requirements are met with the defaults):
If a GPU is available and all the arguments to the layer meet the requirement of the CuDNN kernel (see below for details), the layer will use a fast cuDNN implementation.
The requirements to use the cuDNN implementation are:
activation == tanh
recurrent_activation == sigmoid
recurrent_dropout == 0
unroll is False
use_bias is True
Inputs are not masked or strictly right padded.
Related
I've been training an efficientnetV2 network using this repository.
The train process goes well and I reach around 93-95% validation accuracy. After that I run an inference process over a set test which contains new images with an acceptable accuracy, around 88% (for example).
After I check if the model works fine on pytorch I need to convert it to ONNX and then to a tensorrt engine. I have a script to run inference with an ONNX model to check if I'm having some problems with the conversion process.
I'm using this code to convert the model:
import torch
from timm.models import create_model
import os
# create model
base_model = create_model(
model_arch,
num_classes=num_classes,
in_chans=3,
checkpoint_path=model_path)
model = torch.nn.Sequential(
base_model,
torch.nn.Softmax(dim=1)
)
model.cpu()
model.eval()
dummy_input = torch.randn(1, 3, 224, 224, requires_grad=True)
torch.onnx.export(model,
dummy_input,
model_export,
verbose=False,
export_params=True,
do_constant_folding=True
)
I've tried several tutorials like this one but unfortunately I'm getting the same result.
I've tried different onset combinations, with and without do_constant_folding, I've even trained another model with parameter called 'exportable', which is a bool and tells the train script if the model is exportable or not (is an experimental feature according to repository's documentation).
Do you have any idea about this issue?
Thanks in advance.
Hard to guess which bug you get, here is few typical:
some layers have not properly converted even after eval
you may need to write timm.create_model(...scriptable=True, exportable=True)
different preprocessing, e.g. timm model input normalized to specific values, after conversion - not.
Do those models output the near values on model(dummy_input) ?
I have been using bert and trying to compile the model using the below line of code.
model = TFBertForSequenceClassification.from_pretrained('bert-base-uncased')
optimizer = tf.keras.optimizers.Adam(learning_rate=learning_rate, epsilon=1e-08)
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy')
callbacks = [tf.keras.callbacks.ModelCheckpoint(filepath=OUTPUT_FOLDER+"\\bert_model.h5",
save_weights_only=True,
monitor='val_loss',
mode='min',
save_best_only=True)]
model.compile(optimizer=optimizer, loss=loss, metrics=[metric])
While compiling the code, I get the type error.
TypeError: Invalid keyword argument(s) in compile: {'steps_per_execution'}
The transformers package version im currently using is 4.11.3. The above code worked like a charm when I used tranformers 4.10.2
How do i get this to work with 4.11.3?
It's because your TensorFlow version is not up to date. I had the same issue a few hours ago. I updated my TensorFlow version to the newest version (and CUDA and cudnn to the respective version) and now it works fine. So just update your TensorFlow version and you will be fine.
Edit: I went from TensorFlow 2.3.0 to 2.7.0
This issue also arise when using an up to date Tensorlflow version, such as v2.8.0, and disabling eager execution.
tf.compat.v1.disable_eager_execution()
This happens because keras models rely on training_v1.py instead of training.py when using graph execution, and the former does not provide the steps_per_execution option.
The file training.py implements the base class for Keras models when using eager execution (default) and training_v1.py does the same for graph mode. While they intend to provide the same functionality, the option steps_per_execution is not available for graph mode though.
This is more of an 'issue' rather than a question but, I noticed something today while trying some transfer learning using Keras. I found that the InceptionV3 model and pre-trained weights on Francois Chollet's repository are different from the Kaggle one. I checked that using the diff command.
Not only that, when I use the code block as below--
from keras.applications.inception_v3 import InceptionV3
Inception_pretrained_weight = '../pre_weight/inception_v3_weights_tf_dim_ordering_tf_kernels_chollet.h5'
pre_trained_model = InceptionV3(input_shape=(160, 160, 3),
include_top=False,
weights=Inception_pretrained_weight)
for lr in pre_trained_model.layers:
lr.trainable = False
for layr in pre_trained_model.layers:
print("layer names: ", layr.name)
I get an error as--
ValueError: Layer #0 (named "conv2d_753" in the current model) was found to correspond to layer convolution2d_1 in the save file. However the new layer conv2d_753 expects 1 weights, but the saved weights have 2 elements.
This does not happen with the model available on the Kaggle page. Have anyone noticed this yet? Does anyone know how and why these models differ ? I found another post on this-here, but it does not really help. Any suggestions would be appreciated.
After a training procedure, I wanted to check the accuracy by loading the created model.h5 and executing an evaluation procedure. However, I am getting a following warning:
/usr/local/lib/python3.5/dist-packages/keras/engine/saving.py:269:
UserWarning: No training configuration found in save file: the model
was not compiled. Compile it manually. warnings.warn('No training
configuration found in save file:
This dist-packages/keras/engine/saving.py file
so the problem in loading created model -> this line of code
train_model = load_model('model.h5')
Problem indicates that the model was not compiled, however, I did it.
optimizer = Adam(lr=lr, clipnorm=0.001)
train_model.compile(loss=dummy_loss, optimizer=optimizer)
I can't understand what I am doing wrong . . .
Please help me! SOS :-(
Intro
I'd like to add to olejorgenb's answer - for a specific scenario, where you don't want to train the model, just use it (e.g. in production).
"Compile" means "prepare for training", which includes mainly setting up the optimizer. It could also have been saved before, and then you can continue the "same" training after loading the saved model.
The fix
But, what about the scenario - I want to just run the model? Well, use the compile=False argument to load_model like that:
trained_model = load_model('model.h5', compile=False)
You won't be able to .fit() this model without using trained_model.compile(...) first, but most importantly - the warning will go away.
Misc Notes
Btw, in my Keras version, the argument include_optimizer has a default of True. This should work also for trainig callbacks like Checkpoint. This means, when loading a model saved by Keras, you can usually count on the optimizer being included (except for the situation: see Hull Gasper's answer).
But, when you have a model which was not trained by Keras (e.g. when converting a model trained by Darknet), the model is saved un-compiled. This produces the warning, and you can get rid of it in the way described above.
Do you get this warning when saving the model?
WARNING:tensorflow:TensorFlow optimizers do not make it possible to access
optimizer attributes or optimizer state after instantiation. As a result, we
cannot save the optimizer as part of the model save file.You will have to
compile your model again after loading it. Prefer using a Keras optimizer
instead (see keras.io/optimizers).
Seems tensorflow optimizers can't be preserved by keras :/
As mentioned keras can't save Tensorflow optimizers. Use the keras one:
optimizer = keras.optimizers.Adam(lr=0.0001, beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.0, amsgrad=False)
model.compile(optimizer=optimizer,
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
model.fit(...)
model.save('...')
This way works for me without manual compiling after calling load.
I am using Keras 2.0.8 with Tensorflow 1.3.0 in Ubuntu 16.04 with Cuda 8.0 and cuDNN 6.
I am using two BatchNormalization layers( keras layers ) in my model and training using tensorflow pipeline.
I am facing two problems here -
BatchNorm layer population parameters( mean and variance ) are not being updated while training even after setting K.learning_phase to True. As a result, inference is failing completely. I need some advice on how to update these parameters between training steps manually.
Secondly, after saving the trained model using tensorflow saver op, when I try to load it, the results cannot be reproduced. It seems the weights are changing. Is there a way to keep the weights same in save-load operation?
I ran into the same problem a few weeks ago. Internally, keras layers can add additional update operations to a model (e.g. batchnorm). So you need to run these additional ops explicitly. For the batchnorm these updates seem to be just some assign_ops which swap the current mean/variance with the new values. If you do not create a keras model this might work; assuming x is a tensor you like to normalize
bn = keras.layers.BatchNormalization()
x = bn(x)
....
sess.run([minimizer_op,bn.updates],K.learning_phase(): 1)
In my workflow, I am creating a keras model (w/o compiling it) and then run the following
model = keras.Model(inputs=inputs, outputs=prediction)
sess.run([minimizer_op,model.updates],K.learning_phase(): 1)
where inputs can be something like
inputs = [keras.layers.Input(tensor=input_variables)]
and outputs is a list of tensorflow tensors. The model seems to aggregate all additional updates operations between inputs and outputs automatically.