The title says it all - I want to save a pytorch model in an s3 bucket. What I tried was the following:
import boto3
s3 = boto3.client('s3')
saved_model = model.to_json()
output_model_file = output_folder + "pytorch_model.json"
s3.put_object(Bucket="power-plant-embeddings", Key=output_model_file, Body=saved_model)
Unfortunately this doesn't work, as .to_json() only works for tensorflow models. Does anyone know how to do it in pytorch?
Try serializing model to a buffer and write it to S3:
buffer = io.BytesIO()
torch.save(model, buffer)
s3.put_object(Bucket="power-plant-embeddings", Key=output_model_file, Body=buffer.getvalue())
First step it's to serialize your model to the file. There are many ways to do it, with basic PyTorch library you do it with out of box tools:
#Serialize entire Model to the
torch.save(the_model, 'you/path/to/model')
Once you have it on disk, you can then upload to S3.
s3 = boto3.resource('s3')
s3.Bucket('bucketname').upload_file('you/path/to/model', 'folder/sub/path/to/s3key')
Later you can simple download and serialize back to the Model.
s3 = boto3.resource('s3')
s3.Bucket('bucketname').download_file(
'folder/sub/path/to/s3key',
'you/path/to/model'
)
the_model = torch.load(PATH)
To expand a bit on the previous answers: there are two different guidelines in the PyTorch documentation on how to save a model, based on what you want to do with it later when you load it again.
If you want to load the model for inference (i.e., to run predictions), then the documentation recommends using torch.save(model.state_dict(), PATH).
If you want to load the model to resume training then the documentation recommends doing a bit more, so that you can properly resume training:
torch.save({
'epoch': epoch,
'model_state_dict': model.state_dict(),
'optimizer_state_dict': optimizer.state_dict(),
'loss': loss,
...
}, PATH)
In terms of moving those saved models into s3, the modelstore open source library could help you with that. Under the hood, this library is calling those same save() functions, creating a zip archive of the resulting files, and then storing models into a structured prefix in an s3 bucket. In practice, using it would look like this:
from modelstore import ModelStore
modelstore = ModelStore.from_aws_s3(os.environ["AWS_BUCKET_NAME"])
model, optim = train() # Your training code
# The upload function takes a domain string to organise and version your models
model_store.pytorch.upload("my-model-domain", model=model, optimizer=optim)
With PyTorch we use a cloudpickle to serialize and save our model:
# Serialize the model
import cloudpickle
with open(path.join(path_to_generic_model_artifact, "model.pkl"), "wb") as outfile:
# regressor is an object of a trained model
cloudpickle.dump(model, outfile)
Deserialize the model:
import pickle
import os
model=pickle.load(open(os.path.join(model_dir, model_file_name), 'rb'))
Related
I trained a Pipeline model, which uses CountVectorizer, TfidfTransformer, OneVsRestClassifier and also a GridSearchCV.
Now I want to save it into a tflite file, to use it on my Android app.
For a Sequential model (where my tflite file was created successfully), I did:
sequential_model = Sequential()
...
# train and fit the model
...
h5_file = "h5_model.h5"
tflite_file = "tflite_model.tflite"
sequential_model.save(h5_file)
converter = tf.lite.TFLiteConverter.from_keras_model_file(h5_file)
tflite_model = converter.convert()
open(tflite_file, "wb").write(tflite_model)
All good to save Sequential model into a tflite file.
Well, Pipeline has no attribute "save", unlike a Sequential model, so I tried saving the Pipeline model with joblib and then with pickle, but none of them worked.
Let's say that pipeline_model is my trained model (the one described in the first sentence).
pb_file = 'pipeline_model.pb'
# I also tried with other extensions, like h5, hdf5, sav, pkl
joblib.dump(pipeline_model, filename)
# or with pickle equivalent and pkl extension
# pickle.dump(pipeline_model, open(pb_file, 'wb'))
Now the pb file is created and I want to create a tflite one. Since it's not a Keras model, I can't use from_keras_model_file, so I tried instead with from_saved_model.
pb_file = 'pipeline_model.pb'
tflite_file = "tflite_model.tflite"
converter = tf.lite.TFLiteConverter.from_saved_model(pb_file)
tflite_model = converter.convert()
open(tflite_file, "wb").write(tflite_model)
It generates the error on line of converter = ...:
OSError: SavedModel file does not exist at: pb_file.pb/{saved_model.pbtxt|saved_model.pb}
I tried running it on Kaggle, Colab, PyCharm IDE, with both versions of tensorflow (1 and 2), with different file extensions and nothing seems to work.
I also noticed that TFLiteConverter contains the methods from_frozen_graph and from_session, but these two requires an extra parameter, so I don't think these could be the solution.
So, how can I obtain my tflite file from the trained Pipeline model? Please, if you find any solution, tell me the library versions that you used, since there could be a different behaviour on different libs.
I'm fairly new to working with AWS, and I want to use SageMaker to train a certain image data set using fast.ai. But I have no clue how to link all the image data from S3 to SageMaker.
I tried almost everything I could think of, used s3fs and I can read the images separately and the list of the images, but how do I feed that info to my databunch or learning algorithm?
My code:
import boto3
import pandas as pd
from sagemaker import get_execution_role
role = get_execution_role()
bucket='sagemaker-sst-images'
data_key = 'SST_Data/sst-images'
data_location = 's3://{}/{}'.format(bucket, data_key)
This code, I think, gives a URL to the data.
But what comes next? Either get it into a path, or load the data properly?
Since you have boto3 imported, you can use that to start a training job in Sagemaker. It can read your training data straight from S3 to train your model on.
If you are using a custom model, this would require having your inference code in EMR
The way you could do it is like this (I use some of the vars you put in your question):
# Get a sagemaker client object for S3
sagemaker_client = boto3.client('sagemaker')
# Create an input channel definition (for the training job call)
input_data = {
'ChannelName': 'train',
'DataSource': {
'S3DataSource': {
'S3DataType': 'S3Prefix',
'S3Uri': data_location
}
},
'ContentType': 'application/png' # This will depend on the type of your images
}
# Start the training job on Sagemaker
sagemaker_client.create_training_job(
TrainingJobName="My training Job",
HyperParameters={ # These will be any hyperparameters you need },
AlgorithmSpecification={
'TrainingImage': # The ECR url to your algorithm goes here,
'TrainingInputMode': 'File'
},
RoleArn=role,
InputDataConfig=input_data,
OutputDataConfig={'S3OutputPath': # Your S3 output path goes here for your trained model artifacts},
ResourceConfig={
'InstanceType': # whichever type/size of instance you want here
'InstanceCount': # however many instances you want to train with,
'VolumeSizeInGB': # however much storage space you need for your training data on the instance
}
)
Here is some information regarding the instances sizes and prices:
https://aws.amazon.com/sagemaker/pricing/instance-types/
https://aws.amazon.com/sagemaker/pricing/
for ContentType in the input config, here is a resource for more info on the MIME type you need: https://www.sitepoint.com/mime-types-complete-list/
After the training completes, you can use the model artifacts it creates to make a SageMaker model, and use it to perform inference.
Context:
I have a simple classifier based on tf.estimator.DNNClassifier that takes text and output probabilities over an intent tags. I am able to train an export the model to a serveable as well as serve the serveable using tensorflow serving. The problem is this servable is too big (around 1GB) and so I wanted to try some tensorflow graph transforms to try to reduce the size of the files being served.
Problem:
I understand how to take the saved_model.pb and use freeze_model.py to create a new .pb file that can be used to call transforms on. The result of these transforms (a .pb file as well) is not a servable and cannot be used with tensorflow serving.
How can a developer go from:
saved model -> graph transforms -> back to a servable
There's documentation that suggests that this is certainly possible, but its not at all intuitive from the docs as to how to do this.
What I've Tried:
import tensorflow as tf
from tensorflow.saved_model import simple_save
from tensorflow.saved_model import signature_constants
from tensorflow.saved_model import tag_constants
from tensorflow.tools.graph_transforms import TransformGraph
with tf.Session(graph=tf.Graph()) as sess_meta:
meta_graph_def = tf.saved_model.loader.load(
sess_meta,
[tag_constants.SERVING],
"/model/path")
graph_def = meta_graph_def.graph_def
other_graph_def = TransformGraph(
graph_def,
["Placeholder"],
["dnn/head/predictions/probabilities"],
["quantize_weights"])
with tf.Graph().as_default():
graph = tf.get_default_graph()
tf.import_graph_def(other_graph_def)
in_tensor = graph.get_tensor_by_name(
"import/Placeholder:0")
out_tensor = graph.get_tensor_by_name(
"import/dnn/head/predictions/probabilities:0")
inputs = {"inputs": in_tensor}
outputs = {"outputs": out_tensor}
simple_save(sess_meta, "./new", inputs, outputs)
My idea was to load the servable, extract the graph_def from the meta_graph_def, transform the graph_def and then try to recreate the servable. This seems to be the incorrect approach.
Is there a way to successfully perform transforms (to reduce file size at inference) on a graph from an exported servable, and then recreate a servable with the transformed graph?
Thanks.
Update (2018-08-28):
Found contrib.meta_graph_transform() which looks promising.
Update (2018-12-03):
A related github issue I opened that seems to be resolved in a detailed blog post which is listed at the end of the ticket.
I trained my model with estimator of TensorFlow. It seems that export_savedmodel should be used to make .pb file, but I don't really know how to construct the serving_input_receiver_fn. Anybody any ideas?
Example code is welcomed.
Extra questions:
Is .pb the only file I need when I want to reload the model? Variable unnecessary?
How much will .pb reduced the model file size compared with .ckpt with adam optimizer?
You can use freeze_graph.py to produce a .pb from .ckpt + .pbtxt
if you're using tf.estimator.Estimator, then you'll find these two files in the model_dir
python freeze_graph.py \
--input_graph=graph.pbtxt \
--input_checkpoint=model.ckpt-308 \
--output_graph=output_graph.pb
--output_node_names=<output_node>
Is .pb the only file I need when I want to reload the model? Variable unnecessary?
Yes, You'll have to know you're model's input nodes and output node names too. Then use import_graph_def to load the .pb file and get the input and output operations using get_operation_by_name
How much will .pb reduced the model file size compared with .ckpt with adam optimizer?
A .pb file is not a compressed .ckpt file, so there is no "compression rate".
However, there is a way to optimize your .pb file for inference, and this optimization may reduce the file size as it removes parts of the graph that are training only operations (see the complete description here).
[comment] how can I get the input and output node names?
You can set the input and output node names using the op name parameter.
To list the node names in your .pbtxt file, use the following script.
import tensorflow as tf
from google.protobuf import text_format
with open('graph.pbtxt') as f:
graph_def = text_format.Parse(f.read(), tf.GraphDef())
print [n.name for n in graph_def.node]
[comment] I found that there is a tf.estimator.Estimator.export_savedmodel(), is that the function to store model in .pb directly? And I'm struggling in it's parameter serving_input_receiver_fn. Any ideas?
export_savedmodel() generates a SavedModel which is a universal serialization format for TensorFlow models. It should contain everything's needed to fit with TensorFlow Serving APIs
serving_input_receiver_fn() is a part of those needed things you have to provide in order to generate a SavedModel, it determines the input signature of your model by adding placeholders to the graph.
From the doc
This function has the following purposes:
To add placeholders to the graph that the serving system will feed
with inference requests.
To add any additional ops needed to convert
data from the input format into the feature Tensors expected by the
model.
If you're receiving your inference requests in the form of serialized tf.Examples (which is a typical pattern) then you can use the example provided in the doc.
feature_spec = {'foo': tf.FixedLenFeature(...),
'bar': tf.VarLenFeature(...)}
def serving_input_receiver_fn():
"""An input receiver that expects a serialized tf.Example."""
serialized_tf_example = tf.placeholder(dtype=tf.string,
shape=[default_batch_size],
name='input_example_tensor')
receiver_tensors = {'examples': serialized_tf_example}
features = tf.parse_example(serialized_tf_example, feature_spec)
return tf.estimator.export.ServingInputReceiver(features, receiver_tensors)
[comment] Any idea to list the node names in '.pb'?
It depends on how it was generated.
if it's a SavedModel the use:
import tensorflow as tf
with tf.Session() as sess:
meta_graph_def = tf.saved_model.loader.load(
sess,
[tf.saved_model.tag_constants.SERVING],
'./saved_models/1519232535')
print [n.name for n in meta_graph_def.graph_def.node]
if it's a MetaGraph then use:
import tensorflow as tf
from tensorflow.python.platform import gfile
with tf.Session() as sess:
with gfile.FastGFile('model.pb', 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
sess.graph.as_default()
tf.import_graph_def(graph_def, name='')
print [n.name for n in graph_def.node]
How do I store the weights and biases in nolearn.lasagne NeuralNet model? From the documentation, I can't see how to access the NeuralNet's weights and biases and store them.
To save the entire nolearn model ( training history, parameters and architecture), you can do this :
import cPickle as pickle
sys.setrecursionlimit(10000) # you may need this if the network is large
with open("model_file", 'wb') as f:
pickle.dump(nolearnnet , f, -1)
Please note that incase you train your model on GPU and pickle it using the above but want to unpickle it on CPU ( or vice versa) , this won't work. In that case you should just save the parameter values , which you can do like this:
weights = lasagne.layers.get_all_param_values(nolearnnet.get_all_layers()[-1])
And now you can save these weights . When you want to load them into another nolearn model, you can just do the following:
lasagne.layers.set_all_param_values(nolearnnet2.get_all_layers()[-1], weights)
It may help to refer to this discussion : https://groups.google.com/forum/#!topic/lasagne-users/BbG95R6SZ0I