I'm currently developing a prediction model using Tensorflow and my model works well for a customer, so I'm tring to make it as a real product.
My model needs to be retrained using customer's input as time passes, and it should be deployed on customers infrastructure. (Not a SaaS or cloud.) Moreover, I'd like to protect my codes and models.
From my understanding of Tensorflow, trained model can be exported as protobuf, freezed and kept nodes that are required by prediction. freeze_graph.py at Tensorflow repo, I tried it and I successfully ran my prediction model using Golang + libtensorflow.so runtime. (Or, I could use Tensorflow Serving & C++)
If I can train my model on our company's infra, I could say "Okay, let's get some beers". However, my model has to be trained on the customer's infra, and without python code, it seems like I cannot train my model.
https://www.tensorflow.org/versions/r0.12/how_tos/language_bindings/index.html
At this time, support for gradients, functions and control flow operations ("if" and "while") is not available in languages other than Python. This will be updated when the C API provides necessary support.
Is there any workaround deploying TF app without exposing python code or model? Thanks in advance.
You can still use Python with a pre-trained model, without exposing all the code you needed to build it in the first place. As an example of this, have a look at the Inception retraining code, which loads a pretrained GraphDef and then retrains a new top layer:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/image_retraining/retrain.py
Related
I already have a built predictive model written in Python, however currently it is executed by hand and functions on a single data file. I am hoping to generalize the model so that it can read in different datasets from my backend, each time effectively producing a different model since we are using different data for training as well. How would I be able to add the model onto my backend then?
Store the model as a pickle and read it from your backend when you need analog to your training data.
But you might want to checkout MLFlow for an integrated model handling solution. It is possible to run it on prem. With MLFlow you can easily implement a proper ML lifecycle. You can store your training stats and keep the history of your trained models.
I've been trying 2 years(not continuously, moving to other things and then going back) to compile and run a keras.applications based model in the vision kit, i have unsuccessfully tried lots of approaches(even forgot some), asked questions in forums, in the official project repo, stack overflow, etc with no luck,
for example:
https://github.com/google/aiyprojects-raspbian/issues/698,
https://github.com/google/aiyprojects-raspbian/issues/518,
How to remove layers from a keras model in order to use as baseline for creating another model
https://github.com/keras-team/keras/issues/6464#issuecomment-670829431
After asking many different questions i thought maybe posting directly my use case and asking directly for that could be more promising:
I need to compile and run a keras.applications based model to run it in the aiy vision kit, i know the device is limited so i'm trying to use a model that the documentation says is supported in the vision kit: MobilenetV2 and i'm doing transfer-learning through freezing some of the layers and removing others from keras.applications.MobileNetV2 and then adding custom trainable layers, for testing also i'm trying VGG16, had many issues in the past but most recent issues are:
Even by just using the first layers of the pre-trained model and discarding the rest(the exported .pb file is small, around 2.5 mb) i get(for vgg16,this happens in my computer at compilation time, not in the raspberry device) : Not enough on-device memory to run model.
For MobileNetv2 even when the documentation says it is supported i get:
Check failed: other_op->type == OperatorType::kTensorFlowMerge
Any suggestions for my case? or it's simply just impossible to run keras.applications based model in the vision kit? If it's impossible is it possible to combine the tf for poets mobilenet .pb file with keras output layers and compile that one?
Would really appreciate some help with this or at least a definitive: "no it's not possible" so i don't keep pursuing something that is just not possible.
I am very new to machine learning. I have a python file with a very simple TensorFlow model that I need to deploy on Android using Google's ML Kit (which would be creating tflite file). I absolutely don't understand what should be the structure of my python file and Google's documentation doesn't make it any easier. Maybe someone has a good example of converting CUSTOM MODEL WRITTEN FROM SCRATCH and then using it in Java. I need to pass a string from Android's text field and get a predicted answer.
You need to first train your model on whatever the dataset you have. The layers in the model must comply with the supported layers by the TFLite library. Here is a list of layers that are supported and unsupported.
Once you have trained it, based on how you saved it (Let's say using kerasmodel.save). Convert it to TFLite following this tutorial or other tutorials on this page.
Now you can use this .tflite model in Android studio. For this you can follow this good tutorial.
Given I trained a simple neural network using Tensorflow and Python on my laptop and I want to use this model on my phone in C++ app.
Is there any compatibility format I can use? What is the minimal framework to run neural networks (not to train)?
UDP. I'm also interested in Tensorflow to NOT-Tensorflow compatibility. Do I need to build it up from scratch or there're any best practices?
Yes if you are using iOS or Android. Depending on your specific needs, you have a choice between TensorFlow for Mobile and TensorFlow Lite
https://www.tensorflow.org/mobile/
In particular, to load pre-trained models
https://www.tensorflow.org/mobile/prepare_models
Technically you don't need a framework at all. A conventional fully connected neural network is simple enough that you can implement it in straight C++. It's about 100 lines of code for the matrix multiplication and a dozen or so for the non-linear activation function.
The biggest part is figuring out how to parse a serialized Tensorflow model, especially given that there are quite a few ways to do so. You probably will want to freeze your TensorFlow model; this inserts the weights from the latest training into the model.
I have trained a TensorFlow model and saved a checkpoint, and would like to deploy it to Google Cloud Platform. In the model deployment documentation it says that you need to create a SavedModel. It seems that others also use checkpoints instead of SavedModel.
Given that I have already spent time training this model and only have checkpoints instead of a SavedModel, is there a method I can use to deploy the model still or will I need to retrain?
A checkpoint maps variable names to tensor values. This, as is, is not enough for higher-level systems to use your model. On the other hand, a SavedModel is complete and airtight. As is made clear in the answer you link to in your post, a SavedModel provides all the info needed for serving TensorFlow models: a set of MetaGraphs, a checkpoint compatible with these Graphs and all necessary asset files. If you look at it this way, it makes sense that you need to export your model to a SavedModel in order to deploy it to ML Engine. Now, this does not imply that you need to retrain. What you need to do instead is to wrap one of your checkpoints into a SavedModel.