My use case is I am training machine learners in python to find which model is most performant. But once I have finished choosing a model, I need to recapitulate that model in C for the shippable product.
What we have been planning to do is pick apart the model in python, save all its parameters to a file, and load these in to a different model with the same structure in C. But this strikes me as unnecessarily complex if there is actually a way to save a trained model as some kind of black-box, compiled library with a predict() function.
So is there a way to compile an object with all its data to a machine-code library? All I find about saving python objects is pickling them, which is clearly not what I need.
Related
Basically I have two models to run in sequence. However, the first is a object-based model trained in TF2, the second is trained in TF1.x saved as name-based ckpt.
The foundamental conflict here is that in tf.compat.v1 mode I have to disable_eager_execution to run the mode, while the other model needs Eager execution (otherwise ~2.5 times slower).
I tried to find a way to convert the TF1 ckpt to object-based TF2 model but I don't think it's an easy way... maybe I have to rebuild the model and copy the weights according to the variable one by one (nightmare).
So Anyone knew if there's a way to just temporarily turn off the eager_excution? That would solve everything here... I would really appreciate it!
I regretfully have to inform you that, in my experience, this is not possible. I had the same issue. I believe the tensorflow documentation actually states that once it is turned off it stays off for the remainder of the session. You cannot turn it back on even if you try. This is a problem anytime you turn off eager execution, and the status will remain as long as the Tensorflow module is loaded in a particular python instance.
My suggestion for transferring the model weights and biases is to dump them to a pickle file as numpy arrays. I know it's possible because the Tensorflow 1.X model I was using did this in its code (I didn't write that model). I ended up loading that pickle file and reconstructing a new Tensorflow 2.X model via a for loop. This works well for sequential models. If any branching occurs the looping method won't work super well, or it will be hard successfully implement.
As a heads up, unless you want to train the model further the best way to load initialize those weights is to use the tf.constant_initializer (or something along those lines). When I did convert the model to Tensorflow 2.X, I ended up creating a custom initializer, but apparently you can just use a regular initializer and then set weights and biases via model or layer attributes or functions.
I ultimately had to convert the Tensorflow 1.x + compat code to Tensorflow 2.X, so I could train the models natively in Tensorflow 2.X.
I wish I could offer better news and information, but this was my experience and solution to the same problem.
I have a model built by following roughly the tutorial provided for the tf.estimator.BoostedTreesClassifier in the docs. I then exported it by using the tf.Estimator.export_saved_model method as described in the SavedModels from Estimators section of the SavedModel docs. This loads in to TensorFlow Serving and answers gRPC and REST requests.
I'd now like to include the explanation factors along with any predictions. Or, less ideally, as a second signature available on the exported model. tf.estimator._BoostedTreesBase.experimental_predict_with_explanations already implements an appropriate algorithm, as described in Local Interpretability section of the docs.
I thought it would be possible to 'extend' the existing estimator in a way that would let me expose this method as another served signature. I've thought of several approaches, but only tried the first two so far:
I've Tried
Change which signatures export_saved_model exports
This didn't go very far. The exposed signatures are a little dynamic, but seem to be limited to the train, predict or eval options defined by tensorflow_core.python.saved_model.model_utils.mode_keys.KerasModeKeys.
Just use an eval_savedmodel?
I briefly thought Eval might be what I was looking for, and followed some of the getting started guide for TensorFlow Model Analysis. The further I go on this path the more it seems like the main difference with an Eval model is how the data is loaded, and that isn't what I want to change.
Subclass the estimator
There are extra caveats with exporting subclassed models. And on top of that an Estimator isn't a Model. It's a model with extra metadata around inputs, outputs and configuration, so I am not clear if a subclassed estimator would even be exportable in the same way a Keras Model is.
I abandoned this subclassing approach without writing much code.
Pull the BoostedTrees Model out of the Estimator
I am not savvy enough to arrange a BoostedTrees model myself, using the low-level primitives. The code in the Estimator that sets it up looks fairly complex. It would be nice to leverage that work, but it seems that the Estimator deals in model_fns, they change depending on the train/predict/eval mode, and it isn't clear what the relationship to a Keras Model is.
I wrote a little code for this, but also gave up on it quickly.
What Next?
Given the above dead ends, which angle should I be persuing further?
Both the low-level export API, and the low-level model building API look like they could get me closer to a solution. The gap between setting up an Estimator, and re-creating one using either API seems fairly wide.
Is it possible I could continue using the existing Estimator, but use the low-level export API to create something with an "interpret" signature that calls through to experimental_predict_with_explanations? Or even "predict and interpret" in a single step? Which tutorial will put me on that path?
I'm using Tensorflow 1.14 and the tf.keras API to build a number (>10) of differnet neural networks. (I'm also interested in the answers to this question using Tensorflow 2). I'm wondering how I should organize my project.
I convert the keras models into estimators using tf.keras.estimator.model_to_estimator and Tensorboard for visualization. I'm also sometimes using model.summary(). Each of my models has a number (>20) of hyperparameters and takes as input one of three types of input data. I sometimes use hyperparameter optimization, such that I often manually delete models and use tf.keras.backend.clear_session() before trying the next set of hyperparameters.
Currently I'm using functions that take hyperparameters as arguments and return the respective compiled keras model to be turned into an estimator. I use three different "Main_Datatype.py" scripts to train models for the three different input data types. All data is loaded from .tfrecord files and there is an input function for each data type, which is used by all estimators taking that type of data as input. I switch between models (i.e. functions returning a model) in the Main scripts. I also have some building blocks that are part of more than one model, for which I use helper functions returning them, piecing together the final result using the Keras functional API.
The slight incompatibilities of the different models are begining to confuse me and I've decided to organise the project using classes. I'm planing to make a class for each model that keeps track of hyperparameters and correct naming of each model and its model directory. However, I'm wondering if there are established or recomended ways to do this in Tensorflow.
Question: Should I be subclassing tf.keras.Model instead of using functions to build models or python classes that encapsulate them? Would subclassing keras.Model break (or require much work to enable) any of the functionality that I use with keras estimators and tensorboard? I've seen many issues people have with using custom Model classes and am somewhat reluctant to put in the work only to find that it doesn't work for me. Do you have other suggestions how to better organize my project?
Thank you very much in advance.
Subclass only if you absolutely need to. I personally prefer following the following order of implementation. If the complexity of the model you are designing, can not be achieved using the first two options, then of course subclassing is the only option left.
tf.keras Sequential API
tf.keras Functional API
Subclass tf.keras.Model
Seems like a reasonable thing to do: https://www.tensorflow.org/guide/keras/custom_layers_and_models https://www.tensorflow.org/api_docs/python/tf/keras/Model guide
Does anyone know which function should I use if I want to use the pre-trained doc2vec models in this website https://github.com/jhlau/doc2vec?
I know we can use the Keyvectors.load_word2vec_format()to laod the word vectors from pre-trained word2vec models, but do we have a similar function to load pre-trained doc2vec models as well in gensim?
Thanks a lot.
When a model like Doc2Vec is saved with gensim's native save(), it can be reloaded with the native load() method:
model = Doc2Vec.load(filename)
Note that large internal arrays may have been saved alongside the main filename, in other filenames with extra extensions – and all those files must be kept together to re-load a fully-functional model. (You still need to specify only the main save file, and the auxiliary files will be discovered at expected names alongside it in the same directory.)
You may have other issues trying to use those pre-trained models. In particular:
as noted in the linked page, the author used a custom variant of gensim that forked off about 2 years ago; the files might not load in standard gensim, or later gensims
it's not completely clear what parameters were used to train those models (though I suppose if you succeed in loading them you could see them as properties in the model), and how much meta-optimization was used for which purposes, and whether those purposes will match your own project
if the parameters are as shown in one of the repo files, [train_model.py][1], some are inconsistent with best practices (a min_count=1 is usually bad for Doc2Vec) or apparent model-size (a mere 1.4GB model couldn't hold 300-dimensional vectors for all of the millions of documents or word-tokens in 2015 Wikipedia)
I would highly recommend training your own model, on a corpus you understand, with recent code, and using metaparameters optimized for your own purposes.
Try this:
import gensim.models as g
model="model_folder/doc2vec.bin" #point to downloaded pre-trained doc2vec model
#load model
m = g.Doc2Vec.load(model)
joblib.dump() seems to be the intended method for storing a trained sklearn models for later load and usage. I like the compression option and ease of use, but later loading with joblib.load() is slow. It takes 20 seconds to load a SVM model, trained on a reasonable small dataset (~10k texts). The model (stored with compress=3, as a recommended compromise) takes 100MB as dumped file.
For my own use (analysis), I needn't worry about speed of loading a model, but I have one I'd like to share with colleagues, and would like to make it as easy and quick as possible for them. I find examples of alternatives to joblib, e.g. pickle, json or hdfs. All are basically the same idea, dump a binary object to disk.
I find langid.py, as an example that I suspect is created in a similar manner as a sklearn model. To me it looks like the whole model is encoded as some kind of string (base64?) and just pasted into the script itself. Is this a solution? How is this done?
Are there any other strategies that might be viable?