We Keep Coding

transformers and BERT downloading to your local machine

I am trying to replicates the code from this page.
At my workplace we have access to transformers and pytorch library but cannot connect to internet from our python environment. Could anyone help with how we could get the script working after manually downloading files to my machine?
my specific questions are -
should I go to the location bert-base-uncased at main and download all the files? Do I have put them in a folder with a specific name?
How should I change the below code
# Load pre-trained model tokenizer (vocabulary)
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
# Tokenize our sentence with the BERT tokenizer.
tokenized_text = tokenizer.tokenize(marked_text)
How should I change the below code
# Load pre-trained model (weights)
model = BertModel.from_pretrained('bert-base-uncased',
output_hidden_states = True, # Whether the model returns all hidden-states.
)
Please let me know if anyone has done this…thanks
###update1
I went to the link and manually downloaded all files to a folder and specified path of that folder in my code. Tokenizer works but this line model = BertModel.from_pretrained('bert-base-uncased', output_hidden_states = True, # Whether the model returns all hidden-states. ) fails. Any idea what should i do? I noticed that 4 big files when downloaded have very strange name...should I rename them to same names as shown on the above page? Do I need to download any other files?
the error message is OSErrr: unable to load weights from pytorch checkpoint file for bert-base-uncased2/ at bert-base-uncased/pytorch_model.bin If you tried to load a pytroch model from a TF 2 checkpoint, please set from_tf=True

clone the model repo for downloading all the files
git lfs install
git clone https://huggingface.co/bert-base-uncased
# if you want to clone without large files – just their pointers
# prepend your git clone with the following env var:
GIT_LFS_SKIP_SMUDGE=1
git usage:
download git from here https://git-scm.com/downloads
paste these to your cli(terminal):
a. git lfs install
b. git clone https://huggingface.co/bert-base-uncased
wait for download, it will take time. if you want monitor your web performance
find the current directory simply pasting cd to your cli and get the file path(e.g "C:/Users/........./bert-base-uncased" )
use it as:
from transformers import BertModel, BertTokenizer
model = BertModel.from_pretrained("C:/Users/........./bert-base-uncased")
tokenizer = BertTokenizer.from_pretrained("C:/Users/........./bert-base-uncased")
Manual download, without git:
Download all the files from here https://huggingface.co/bert-base-uncased/tree/main
Put them in a folder named "yourfoldername"
use it as:
model = BertModel.from_pretrained("C:/Users/........./yourfoldername")
tokenizer = BertTokenizer.from_pretrained("C:/Users/........./yourfoldername")
For only model(manual download, without git):
just click the download button here and download only pytorch pretrained model. its about 420mb
https://huggingface.co/bert-base-uncased/blob/main/pytorch_model.bin
download config.json file from here https://huggingface.co/bert-base-uncased/tree/main
put both of them in a folder named "yourfilename"
use it as:
model = BertModel.from_pretrained("C:/Users/........./yourfilename")

Answering "###update1" for the error: 'OSErrr: unable to load weights from pytorch checkpoint file for bert-base-uncased2/ at bert-base-uncased/pytorch_model.bin If you tried to load a pytroch model from a TF 2 checkpoint, please set from_tf=True'
Please try this methods from -> https://huggingface.co/transformers/model_doc/bert.html
from transformers import BertTokenizer, BertForMaskedLM
import torch
tokenizer = BertTokenizer.from_pretrained("C:/Users/........./bert-base-uncased")
model = BertForMaskedLM.from_pretrained("C:/Users/........./bert-base-uncased")
inputs = tokenizer("The capital of France is [MASK].", return_tensors="pt")
labels = tokenizer("The capital of France is Paris.", return_tensors="pt")["input_ids"]
outputs = model(**inputs, labels=labels)
loss = outputs.loss
logits = outputs.logits
if this works we understand that there is nothing wrong with filesystem or foldernames.
If it works try to get hiddenstate after(note that bert model already returns hiddenstate as explained: " The bare Bert Model transformer outputting raw hidden-states without any specific head on top." so you dont need to use "output_hidden_states = True,")
from transformers import BertTokenizer, BertModel
import torch
tokenizer = BertTokenizer.from_pretrained("C:/Users/........./bert-base-uncased")
model = BertModel.from_pretrained("C:/Users/........./bert-base-uncased")
inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
outputs = model(**inputs)
last_hidden_states = outputs.last_hidden_state
if this not works try to load pytorch model with one of these methods
# Load all tensors onto the CPU
torch.load("C:/Users/........./bert-base-uncased/pytorch_model.bin", map_location=torch.device('cpu'))
# Load all tensors onto GPU 1
torch.load("C:/Users/........./bert-base-uncased/pytorch_model.bin", map_location=lambda storage, loc: storage.cuda(1))
if pytorch load method is not worked, we understand that there is pytorch version compatibility problem between pytorch 1.4.0 and released bert pytorch model. Or maybe your pytorch_model.bin file not downloaded very well. And please pay attention when pytorch 1.4.0 released the last python was python3.4

How to load a TensorRT SavedModel to a TensorFlow Estimator?

I'm using TensorFlow 1.14, and to load a TensorFlow SavedModel to an Estimator the following code works for me:
estimator = tf.contrib.estimator.SavedModelEstimator(saved_model_dir)
prediction_results = estimator.predict(input_fn)
However, when I used TensorRT to convert the TensorFlow SavedModel to TensorRT SavedModel, it returns an error message:
ValueError: Directory provided has an invalid SavedModel format: saved_models/nvidia_fp16_converted
I have looked at it further, and it looks like the problem is that TensorRT does not generate any variables information (including variables.index) in the SavedModel directory, which makes the above error. Does anyone know how to resolve this problem?

For anyone who is interested, below is the solution I have figured out by myself:
Normally, a TF SavedModel can be loaded to Estimator using:
estimator = tf.contrib.estimator.SavedModelEstimator(SAVED_MODEL_DIR)
However, when loading TensorRT SavedModel errors occur because TensorRT converts all variables to constants, thus there is no variables' information in the SavedModel directory (e.g. no variables.index) → errors occur since Estimator try to load variables. Steps to fix the problem:
We need to go to file: "/usr/local/lib/python3.6/dist-packages/tensorflow_estimator/python/estimator/estimator.py", line 2330, in _get_saved_model_ckpt and comment out the check for variables.index
"""if not gfile.Exists(os.path.join(saved_model_utils.get_variables_dir(saved_model_dir),
compat.as_text('variables.index'))):
raise ValueError('Directory provided has an invalid SavedModel format: %s'
% saved_model_dir)"""
Go to file: "/usr/local/lib/python3.6/dist-packages/tensorflow_estimator/python/estimator/canned/saved_model_estimator.py", line 145, in __init__checkpoint_utils.list_variables(checkpoint)] and make changes so that Estimator does not try to load variables from SavedModel:
"""checkpoint = estimator_lib._get_saved_model_ckpt(saved_model_dir) # pylint: disable=protected-access
vars_to_warm_start = [name for name, _ in
checkpoint_utils.list_variables(checkpoint)]
warm_start_settings = estimator_lib.WarmStartSettings(
ckpt_to_initialize_from=checkpoint,
vars_to_warm_start=vars_to_warm_start)"""
warm_start_settings = estimator_lib.WarmStartSettings(ckpt_to_initialize_from = estimator_lib._get_saved_model_ckpt(saved_model_dir))
Go to file: "/usr/local/lib/python3.6/dist-packages/tensorflow_estimator/python/estimator/canned/saved_model_estimator.py", line 256, in _model_fn_from_saved_modeltraining_util.assert_global_step(global_step_tensor) and comment out the check for "global_step" in case the model is generated from NVIDIA examples (thus no training was done and "global_step" was not set):
#global_step_tensor = training_util.get_global_step(g)
#training_util.assert_global_step(global_step_tensor)
Go to file: "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/checkpoint_utils.py", line 291, in init_from_checkpoint init_from_checkpoint_fn) and just put return at the beginning of the init_from_checkpoint function so that it does not try to load the checkpoint:
def _init_from_checkpoint(ckpt_dir_or_file, assignment_map):
"""See `init_from_checkpoint` for documentation."""
return
The loading process should be fine after all the above changes.

vocab_model.docvecs.doctag_syn0.npy not generated after saving doc2vec model

Following is the code :-
modelDoc = Doc2Vec(size=300, window=5, dm=0, dbow_words=1, hs=0, negative=10, alpha=0.05, min_count=20,
workers=cores, sample=1e-5, seed=0, iter=10)
modelDoc.build_vocab(finalSent)
modelDoc.save(save_model)
my version :
gensim==3.8.1
numpy==1.16.2
after saving the model
only vocab_model file is generated
vocab_model.docvecs.doctag_syn0.npy is not generated.
what is the use of this file and does it is necessary to generate this file.

Were there any errors shown during the .save()?
Does the saved file load & work as expected? (In this case, since the original model wasn't trained, does it train alright as if the save-then-load hadn't happened?)
If there's no error, & it works, it's fine.
(Wha's the reason that a file of this name was expected, and its absence was a concern?)

How to save a trained Pipeline model into a single tflite file?

I trained a Pipeline model, which uses CountVectorizer, TfidfTransformer, OneVsRestClassifier and also a GridSearchCV.
Now I want to save it into a tflite file, to use it on my Android app.
For a Sequential model (where my tflite file was created successfully), I did:
sequential_model = Sequential()
...
# train and fit the model
...
h5_file = "h5_model.h5"
tflite_file = "tflite_model.tflite"
sequential_model.save(h5_file)
converter = tf.lite.TFLiteConverter.from_keras_model_file(h5_file)
tflite_model = converter.convert()
open(tflite_file, "wb").write(tflite_model)
All good to save Sequential model into a tflite file.
Well, Pipeline has no attribute "save", unlike a Sequential model, so I tried saving the Pipeline model with joblib and then with pickle, but none of them worked.
Let's say that pipeline_model is my trained model (the one described in the first sentence).
pb_file = 'pipeline_model.pb'
# I also tried with other extensions, like h5, hdf5, sav, pkl
joblib.dump(pipeline_model, filename)
# or with pickle equivalent and pkl extension
# pickle.dump(pipeline_model, open(pb_file, 'wb'))
Now the pb file is created and I want to create a tflite one. Since it's not a Keras model, I can't use from_keras_model_file, so I tried instead with from_saved_model.
pb_file = 'pipeline_model.pb'
tflite_file = "tflite_model.tflite"
converter = tf.lite.TFLiteConverter.from_saved_model(pb_file)
tflite_model = converter.convert()
open(tflite_file, "wb").write(tflite_model)
It generates the error on line of converter = ...:
OSError: SavedModel file does not exist at: pb_file.pb/{saved_model.pbtxt|saved_model.pb}
I tried running it on Kaggle, Colab, PyCharm IDE, with both versions of tensorflow (1 and 2), with different file extensions and nothing seems to work.
I also noticed that TFLiteConverter contains the methods from_frozen_graph and from_session, but these two requires an extra parameter, so I don't think these could be the solution.
So, how can I obtain my tflite file from the trained Pipeline model? Please, if you find any solution, tell me the library versions that you used, since there could be a different behaviour on different libs.

Export Tensorflow Estimator

I'm trying to build a CNN with Tensorflow (r1.4) based on the API tf.estimator. It's a canned model. The idea is to train and evaluate the network with estimator in python and use the prediction in C++ without estimator by loading a pb file generated after the training.
My first question is, is it possible?
If yes, the training part works and the prediction part works too (with pb file generated without estimator) but it doesn't work when I load a pb file from estimator.
I got this error : "Data loss: Can't parse saved_model.pb as binary proto"
My pyhon code to export my model :
feature_spec = {'input_image': parsing_ops.FixedLenFeature(dtype=dtypes.float32, shape=[1, 48 * 48])}
export_input_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(feature_spec)
input_fn = tf.estimator.inputs.numpy_input_fn(self.eval_features,
self.eval_label,
shuffle=False,
num_epochs=1)
eval_result = self.model.evaluate(input_fn=input_fn, name='eval')
exporter = tf.estimator.FinalExporter('save_model', export_input_fn)
exporter.export(estimator=self.model, export_path=MODEL_DIR,
checkpoint_path=self.model.latest_checkpoint(),
eval_result=eval_result,
is_the_final_export=True)
It doesn't work neither with tf.estimator.Estimator.export_savedmodel()
If one of you knows an explicit tutorial on estimator with canned model and how to export it, I'm interested

Please look at this issue on github, it looks like you have the same problem. Apparently (at least when using estimator.export_savedmodel) you should load the graph with LoadSavedModel instead of ReadBinaryProto, because it's not saved as a graphdef file.
You'll find here a bit more instructions about how to use it:
const string export_dir = ...
SavedModelBundle bundle;
...
LoadSavedModel(session_options, run_options, export_dir, {kSavedModelTagTrain},
&bundle);
I can't seem to find the SavedModelBundle documentation for c++ to use it afterwards, but it's likely close to the same class in Java, in which case it basically contains the session and the graph you'll be using.