I am trying to build and train a network in python using pytorch . My forward method takes two inputs as follows :
def forward(self, x1, x2):
I trained this model in python and saved using torch.jit.script .
Then I load this model in c++ using the torch::jit::load.
How do I now pass the inputs to the model in c++ ?
If I try passing two separate tensors to the forward method like the following
std::vector<torch::jit::IValue> inputs1{tensor1};
std::vector<torch::jit::IValue> inputs2{tensor2};
at::Tensor output = module.forward(inputs1,inputs2).toTensor();
then I receive an error saying that the method forward expects 1 argument, 2 provided.
I can't also concatenate the two tensors since the shapes are different in all axis.
The problem is by concatenating the two tensors and giving the concatenated tensor as input to the model. Then in the forward method, we can create two separate tensors using the concatenated tensor and use them separately for the output computation.
For concatenation to work, I appended the tensors with 0's so that they are of the same size in all axis except the one in which concatenation is to be done.
Related
I can't find anyone who explains to a layman how to load an onnx model into a python script, then use that model to make a prediction when fed an image. All I could find were these lines of code:
sess = rt.InferenceSession("onnx_model.onnx")
input_name = sess.get_inputs()[0].name
label_name = sess.get_outputs()[0].name
pred = sess.run([label_name], {input_name: X.astype(np.float32)})[0]
But I don't know what any of that means. And everywhere I look, everybody already seems to know what they mean, so nobody's explaining it. That would be one thing if I could just run this code, but I can't. It gives me this error:
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid rank for input: Input3 Got: 2 Expected: 4 Please fix either the inputs or the model.
So I need to actually know what those things mean so I can figure out how to fix the error. Will someone knowledgeable please explain?
Let's first start by going over the code you provided, to make everything clear.
sess = ort.InferenceSession("onnx_model.onnx")
This line loads the model into a session object. This means that the layers, functions and weights used in the model are made ready to perform inferences.
input_name = sess.get_inputs()[0].name
label_name = sess.get_outputs()[0].name
The two methods get_inputs and get_outputs each retrieve some meta information about the model, that being what inputs the model expects, and what outputs it can provide. Off of this meta information in these lines, only the first input & output is actually used, and off of these, only the name is being gotten, and saved into variables.
For the last line, let's tackle that part by part.
pred = sess.run(...)[0]
This performs a inference on the model, we'll go over the inputs to this method after this, but for now, the output is a list of different outputs. These outputs are each numpy arrays. In this case only the first output in this list is being used, and saved to the pred variable
([label_name], {input_name: X.astype(np.float32)})
These are the inputs to sess.run. The fist is a list of names of outputs that you want to be computed by the session. The second argument is a dict, where each input's name maps to numpy arrays. These arrays are are expected to be of the same dimension as the ones supplied during creation of the model. Similarly the types of these arrays should also match the types used during creation of the model.
The error you encountered seems to indicate that the supplied array doesn't have the expected dimensions. These intended amount of dimensions seems to be 4.
To gain clarity about what the exact shape and data type of the input array should be, there are visualization tools, like Netron
I'm trying to make an image captioning model using the federated learning library provided by tensorflow, but I'm stuck at this error
Input 0 of layer dense is incompatible with the layer: : expected min_ndim=2, found ndim=1.
this is my input_spec:
input_spec=collections.OrderedDict(x=(tf.TensorSpec(shape=(2048,), dtype=tf.float32), tf.TensorSpec(shape=(34,), dtype=tf.int32)), y=tf.TensorSpec(shape=(None), dtype=tf.int32))
The model takes image features as the first input and a list of vocabulary as a second input, but I can't express this in the input_spec variable. I tried expressing it as a list of lists but it still didn't work. What can I try next?
Great question! It looks to me like this error is coming out of TensorFlow proper--indicating that you probably have the correct nested structure, but the leaves may be off. Your input spec looks like it "should work" from TFF's perspective, so it seems it is probably slightly mismatched with the data you have
The first thing I would try--if you have an example tf.data.Dataset which will be passed in to your client computation, you can simply read input_spec directly off this dataset as the element_spec attribute. This would look something like:
# ds = example dataset
input_spec = ds.element_spec
This is the easiest path. If you have something like "lists of lists of numpy arrays", there is still a way for you to pull this information off the data itself--the following code snippet should get you there:
# data = list of list of numpy arrays
input_spec = tf.nest.map_structure(lambda x: tf.TensorSpec(x.shape, x.dtype), data)
Finally, if you have a list of lists of tf.Tensors, TensorFlow provides a similar function:
# tensor_structure = list of lists of tensors
tf.nest.map_structure(tf.TensorSpec.from_tensor, tensor_structure)
In short, I would reocmmend not specifying input_spec by hand, but rather letting the data tell you what its input spec should be.
I am trying to get the result of a single convolution over an image using tf.keras.backend.conv2d.
The specifications of the input are 227 pixels by 227 pixels, with a channel size of 3 (RGB image.)
The filter size I would like to use is 11x11 and a stride of 4. There is no zero padding included.
I am not married to the idea of using tf.keras.backend.conv2d. I am willing to change methods/packages, just as long as I get a convolved image with the specified requirements above.
Here is the chunk of code I'm trying to make work:
import tensorflow as tf
from tensorflow import keras
import cv2
image = cv2.imread('pic.jpg')
tf.keras.backend.conv2d(image,11,strides=4,data_format="channels_last",dilation_rate=(1))
I get this error message
InvalidArgumentError: cannot compute Conv2D as input #1(zero-based) was expected to be a double tensor but is a int32 tensor [Op:Conv2D] name: convolution/
If there is anything I can add to clarify, please let me know. I can post the entirety of the code, but most of it is irrelevant, at least in my opinion.
Thank you to whoever takes their time to help me!
You are using the wrong function. What you are using is the convolution op, which takes an input and a filter tensor and performs the convolution. As such, the second argument should be the filter tensor itself. You are trying to pass 11 as the filter tensor which obviously doesn't make sense. What I suspect you want to use is tf.keras.layers.Conv2D which takes care of creating the filter according to some specification and then wraps the convolution op as well. Try this:
conv_layer = tf.layers.Conv2D(1, 11, 4)
result = conv_layer(image)
This creates an 11x11 filter and a convolution op with stride 4; the second line then calls the op. I put 1 as the number of filters (first argument) since I don't know what exactly you are trying to do.
I trained an image classifier with tf.keras and exported the model after the training is done to serve it in the cloud and make online predictions.
I served my model on a localhost using :
tensorflow_model_server --model_base $PATH_TO_SAVEDMODEL --rest_api_port=9000 --model_name=saved_model
I was able to make predictions and receive results. When i tried to deploy the model in the cloud i got the error in the title.
The thing is, i want to map the classes names with the prediction results and i was able to achieve that by doing the following :
# after i got the label names i convert the variable to a tensor
label_names_tensor = tf.convert_to_tensor(label_names) # shape (5,)
to export the model i use this :
tf.saved_model.simple_save(
sess,
"./saved_models/v1",
inputs={'image': model.input},
outputs={'label' : label_names_tensor,'prediction': model.output[0]})
NOTE :
model.output has the shape of (?,5)
model.output[0] has the shape of (5,)
this works locally and i get the classes names mapped with the prediction results.
It is obvious where the problem is.. how can i get this to work and map the classes names correctly with the prediction result ?
I tried to use the reshape function but i couldn't get it to work. I think i need to have this in the end :
shape of label_names_tensor --> (?,5)
so i can do this :
--outputs = {'label' : label_names_tensor,'prediction': model.output}
any help is much appreciated
A few introductory notes. First, the reason for the requirement that the outer dimension of inputs be None is to allow for optimizations involving batching of inputs. The inputs are row-based: one row per input feature vector/matrix/tensor. Another assumption is that each input row produces exactly one output row. Since the number of input rows is variable, the number of output rows will be, too.
One consequence of this is that there is no way to output "static" information without repeating it in each of the rows. That said, if you're generally going to only be passing in one input at a time, there won't be any repetition, but you do have the extra overhead of handling the case as if there would be more than one input/output row. You can repeat the labeled rows as follows:
batch_size = tf.expand_dims(tf.shape(model.output)[0], [-1])
new_shape = tf.stack([batch_size[0], -1])
label_names_tensor = tf.reshape(tf.expand_dims(tf.tile(label_names, batch_size), [-1]), new_shape)
# ...
tf.saved_model.simple_save(
sess,
"./saved_models/v1",
inputs={'image': model.input},
outputs={'label' : label_names_tensor,'prediction': model.output})
In Tensorflow, how would I go about selecting between a python list of Tensors in the middle of my graph as an input to the rest of the graph?
Basically, I have a python list of Tensors that are candidates to be used as inputs in the rest of the graph. I want to select from one of them without adding extra dependencies that require all of the Tensors in the list to be computed (I think that would happen if I used tf.cond). How can I select one of them? I can't do it at the python level because I choose the tensor based on a value computed from a placeholder. So for example:'
x = tf.placeholder(tf.int32, shape=(num_steps, None))
y = tf.placeholder(tf.int32, shape=(None,))
lengths = tf.placeholder(tf.int32, shape=(None,))
# Pretend there is a bunch of lines of code here
output_index = max_sequence_length = tf.reduce_max(lengths)
final_output = potential_outputs[output_index] # won't work, output_index is Tensor
# Pretend the rest of the model uses final_output
More info if you want it:
I am unrolling an RNN and I want to only unroll to the maximum length of the sequence. When this is less then the number of unrolling steps, there is a lot of wasted computation. Dynamic_rnn and static_rnn do not meet my needs, so I am trying to come up with my own custom method of unrolling the graph.
To index in tensorflow use tf.slice.
It should be noted that based on the code you provided, I don't think you are indexing the outputs correctly using tf.reduce_max function since this is providing the actual maximum value across a given axis which may not be an integer (but I'm not sure how your network works). You may be looking for tf.argmax that returns to index for the maximum value. The issue with this however is that tensorflow does not a have a gradient defined for tf.argmax so that function cannot be a learned part of your network.