How can I find the input size of an onnx model? I would eventually like to script it from python.
With tensorflow I can recover the graph definition, find input candidate nodes from it and then obtain their size. Can I do something similar with ONNX (or even simpler)?
Thank you
Yes, provided the input model has the information. Note that inputs of an ONNX model may have an unknown rank or may have a known rank with dimensions that are fixed (like 100) or symbolic (like "N") or completely unknown. You can access this as below:
import onnx
model = onnx.load(r"model.onnx")
# The model is represented as a protobuf structure and it can be accessed
# using the standard python-for-protobuf methods
# iterate through inputs of the graph
for input in model.graph.input:
print (input.name, end=": ")
# get type of input tensor
tensor_type = input.type.tensor_type
# check if it has a shape:
if (tensor_type.HasField("shape")):
# iterate through dimensions of the shape:
for d in tensor_type.shape.dim:
# the dimension may have a definite (integer) value or a symbolic identifier or neither:
if (d.HasField("dim_value")):
print (d.dim_value, end=", ") # known dimension
elif (d.HasField("dim_param")):
print (d.dim_param, end=", ") # unknown dimension with symbolic name
else:
print ("?", end=", ") # unknown dimension with no name
else:
print ("unknown rank", end="")
print()
Please do NOT use input as a variable name because it's a built-in function.
The first idea that comes to mind is that using the google.protobuf.json_format.MessageToDict() method if I need the name, data_type, or some properties of a protobuf object. For example:
form google.protobuf.json_format import MessageToDict
model = onnx.load("path/to/model.onnx")
for _input in model.graph.input:
print(MessageToDict(_input))
will gives the output like:
{'name': '0', 'type': {'tensorType': {'elemType': 2, 'shape': {'dim': [{'dimValue': '4'}, {'dimValue': '3'}, {'dimValue': '384'}, {'dimValue': '640'}]}}}}
I'm not very clear whether every model.graph.input is a RepeatedCompositeContainer object or not, but it would be necessary to use the for loop when it is a RepeatedCompositeContainer.
Then you need to get the shape information from the dim field.
model = onnx.load("path/to/model.onnx")
for _input in model.graph.input:
m_dict = MessageToDict(_input))
dim_info = m_dict.get("type").get("tensorType").get("shape").get("dim") # ugly but we have to live with this when using dict
input_shape = [d.get("dimValue") for d in dim_info] # [4,3,384,640]
If you need the only dim, please use message object instead.
model = onnx.load("path/to/model.onnx")
for _input in model.graph.input:
dim = _input.type.tensor_ype.shape.dim
input_shape = [MessgeToDict(d).get("dimValue") for d in dim]
# if you prefer the python naming style, using the line below
# input_shape = [MessgeToDict(d, preserving_proto_field_name=True).get("dim_value") for d in dim]
One line version:
model = onnx.load("path/to/model.onnx")
input_shapes = [[d.dim_value for d in _input.type.tensor_type.shape.dim] for _input in model.graph.input]
Refs:
https://github.com/googleapis/python-vision/issues/70
AttributeError: 'google.protobuf.pyext._message.RepeatedCompositeCo' object has no attribute 'append'
If you use onnxruntime instead of onnx for inference.
Try using the below code.
import onnxruntime as ort
model = ort.InferenceSession("model.onnx", providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
input_shape = model.get_inputs()[0].shape
Related
I tried to print the summary of the SinGAN model, but I got an error which is:
This is the code:
def init_models(opt):
#generator initialization:
netG = models.GeneratorConcatSkip2CleanAdd(opt).to(opt.device)
netG.apply(models.weights_init)
if opt.netG != '':
netG.load_state_dict(torch.load(opt.netG))
summary(netG,input_size=(3, 201, 256))
print(netG)
#discriminator initialization:
netD = models.WDiscriminator(opt).to(opt.device)
netD.apply(models.weights_init)
if opt.netD != '':
netD.load_state_dict(torch.load(opt.netD))
print(netD)
return netD, netG
The problem when i add this line:
summary(netG,input_size=(3, 201, 256))
And I get the complete code from here.
So, is my way wrong? should I use a different variable as a model?
The forward function of your model expects two input images. In torchsummary.summary, you are providing only one input shape, so it is trying to pass only one input image to your model, leaving the second required argument unpassed and hence raising the issue. Read here how to pass inputs to torchsummary.summary when model expects multiple inputs in the forward method.
I want to implement the Style Transfer paper using Pytorch and the VGG19 network.
For this, I need the intermediate output features for some layers.
I named the convolutional modules: ['conv_1', 'conv_2',..., 'conv_16']
For managing the hooks and features I use this method:
class SaveOutput:
#Callable object for saving the layers outputs
def __init__(self):
self.outputs = []
def __call__(self, module, module_in, module_out):
self.outputs.append(module_out)
def clear(self):
self.outputs = []
def addHooksToModel(model, layerNames, hookHandles):
#Remove hooks
for hook in hookHandles:
hook.remove()
hookHandles = []
features = SaveOutput()
for name, module in model.named_modules():
if name in layerNames:
hookHandles.append(module.register_forward_hook(features))
return features, hookHandles
I want to store the features of the content and style separately:
CONTENT_LAYERS = ["conv_14"]
STYLE_LAYERS = ["conv_1","conv_3","conv_5","conv_9","conv_13"]
hook_handles_content = []
hook_handles_style = []
content_features, hook_handles_content = addHooksToModel(model, CONTENT_LAYERS, hook_handles_content)
style_features, hook_handles_style = addHooksToModel(model, STYLE_LAYERS, hook_handles_style)
I then pass the contentImage and styleImage through the network and I expect the content_features.outputs to contain 1 tensor and the style_features.outputs to contain 5 tensors.
model(contentImage)
contentImg_content = content_features.outputs
content_features.clear()
model(styleImage)
styleImg_style = style_features.outputs
style_features.clear()
But in reality, I get 1 tensor for the content_features.outputs(as expected), but 10 tensors for the style_features.outputs(two times the expected).
Same thing happens if I first pass the styleImage and then the contentImage. I get 5 tensors for the style_features.outputs(as expected), but 2 tensors for the content_features.outputs(two times the expected).
Could somebody point me in the right direction. I know I'm missing something, probably in the way Pytorch hooks are working, but I can't figure out what. Thank you!
I don't know if I am the luckiest(for figuring out what's wrong) or the most distracted(for making the mistake in the first place) person.
When passing the contentImage through the network, the features get saved in both the content_features(1 tensor) and style_features(5 tensors) objects. Then, when passing the the styleImage through the network, I get an additional 5 tensors.
The solution was to clear both objects after passing the contentImage:
model(contentImage)
...
content_features.clear()
style_features.clear()
model(styleImage)
I want to pass a list of tf.Strings to the .map(_parse_function) function.
def _parse_function(self, img_path):
img_str = tf.read_file(img_path)
img_decode = tf.image.decode_jpeg(img_str, channels=3)
img_decode = tf.divide(tf.cast(img_decode , tf.float32),255)
return img_decode
When the tf.data.Dataset is of type TensorSliceDataset,
dataset_from_slices = tf.data.Dataset.from_tensor_slices((tensor_with_filenames))
I can simply do
dataset_from_slices.map(_parse_function), which works.
However, dataset_from_generator = tf.data.Dataset.from_generator(...) returns a Dataset which is an instance of FlatMapDataset type and dataset_from_generator.map(_parse_function) gives the following error:
InvalidArgumentError: Input filename tensor must be scalar, but had shape: [32]
If I change the first line to:
img_str = tf.read_file(img_path[0])
that also works but then I only get the first image, which is not what I am looking for. Any suggestions?
It sounds like the elements of your dataset_from_generator are batched. The simplest remedy is to use tf.contrib.data.unbatch() to convert them back into individual elements:
# Each element is a vector of strings.
dataset_from_generator = tf.data.Dataset.from_generator(...)
# Converts each vector of strings into multiple individual elements.
dataset = dataset_from_generator.apply(tf.contrib.data.unbatch())
dataset = dataset.map(_parse_function)
I have created a custom op using the tutorial, and modified it a bit. I want to use this op as input for the compute_gradients method.
My op expects three inputs: target values, predicted values, and another matrix. It returns a new matrix with the same shape as the target values.
However, when I use #ops.RegisterGradient for this method, it expects strange return values, giving this message in one script:
ValueError: Num gradients 1 generated for op name: "SoftDtwJacobianSqEuc"
op: "SoftDtwJacobianSqEuc"
input: "targets"
input: "mul"
input: "strided_slice"
do not match num inputs 3
And this in another script:
ValueError: Num gradients 1 generated for op name: "SoftDtwJacobianSqEuc"
op: "SoftDtwJacobianSqEuc"
input: "decoder_targets"
input: "Reshape"
input: "strided_slice_8"
do not match num inputs 3
A snippet of the code I am running (full example below):
# Calling the op
backwards = soft_dtw_jacobian_sq_euc_module.soft_dtw_jacobian_sq_euc(decoder_targets, decoder_predictions, alignment_matrix[1:-1,1:-1])
# Need to register, other wise get error: No gradient defined for operation...
#ops.RegisterGradient("SoftDtwJacobianSqEuc")
def _soft_dtw_jacobian_sq_euc_grad(op, grad):
# To generate the error which mentions the expected return values:
return(None)
# This kind of works in the first script, as I can 'mul'tiply the gradient with the output of the op
#return(None, soft_dtw_jacobian_sq_euc_module.soft_dtw_jacobian_sq_euc(decoder_targets, decoder_predictions, alignment_matrix[1:-1,1:-1]), None)
optimizer = tf.train.AdamOptimizer(0.02)
params = tf.trainable_variables()
gradients = optimizer.compute_gradients(backwards, params)
train_op = optimizer.apply_gradients(gradients)
Why is RegisterGradients expecting different return values? How does RegisterGradients determine these?
Preferably I would just return the output of the op (since that's what I made it for), but if I do not use RegisterGradient I get a "No gradient defined for operation..." error.
I have a complete working example here: python part and c++ op
Using TensorFlow 1.2.1 and python 2.7
I had the same error before. The error message was generated in tensorflow function gradients_impl.py.
def _VerifyGeneratedGradients(grads, op):
"""Verify that gradients are valid in number and type.
Args:
grads: List of generated gradients.
op: Operation for which the gradients where generated.
Raises:
ValueError: if sizes of gradients and inputs don't match.
TypeError: if type of any gradient is not valid for its input.
"""
if len(grads) != len(op.inputs):
raise ValueError("Num gradients %d generated for op %s do not match num "
"inputs %d" % (len(grads), op.node_def, len(op.inputs)))
Now we can see this error means for your op SoftDtwJacobianSqEuc , you gave it three inputs. TensorFlow expects you to give three output for the function that generates gradients _soft_dtw_jacobian_sq_euc_grad, each output w.r.t each input for the function for forward propagation.
Note that actually you only want the gradients for the first input, and the other two gradients will not be used anymore in the back propagation, you can just return two more fake_gradients with the right shape.
Although my answer seems to be a little bit late, still hope it helps!
I'm trying to tf.split a tensor based on the dimension of an input fed in using feed_dict (dimension of input changes with each batch). Currently I keep getting an error saying that a tensor cannot be split with a "Dimension". Is there a way to get the value of the dimension and split using it?
Thanks!
input_d = tf.placeholder(tf.int32, [None, None], name="input_d")
# toy feed dict
feed = {
input_d: [[20,30,40,50,60],[2,3,4,5,-1]] # document
}
W_embeddings = tf.get_variable(shape=[vocab_size, embedding_dim], \
initializer=tf.random_uniform_initializer(-0.01, 0.01),\
name="W_embeddings")
document_embedding = tf.gather(W_embeddings, input_d)
timesteps_d = document_embedding.get_shape()[1]
doc_input = tf.split(1, timesteps_d, document_embedding)
tf.split takes a python integer for the num_split argument. However, document_embedding.get_shape() returns a TensorShape, and document_embedding.get_shape()[1] gives a Dimension instance, hence you get an error says "can't split with a Dimension".
Try timestep_ds = document_embedding.get_shape().as_list()[1], this statement should give you a python integer.
Here are some relevant documentations for tf.split and tf.Tensor.get_shape