I need to change the input size of an ONNX model from [1024,2048,3] to [1,1024,2048,3].
For this, I've tried using update_inputs_outputs_dims by ONNX
import onnx
from onnx.tools import update_model_dims
model = onnx.load("./0818_pspnet_1.0_713_resnet_v1/pspnet_citysc.onnx")
updated_model = update_model_dims.update_inputs_outputs_dims(model, {"inputs:0":[1,1024,2048,3]}, {"predictions:0":[1, 1025, 2049, 1]})
onnx.save(updated_model, 'pspnet_citysc_upd.onnx')
However, this is the error I end up with.
ValueError: Unable to set dimension value to 1 for axis 0 of inputs:0. Contradicts existing dimension value 1024.
The ONNX model is exported from a Tensorflow frozen graph of PSPNet. If the above approach does not work, would I need to modify the frozen graph?
Any help is greatly appreciated.
You can use the dynamic shape fixed tool from onnxruntime
python -m onnxruntime.tools.make_dynamic_shape_fixed --dim_param batch --dim_value 1 model.onnx model.fixed.onnx
Related
I have this model: https://github.com/williamyang1991/DualStyleGAN and try to convert it to CoreML. So far I create copy of original Colab notebook and append at the end two blocks:
!pip install coremltools
import coremltools as ct
and
##title Convert inverted image.
inverted_latent = torch.Tensor(result_latents[0][4]).cuda().unsqueeze(0).unsqueeze(1)
with torch.no_grad():
net.eval()
[sampled_src, sampled_dst] = net(inverted_latent, input_is_latent=True)[0]
traced_model = torch.jit.trace(net, inverted_latent)
mlmodel = ct.convert(traced_model, inputs=[ct.ImageType(name="input", shape=inverted_latent.shape,bias=[-1,-1,-1],scale=2.0/255.0)])
mlmodel.save("modelsaved.mlmodel")
To run it, you should put any image with face to /content and in /usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py
replace round method at 545, 546 lines with np.round
But then it fails at
mlmodel = ct.convert(...
with:
RuntimeError: PyTorch convert function for op 'pythonop' not implemented.
I suggest that there the way to rewrite this module with methods that could be convert, am I right? But I can't to figure out how to find the source of this module.
So my question is:
If I think in a right way, how I can find the source of module?
And if I wrong, please advise me the right way to do it.
The code starts by loading the model into PyTorch.
The code then converts the model into CoreML format and saves it to a .mlmodel file.The code below will take the existing PyTorch model and convert it into a CoreML model with input and output features.
The outputs are saved in the file example.mlmodel which can be opened in Xcode or any other development environment that supports CoreML models.
import torch
import coremltools
model = torch.load('MyPyTorchModel.pt')
coreml_model = coremltools.converters.pytorch.from_pytorch(model,
input_features=
['input'],
output_features=
['output'])
coreml_model.save('MyCoreMLModel.mlmodel')
I have read the Coreml guide which shows how to convert a pb model to mlmodel by using coremltools. However, I get the error below when trying to follow the guide. Which means the input shape must be specific.
ValueError: "ResizeBilinear" op: the second input, which is the output size, must be known statically
So, have anyone know how to convert the flexible input shape mlmodel?
Here is my code:
import coremltools as ct
def mlmodel_image(pb):
input_shape = ct.Shape(shape=(1, ct.RangeDim(1, 720), ct.RangeDim(1, 1280), 3))
model_input = ct.ImageType(shape=input_shape)
mlmodel = ct.convert(pb, inputs=[model_input], source='TensorFlow')
mlmodel.save(pb.replace(".pb", "_img.mlmodel"))
print('------save to ', pb.replace(".pb", "_img.mlmodel"))
please try my sample:
https://github.com/dhrebeniuk/RealTimeFastStyleTransfer
And look my article with attached Google Colab Notebook in PyTorch.
There is instructions how run Style Transfer on iOS with maximum performance.
I was trying to convert my .pb file to a CoreML model using tfcoreml (with the script below) when I got this error: ValueError: Input and filter shapes must be int or symbolic in Conv2D node detector/darknet-53/Conv/Conv2D. How would I resolve this error and convert my model to CoreML successfully?
I opened up my .pb model with Netron and found the layer in question:
Here is the conversion script I am using:
import tfcoreml
tfcoreml.convert(tf_model_path='model.pb',
mlmodel_path='model.mlmodel',
output_feature_names=['output_boxes'], # name of the output op
input_name_shape_dict={'inputs': [None, 416, 416, 3]}, # map from the placeholder op in the graph to shape (can have -1s)
minimum_ios_deployment_target='13')
From what I can gather it seems to me that I need to change the input types of all Conv2D nodes to get this to work. But I am not an expert by any means so I could be wrong. Is there a way I can fix this model to convert successfully by using a Python script, and if so what would it look like?
EDIT: After changing the None in the input_name_shape_dict I got a different error. This one says: ValueError: Incompatible dimension 3 in Sub operation detector/darknet-53/Conv/BatchNorm/FusedBatchNorm/Sub. So I opened up Netron once again I took a look. Here is what I got, any idea how to fix it?
It seems the script got past the previous error just to get stuck on the next layer. Is my .pb completely useless or does just need to be fixed in some places?
Try 1 instead of None in the input_name_shape_dict.
I'm trying to do post-training full 8-bit quantization of a Keras model to compile and deploy to EdgeTPU.
I have a trained Keras model saved as .h5 file, and am trying to go through the steps as specified here: https://coral.withgoogle.com/docs/edgetpu/models-intro/, for deployment to the Coral Dev Board.
I'm following these instructions for quantization: https://www.tensorflow.org/lite/performance/post_training_quantization#full_integer_quantization_of_weights_and_activations)
I’m trying to use the following code:
import tensorflow as tf
num_calibration_steps = 100
def representative_dataset_gen():
for _ in range(num_calibration_steps):
# Get sample input data as a numpy array in a method of your choosing.
yield [X_train_quant_conv]
converter = tf.compat.v1.lite.TFLiteConverter.from_keras_model_file('/tmp/classNN_simple.h5')
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
converter.representative_dataset = representative_dataset_gen
tflite_full_integer_quant_model = converter.convert()
where X_train_quant_conv is a subset of my training data converted to np.array and of type np.float32
When running this piece of code, I get the following error:
ValueError: Cannot set tensor: Dimension mismatch
I’ve tried changing the function representative_dataset_gen() in different ways, but every time I get a new error. I’m not sure how this function should be. I’m also in doubt of what value num_calibration_steps should have.
Any suggestions or working examples are very appreciated.
This question is very similar to this answered question: Convert Keras model to quantized Tensorflow Lite model that can be used on Edge TPU
You might want to look at my demo script for quantization, on github.
It's just a guess since I can't see what X_train_quant_conv really is, but in my working demo, I yield one image at a time (random data created on the fly, in my case) in representative_dataset_gen(). The image is stored as batch of size 1 (e.g., tensor shape is (1, 56, 56, 32) for my 52x52x32 image). There are 32 channels, though there would typically just be 3, for a color image. I think representative_dataset_gen() has to yield a list containing a tensor (or more than one?) for which the first dimension is of length 1.
image_shape = (56, 56, 32)
def representative_dataset_gen():
num_calibration_images = 10
for i in range(num_calibration_images):
image = tf.random.normal([1] + list(image_shape))
yield [image]
I have a trained Chainer model that I want to use to perform predictions. I can predict images on CPU by default, but I want to use a GPU, and I'm not sure how I should do it.
Here is what my code looks like:
model = MyModel()
chainer.serializers.load_npz("snapshot", model)
image = load_image(path) # returns a numpy array
with chainer.no_brackprop_mode(), chainer.using_config("train", False):
pred = model.__call__(image)
This works fine on CPU. What should I add to it to predict on GPU ?
I tried:
model.to_gpu(0)
chainer.cuda.get_device_from_id(0).use()
convert image to a CuPy array with image = cupy.array(image)
With all of these options, I get an error:
ValueError: numpy and cupy must not be used together
type(W): <type 'cupy.core.core.ndarray'>, type(x): <type 'numpy.ndarray'>
What am I doing wrong here ? How do I perform predictions on GPU ?
Thanks in advance for help.
I finally figured out a way to have it to work:
Instead of using cupy.array(image), I used cuda.to_gpu(image) and then cuda.to_cpu(image). I'm not sure of the difference between these but still, it works.