Tensorflow object detection API using cnn
Traceback (most recent call last):
File "export_inference_graph.py", line 147, in tf.app.run()
File "C:\Users\Ali Salar\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\platform\app.py", line 126, in run_sys.exit(main(argv))
File "export_inference_graph.py", line 143, in main
FLAGS.output_directory, input_shape)
File "C:\tensorflow2\models\research\object_detection\exporter.py", line 453, in export_inference_graph
graph_hook_fn=None)
File "C:\tensorflow2\models\research\object_detection\exporter.py", line 421, in _export_inference_graph
placeholder_tensor, outputs)
File "C:\tensorflow2\models\research\object_detection\exporter.py", line 280, in write_saved_model
builder = tf.saved_model.builder.SavedModelBuilder(saved_model_path)
File "C:\Users\Ali Salar\Anaconda3\envs\tensorflow\lib\sit`enter code here`e-packages\tensorflow\python\saved_model\builder_impl.py", line 90, in init
"directory: %s" % export_dir)
AssertionError: Export directory already exists. Please specify a different export directory: inference_graph\saved_model
It's saying the directory already exists. Either change your command in the terminal to point to a new directory or remove everything in this directory that you are currently setting as the output directory.
python3 export_inference_graph.py \
--input_type image_tensor \
--pipeline_config_path YOUR_PIPELINE.CONFIG_FILE_PATH \
--trained_checkpoint_prefix YOUR_MODEL_CHECKPOINT_PATH \
--output_directory exported_graphs/saved_model2
Your try changing output directory.For example exported_graphs/saved_model2 as above.
Related
I have written the code for an entity extraction model using bert but when I run the train.py file I get a value error.
This is the structure of my code with the configuration file in VSCode, I have downloaded bert models from here
Error
>> (myenv) PS D:\Transformers\bert-entity-extraction> python src/train.py
Configuration Complete!
Traceback (most recent call last):
File "src/train.py", line 83, in <module>
model = EntityModel(num_tag = num_tag, num_pos = num_pos)
File "D:\Transformers\bert-entity-extraction\src\model.py", line 25, in __init__
self.bert = transformers.BertModel.from_pretrained(config.BASE_MODEL_PATH)
File "C:\Users\hp\anaconda3\envs\myenv\lib\site-packages\transformers\modeling_utils.py", line 1080, in from_pretrained
**kwargs,
File "C:\Users\hp\anaconda3\envs\myenv\lib\site-packages\transformers\configuration_utils.py", line 427, in from_pretrained
config_dict, kwargs = cls.get_config_dict(pretrained_model_name_or_path, **kwargs)
File "C:\Users\hp\anaconda3\envs\myenv\lib\site-packages\transformers\configuration_utils.py", line 492, in get_config_dict
user_agent=user_agent,
File "C:\Users\hp\anaconda3\envs\myenv\lib\site-packages\transformers\file_utils.py", line 1289, in cached_path
raise ValueError(f"unable to parse {url_or_filename} as a URL or as a local path")
ValueError: unable to parse D:\Transformers\bert-entity-extraction\input\bert-base-uncased_L-12_H-768_A-12\config.json as a URL or as a local path
How to fix this?
I'm trying to traing Mobilenet to recognize custom objects.
I'm following this guide:
https://towardsdatascience.com/how-to-train-your-own-object-detector-with-tensorflows-object-detector-api-bec72ecfe1d9
and using a checkpoint and pipeline.config from here:
ssdlite_mobilenet_v2_coco
The Problem
When I start traing with the following command:
python object_detection/model_main.py \
--pipeline_config_path=C:\t\models\pipeline.config \
--model_dir=C:\t\models\ \
--num_train_steps=50000 \
--alsologtostderr
I get the following:
C:\tensorflow\models-master\research>path=C:\t\models\pipeline.config \ --model_dir=C:\t\models\ \ --num_train_steps=50000 \ --alsologtostderr
WARNING:tensorflow:Estimator's model_fn (<function create_model_fn.<locals>.model_fn at 0x0000013B6CD26C80>) includes params argument, but params are not pa
ssed to Estimator.
Traceback (most recent call last):
File "object_detection/model_main.py", line 101, in <module>
tf.app.run()
File "C:\Python36\lib\site-packages\tensorflow\python\platform\app.py", line 125, in run
_sys.exit(main(argv))
File "object_detection/model_main.py", line 97, in main
tf.estimator.train_and_evaluate(estimator, train_spec, eval_specs[0])
File "C:\Python36\lib\site-packages\tensorflow\python\estimator\training.py", line 447, in train_and_evaluate
return executor.run()
File "C:\Python36\lib\site-packages\tensorflow\python\estimator\training.py", line 531, in run
return self.run_local()
File "C:\Python36\lib\site-packages\tensorflow\python\estimator\training.py", line 681, in run_local
eval_result, export_results = evaluator.evaluate_and_export()
File "C:\Python36\lib\site-packages\tensorflow\python\estimator\training.py", line 886, in evaluate_and_export
hooks=self._eval_spec.hooks)
File "C:\Python36\lib\site-packages\tensorflow\python\estimator\estimator.py", line 453, in evaluate
input_fn, hooks, checkpoint_path)
File "C:\Python36\lib\site-packages\tensorflow\python\estimator\estimator.py", line 1346, in _evaluate_build_graph
model_fn_lib.ModeKeys.EVAL))
File "C:\Python36\lib\site-packages\tensorflow\python\estimator\estimator.py", line 985, in _get_features_and_labels_from_input_fn
result = self._call_input_fn(input_fn, mode)
File "C:\Python36\lib\site-packages\tensorflow\python\estimator\estimator.py", line 1074, in _call_input_fn
return input_fn(**kwargs)
File "C:\Python36\lib\site-packages\object_detection\inputs.py", line 493, in _eval_input_fn
transform_input_data_fn=transform_and_pad_input_data_fn)
File "C:\Python36\lib\site-packages\object_detection\builders\dataset_builder.py", line 150, in build
raise ValueError('Unsupported input_reader_config.')
ValueError: Unsupported input_reader_config.
A comment in "dataset_builder.py" says:
Raises:
ValueError: On invalid input reader proto.
ValueError: If no input paths are specified.
Question:
Is it a problem with pipeline.config file?
Does it mean that "dataset_builder.py" can't read it?
OR
Shall I pass some additional input path as it stated in the comment?
If I remember correctly, cause of the problem was I didn't prepare test data. There was a training data only. So I prepared a list of test images with related XML files and generated test TFrecord.
Then the error has gone.
P.S.
I had a lot of other errors later but it's another story :)
I am trying to export trained TensorFlow models to C++ using freeze_graph.py. I am trying to export the ssd_mobilenet_v1_coco_2017_11_17 model using the following syntax:
bazel build tensorflow/python/tools:freeze_graph && \
bazel-bin/tensorflow/python/tools/freeze_graph \
--input_graph=frozen_inference_graph.pb \
--input_checkpoint=model.ckpt \
--output_graph=/tmp/frozen_graph.pb --output_node_names=softmax
The terminal said that the build was successful but showed the following error:
Traceback (most recent call last):
File "/home/my_username/tensorflow/bazel-bin/tensorflow/python/tools/freeze_graph.runfiles/org_tensorflow/tensorflow/python/tools/freeze_graph.py", line 350, in <module>
app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "/home/my_username/tensorflow/bazel-bin/tensorflow/python/tools/freeze_graph.runfiles/org_tensorflow/tensorflow/python/platform/app.py", line 124, in run
_sys.exit(main(argv))
File "/home/my_username/tensorflow/bazel-bin/tensorflow/python/tools/freeze_graph.runfiles/org_tensorflow/tensorflow/python/tools/freeze_graph.py", line 249, in main
FLAGS.saved_model_tags)
File "/home/my_username/tensorflow/bazel-bin/tensorflow/python/tools/freeze_graph.runfiles/org_tensorflow/tensorflow/python/tools/freeze_graph.py", line 227, in freeze_graph
input_graph_def = _parse_input_graph_proto(input_graph, input_binary)
File "/home/my_username/tensorflow/bazel-bin/tensorflow/python/tools/freeze_graph.runfiles/org_tensorflow/tensorflow/python/tools/freeze_graph.py", line 171, in _parse_input_graph_proto
text_format.Merge(f.read(), input_graph_def)
File "/home/my_username/.cache/bazel/_bazel_my_username/3572bc2aff1de1dd37356cf341944e54/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/tools/freeze_graph.runfiles/protobuf_archive/python/google/protobuf/text_format.py", line 525, in Merge
descriptor_pool=descriptor_pool)
File "/home/my_username/.cache/bazel/_bazel_my_username/3572bc2aff1de1dd37356cf341944e54/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/tools/freeze_graph.runfiles/protobuf_archive/python/google/protobuf/text_format.py", line 579, in MergeLines
return parser.MergeLines(lines, message)
File "/home/my_username/.cache/bazel/_bazel_my_username/3572bc2aff1de1dd37356cf341944e54/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/tools/freeze_graph.runfiles/protobuf_archive/python/google/protobuf/text_format.py", line 612, in MergeLines
self._ParseOrMerge(lines, message)
File "/home/my_username/.cache/bazel/_bazel_my_username/3572bc2aff1de1dd37356cf341944e54/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/tools/freeze_graph.runfiles/protobuf_archive/python/google/protobuf/text_format.py", line 627, in _ParseOrMerge
self._MergeField(tokenizer, message)
File "/home/my_username/.cache/bazel/_bazel_my_username/3572bc2aff1de1dd37356cf341944e54/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/tools/freeze_graph.runfiles/protobuf_archive/python/google/protobuf/text_format.py", line 671, in _MergeField
name = tokenizer.ConsumeIdentifierOrNumber()
File "/home/my_username/.cache/bazel/_bazel_my_username/3572bc2aff1de1dd37356cf341944e54/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/tools/freeze_graph.runfiles/protobuf_archive/python/google/protobuf/text_format.py", line 1144, in ConsumeIdentifierOrNumber
raise self.ParseError('Expected identifier or number, got %s.' % result)
google.protobuf.text_format.ParseError: 2:1 : Expected identifier or number, got `.
On running that command again, I got this message:
WARNING: /home/my_username/tensorflow/tensorflow/core/BUILD:1814:1: in includes attribute of cc_library rule //tensorflow/core:framework_headers_lib: '../../external/nsync/public' resolves to 'external/nsync/public' not below the relative path of its package 'tensorflow/core'. This will be an error in the future. Since this rule was created by the macro 'cc_header_only_library', the error might have been caused by the macro implementation in /home/my_username/tensorflow/tensorflow/tensorflow.bzl:1138:30
WARNING: /home/my_username/tensorflow/tensorflow/core/BUILD:1814:1: in includes attribute of cc_library rule //tensorflow/core:framework_headers_lib: '../../external/nsync/public' resolves to 'external/nsync/public' not below the relative path of its package 'tensorflow/core'. This will be an error in the future. Since this rule was created by the macro 'cc_header_only_library', the error might have been caused by the macro implementation in /home/my_username/tensorflow/tensorflow/tensorflow.bzl:1138:30
WARNING: /home/my_username/tensorflow/tensorflow/contrib/learn/BUILD:15:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated target '//tensorflow/contrib/session_bundle:exporter': No longer supported. Switch to SavedModel immediately.
WARNING: /home/my_username/tensorflow/tensorflow/contrib/learn/BUILD:15:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated target '//tensorflow/contrib/session_bundle:gc': No longer supported. Switch to SavedModel immediately.
INFO: Analysed target //tensorflow/python/tools:freeze_graph (0 packages loaded).
INFO: Found 1 target...
Target //tensorflow/python/tools:freeze_graph up-to-date:
bazel-bin/tensorflow/python/tools/freeze_graph
INFO: Elapsed time: 0.419s, Critical Path: 0.00s
INFO: Build completed successfully, 1 total action
Traceback (most recent call last):
File "/home/my_username/tensorflow/bazel-bin/tensorflow/python/tools/freeze_graph.runfiles/org_tensorflow/tensorflow/python/tools/freeze_graph.py", line 350, in <module>
app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "/home/my_username/tensorflow/bazel-bin/tensorflow/python/tools/freeze_graph.runfiles/org_tensorflow/tensorflow/python/platform/app.py", line 124, in run
_sys.exit(main(argv))
File "/home/my_username/tensorflow/bazel-bin/tensorflow/python/tools/freeze_graph.runfiles/org_tensorflow/tensorflow/python/tools/freeze_graph.py", line 249, in main
FLAGS.saved_model_tags)
File "/home/my_username/tensorflow/bazel-bin/tensorflow/python/tools/freeze_graph.runfiles/org_tensorflow/tensorflow/python/tools/freeze_graph.py", line 227, in freeze_graph
input_graph_def = _parse_input_graph_proto(input_graph, input_binary)
File "/home/my_username/tensorflow/bazel-bin/tensorflow/python/tools/freeze_graph.runfiles/org_tensorflow/tensorflow/python/tools/freeze_graph.py", line 171, in _parse_input_graph_proto
text_format.Merge(f.read(), input_graph_def)
File "/home/my_username/.cache/bazel/_bazel_my_username/3572bc2aff1de1dd37356cf341944e54/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/tools/freeze_graph.runfiles/protobuf_archive/python/google/protobuf/text_format.py", line 525, in Merge
descriptor_pool=descriptor_pool)
File "/home/my_username/.cache/bazel/_bazel_my_username/3572bc2aff1de1dd37356cf341944e54/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/tools/freeze_graph.runfiles/protobuf_archive/python/google/protobuf/text_format.py", line 579, in MergeLines
return parser.MergeLines(lines, message)
File "/home/my_username/.cache/bazel/_bazel_my_username/3572bc2aff1de1dd37356cf341944e54/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/tools/freeze_graph.runfiles/protobuf_archive/python/google/protobuf/text_format.py", line 612, in MergeLines
self._ParseOrMerge(lines, message)
File "/home/my_username/.cache/bazel/_bazel_my_username/3572bc2aff1de1dd37356cf341944e54/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/tools/freeze_graph.runfiles/protobuf_archive/python/google/protobuf/text_format.py", line 627, in _ParseOrMerge
self._MergeField(tokenizer, message)
File "/home/my_username/.cache/bazel/_bazel_my_username/3572bc2aff1de1dd37356cf341944e54/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/tools/freeze_graph.runfiles/protobuf_archive/python/google/protobuf/text_format.py", line 671, in _MergeField
name = tokenizer.ConsumeIdentifierOrNumber()
File "/home/my_username/.cache/bazel/_bazel_my_username/3572bc2aff1de1dd37356cf341944e54/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/tools/freeze_graph.runfiles/protobuf_archive/python/google/protobuf/text_format.py", line 1144, in ConsumeIdentifierOrNumber
raise self.ParseError('Expected identifier or number, got %s.' % result)
google.protobuf.text_format.ParseError: 2:1 : Expected identifier or number, got `.
I am exporting the ssd_mobilenet_v1_coco_2017_11_17 just as for practice. I intend to export my own trained models and test the output with this program.
I have built TensorFlow 1.5 using Bazel v0.11.1. I validated the installation using the following code snippet provided in the TensorFlow website;
# Python
import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))
I also ran the object detection iPython example notebook and it worked.
I am using Ubuntu 17.10.1 on a laptop with an Intel Core i5-8250U CPU, 8GB RAM, 1TB HDD and an NVIDIA MX150(2 GB) GPU. Please help. How do I export a trained model to C++?
in order to export object detection models I use the export_inference_graph.py code in research/object-detection. Here's an example of running the code:
python3 export_inference_graph.py \
--input_type image_tensor \
--pipeline_config_path path/to/model.config \
--trained_checkpoint_prefix path/to/model.ckpt-CHECKPOINTNUMBER \
--output_directory path/to/frozen_inference_graph
Then I use the created frozen_inference_graph.pb with a C++ code that is essentially the same as label_image with little modifications to run for a detection model rather than a classification one.
I am following the below page
https://github.com/tensorflow/models/tree/master/inception
I got to the point I have to run:
bazel-bin/inception/imagenet_train --num_gpus=1 --batch_size=32 --train_dir=/tmp/imagenet_train --data_dir=/tmp/imagenet_data
However, I got below error:
Traceback (most recent call last):
File "/home/demo/anaconda3/envs/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/imagenet_train.py", line 41, in <module>
tf.app.run()
File "/home/demo/anaconda3/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "/home/demo/anaconda3/envs/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/imagenet_train.py", line 35, in main
tf.gfile.DeleteRecursively(FLAGS.train_dir)
File "/home/demo/anaconda3/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/lib/io/file_io.py", line 420, in delete_recursively
pywrap_tensorflow.DeleteRecursively(compat.as_bytes(dirname), status)
File "/home/demo/anaconda3/envs/tensorflow/lib/python2.7/contextlib.py", line 24, in __exit__
self.gen.next()
File "/home/demo/anaconda3/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.FailedPreconditionError: /tmp/imagenet_train
My DATA_DIR is /tmp/imagenet_data from previous step bazel-bin/inception/download_and_preprocess_imagenet "${DATA_DIR}"
But what would be my train_dir? The doc doesn't mention it? Look like an empty folder is incorrect.
For me, it works if I set the path of --train_dir=/tmp. Also, you have the processed dataset in the same directory. The --train_dir and --data_dir should not coincide with each other.
Location of where to place the ImageNet data DATA_DIR=$HOME/imagenet-data
Can you tell me if you are still running into problems after changing the directory?
--train_dir is the path to an empty directory where the model checkpoints and events files are stored as the model is trained.
I was trying to train the im2txt model using Tensorflow that I just built from master branch,
I downloaded all the data sets it needed but when I run the training script:
bazel-bin/im2txt/train \ --input_file_pattern="${MSCOCO_DIR}/train-?????-of-00256" \ --inception_checkpoint_file="${INCEPTION_CHECKPOINT}" \ --train_dir="${MODEL_DIR}/train" \ --train_inception=false \ --number_of_steps=1000000
It shows the following:
Traceback (most recent call last):
File "/home/rvl224/models/im2txt/bazel-bin/im2txt/train.runfiles/im2txt/im2txt/train.py", line 111, in tf.app.run()
File "/home/rvl224/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 30, in run sys.exit(main(sys.argv[:1] + flags_passthrough))
File "/home/rvl224/models/im2txt/bazel-bin/im2txt/train.runfiles/im2txt/im2txt/train.py", line 65, in main model.build()
File "/home/rvl224/models/im2txt/bazel-bin/im2txt/train.runfiles/im2txt/im2txt/show_and_tell_model.py", line 358, in build self.build_inputs()
File "/home/rvl224/models/im2txt/bazel-bin/im2txt/train.runfiles/im2txt/im2txt/show_and_tell_model.py", line 165, in build_inputs image = self.process_image(encoded_image, thread_id=thread_id)
File "/home/rvl224/models/im2txt/bazel-bin/im2txt/train.runfiles/im2txt/im2txt/show_and_tell_model.py", line 119, in process_image image_format=self.config.image_format)
File "/home/rvl224/models/im2txt/bazel-bin/im2txt/train.runfiles/im2txt/im2txt/ops/image_processing.py", line 114, in process_image method=tf.image.ResizeMethod.BILINEAR)
TypeError: resize_images() got an unexpected keyword argument 'new_height'
Is it a problem related to the function resize_images() or I just done something wrong?
Thanks
Update: fix applied
Sorry about this! The signature of the function resize_images(...) in TensorFlow was changed last week, which caused this breakage.
I will get a fix in for this shortly. If you would like to use the fix before then, you need to modify the file im2txt/im2txt/ops/image_processing.py.
Simply change this line:
image = tf.image.resize_images(image,
new_height=resize_height,
new_width=resize_width,
method=tf.image.ResizeMethod.BILINEAR)
to this:
image = tf.image.resize_images(image,
size=[resize_height, resize_width],
method=tf.image.ResizeMethod.BILINEAR)