I am trying to train a pretrained "faster_rcnn_resnet101_kitti" model for the tensorflow object detection API.
But everytime I try to run
python3 train.py --logtostderr --train_dir='/training/' --pipeline_config_path='/training/faster_rcnn_resnet101_kitti.config'
I receive the following error
Traceback (most recent call last):
File "train.py", line 167, in <module>
tf.app.run()
File "/usr/local/lib/python3.5/dist- packages/tensorflow/python/platform/app.py", line 126, in run
_sys.exit(main(argv))
File "train.py", line 163, in main
worker_job_name, is_chief, FLAGS.train_dir)
File "/usr/local/lib/python3.5/dist-packages/object_detection-0.1-py3.5.egg/object_detection/trainer.py", line 211, in train
detection_model = create_model_fn()
File "/usr/local/lib/python3.5/dist-packages/object_detection-0.1-py3.5.egg/object_detection/builders/model_builder.py", line 96, in build
add_summaries)
File "/usr/local/lib/python3.5/dist-packages/object_detection-0.1-py3.5.egg/object_detection/builders/model_builder.py", line 272, in _build_faster_rcnn_model
frcnn_config.inplace_batchnorm_update)
AttributeError: 'FasterRcnn' object has no attribute 'inplace_batchnorm_update'
I had this error too, and for me it was because I had not re-compiled my .proto-files after I pulled the last updates from the TF models repository.
To recompile (on Linux):
# From tensorflow/models/research/ folder
protoc object_detection/protos/*.proto --python_out=.
I assume that the failing code tries to read the attribute/field inplace_batchnorm_update from the faster rcnn config, which (assumable) does not exist in the older versions. I hope this helps you too.
My versions are: tensorflow-gpu 1.7.0 and have the TF models commit hash 77d3bbefeb33e89bfa1eee707151e5d794d1222b with message "Merge pull request #3888 from hsm207/patch-3 Fix typo".
Recompiling on Windows
I know from own experience that, compared to Windows, compiling many files as above is easy in Linux as a one-liner. For Windows, here is something to make the process less cumbersome:
In this issue, davemers0160 has shared
a script for compiling on Windows.
Just save this file as a .bat-file:
#echo off
setlocal
echo Searching for new .proto files...
for %%F in (object_detection\protos\*.proto) do (
echo %%F
protoc %%F --python_out=.
)
echo Complete!
Run that file from the same folder as mentioned above. As the question was in Linux, I've just added this to the bottom in case a Windows user come along to read this too.
I had the same error after I updated the models repository.
I re-compiled .proto files, but it still has the error.
According to the log:
File "/home/duane/anaconda3/lib/python3.6/site-packages/object_detection-0.1-py3.6.egg/object_detection/builders/model_builder.py", line 164, in _build_ssd_model
inplace_batchnorm_update=ssd_config.inplace_batchnorm_update)
I think maybe it caused by the version of object_detection-0.1-py3.6.egg is too old , So I re-installed models/research/setup.py:
# Form /models/research/
python setup.py build
python setup.py install
Then it has no error.
NOTE: I did re-compile .proto files before I re-install setup.py.
The more details you can see #3968
Hope this can help you.
Related
I will try to be as much help as I can, but this is certainly a bit out of my depth.
I am trying to run the metagenomics package 'DeepVirFinder' on my fasta file 'my_seqs.fa' within terminal on my Mac. I have followed the GitHub repository instructions (as found here https://github.com/jessieren/DeepVirFinder). I have created a conda environment with all the necessary packages.
Into my terminal I have inputted
python dvf.py -i ~/Documents/PairwiseANI/my_seqs.fna -o ~/Documents/DeepVirFinder/ -l 1000 -c 2
this receives an output error of
Using Theano backend.
1. Loading Models.
model directory /data2/joshcole/DeepVirFinder/models
Traceback (most recent call last):
File "dvf.py", line 131, in <module>
modDict[contigLengthk] = load_model(os.path.join(modDir, modName))
File "/home/ggb_joshcole/miniconda3/envs/dvf/lib/python3.6/site-packages/keras/engine/saving.py", line 419, in load_model
model = _deserialize_model(f, custom_objects, compile)
File "/home/ggb_joshcole/miniconda3/envs/dvf/lib/python3.6/site-packages/keras/engine/saving.py", line 224, in _deserialize_model
model_config = json.loads(model_config.decode('utf-8'))
AttributeError: 'str' object has no attribute 'decode'
From the GitHub repository, what it should return upon a successful run (using example template names) is as follows:
Using Theano backend.
1. Loading Models.
model directory /auto/cmb-panasas2/renj/software/DeepVirFinder/models
2. Encoding and Predicting Sequences.
processing line 1
processing line 1389
3. Done. Thank you for using DeepVirFinder.
output in ./test/crAssphage.fa_gt300bp_dvfpred.txt
Any help for how to fix this error would be greatly appreciated. I have tried to download and jig around with potential conda fixes, but it doesn't appear to be a problem with any dependancies + python is fully up to date.
Thank you for reading
Apologies - I found out it was an error between h5py and tensorflow. Had to downgrade h5py to 2.10.0.
I installed openmdao via the documentation here (windows 10 plus anaconda): http://openmdao.org/twodocs/versions/latest/getting_started/getting_started.html
If I actually use the [all] flag it seems that pip tries to download every version of the packages so I just went with pip install openmdao
When I try to run the sample from the above link I get this error:
AttributeError: 'Problem' object has no attribute 'model'
I tried re-running in spyder with the same error then tried the first few lines just in the terminal to verify no model attribute existed.
I tried to skip further down into the code with terminal and got some more errors:
prob.driver=om.ScipyOptimizeDriver()
Traceback (most recent call last):
File "<ipython-input-6-8ea598efdab2>", line 1, in <module>
prob.driver=om.ScipyOptimizeDriver()
AttributeError: module 'openmdao.api' has no attribute 'ScipyOptimizeDriver'
I assumed that maybe there is a disconnect with the different versions floating around with openmdao so I then installed the latest non-dev version and tried to run a few included files in that master folder. All the examples I ran had the same error though:
runfile('C:/Users/Vicconius/Anaconda3/OpenMDAO1-master/examples/beam_tutorial.py', wdir='C:/Users/Vicconius/Anaconda3/OpenMDAO1-master/examples')
Traceback (most recent call last):
File "<ipython-input-7-7e855a208cb8>", line 1, in <module>
runfile('C:/Users/Vicconius/Anaconda3/OpenMDAO1-master/examples/beam_tutorial.py', wdir='C:/Users/Vicconius/Anaconda3/OpenMDAO1-master/examples')
File "C:\Users\Vicconius\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 710, in runfile
execfile(filename, namespace)
File "C:\Users\Vicconius\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 101, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/Vicconius/Anaconda3/OpenMDAO1-master/examples/beam_tutorial.py", line 218, in <module>
top.setup()
File "C:\Users\Vicconius\Anaconda3\OpenMDAO1-master\openmdao\core\problem.py", line 456, in setup
connections = self._setup_connections(params_dict, unknowns_dict)
File "C:\Users\Vicconius\Anaconda3\OpenMDAO1-master\openmdao\core\problem.py", line 234, in _setup_connections
for node in input_graph.nodes_iter():
AttributeError: 'DiGraph' object has no attribute 'nodes_iter'
Any ideas? THANK YOU!!
I can imagine two scenarios:
you happened to create an openmdao folder in your current local directory. You're opening the interpreter or otherwise running python scripts in that same directory. So when you try import openmdao it picks the local folder first --- instead of the installed package. If thats the case, either cd to a different directory or rename/remove that folder.
Something has gone wrong with your install. You somehow have multiple conflicting versions sitting around.
To test, first uninstall with pip:
pip uninstall OpenMDAO.
Then open an interpreter and try
import openmdao
If that somehow works, after you just uninstalled it, that confirms that you somehow have multiple installs.
you need to manually clean up your environment.
inside the interpreter, openmdao.__file__ should tell you where this wonky second install is hiding. You can go delete it manually, and then repeat this import test till you get an ImportError. At that point you know you have found all the rouge installs and you can safely retry a new pip install.
When I run the fully_connected_feed.py code:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/tutorials/mnist/fully_connected_feed.py
I get an error:
Traceback (most recent call last):
File "C:/Users/AppData/Local/Continuum/Anaconda3/Lib/site-packages/tensorflow/examples/tutorials/mnist/fully_connected_feed.py", line 277, in <module>
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "C:\Users\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\platform\app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "C:/Users/AppData/Local/Continuum/Anaconda3/Lib/site-packages/tensorflow/examples/tutorials/mnist/fully_connected_feed.py", line 222, in main
run_training()
File "C:/Users/AppData/Local/Continuum/Anaconda3/Lib/site-packages/tensorflow/examples/tutorials/mnist/fully_connected_feed.py", line 120, in run_training
data_sets = input_data.read_data_sets(FLAGS.input_data_dir, FLAGS.fake_data)
File "C:\Users\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\contrib\learn\python\learn\datasets\mnist.py", line 211, in read_data_sets
SOURCE_URL + TRAIN_IMAGES)
File "C:\Users\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\contrib\learn\python\learn\datasets\base.py", line 142, in maybe_download
gfile.Copy(temp_file_name, filepath)
File "C:\Users\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 316, in copy
compat.as_bytes(oldpath), compat.as_bytes(newpath), overwrite, status)
File "C:\Users\AppData\Local\Continuum\Anaconda3\lib\contextlib.py", line 66, in __exit__
next(self.gen)
File "C:\Users\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 469, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.OutOfRangeError: Read fewer bytes than requested
How do I resolve this issue?
After doing the following, I was able to run the script without errors. The key for getting it to work for me, was the version of tensorflow installed has to match the tutorial code, otherwise there were exceptions. Although, I got a different exception than you, at first.
After installing tensorflow, check version. Details of this step may be different if you installed it pip or some other method:
$ conda list tensorflow
# packages in environment at /Users/agr/miniconda3/envs/tensorflow:
#
tensorflow 0.11.0 py35_0 conda-forge
Clone the git repo
$ git clone https://github.com/tensorflow/tensorflow.git
Inspect the tags available and checkout the release matching your install:
$ cd tensorflow
$ git tag -l -n1
...
$ git checkout v0.11.0
Run script!
$ cd examples/tutorials/mnist/
$ python fully_connected_feed.py
The key point being, run the script from here, not from the link you posted in the original question.
TL; DR
Something else is altering your files as you create them. Find the process and stop it.
Research
I've just run the demo with Windows 10, Python 3.5, tensorflow 0.12.0 with no errors. It is therefore something about your environment.
Looking at the actual line of the error, you are failing to read the required number of bytes from the open file. Going further up the stack you can see that CopyFile is actually trying to read all the bytes of a file into a string in this function. This starts by finding out the current file size and then trying to read all the bytes.
The problem is that the file size at the start of this process doesn't match the size by the end of the copy. In other words, something else has altered your file.
What next?
Your best bet is to try to find out what else is accessing your file. I suggest you use the techniques explained here to see what else has the file open as you are running the copy.
I encountered the same problem on Windows 2012 Server.
As suggested in the previous post, I downloaded and launched Process Monitor, then set the filter: "Path contains mnist" see the image. The datasets were downloaded and unpacked correctly, while running code both from Spyder and Jupyter.
I suspect that there is a race condition in the library code, i.e. missing synchronization between downloading and unpacking operations. As Process Monitor introduced additional delays, the datasets were sucessfully downloaded before the next operation started, hence the hazardous behavior was not observed.
I don't have experience in downloading libraries, so any help is appreciated. I've got a fresh install of Python3.3 and am trying to get IPython for 64bit Windows 7. The IPython .exe installer ran fine and completed normally, but I can't access the program. I looked through their documentation, and tried the commands there, but
$ python setup.py install
returns invalid syntax, highlighting "setup". I thought the fix might be in the distribute library mentioned on the page, so I went to go try and get that first. However, using the script from distribute's install page revealed it's own errors, namely
Traceback (most recent call last):
File "C:\Python33\distribute_setup.py", line 541, in <module>
sys.exit(main())
File "C:\Python33\distribute_setup.py", line 537, in main
tarball = download_setuptools(download_base=options.download_base)
File "C:\Python33\distribute_setup.py", line 200, in download_setuptools
log.warn("Downloading %s", url)
File "C:\Python33\lib\distutils\log.py", line 47, in warn
self._log(WARN, msg, args)
File "C:\Python33\lib\distutils\log.py", line 30, in _log
if stream.errors == 'strict':
AttributeError: errors
I've been at it for a couple hours now, and I'm fresh out of ideas. What next?
It looks like you ran into Python bug 12967. You need to run distribute_setup.py outside of IDLE (i.e. in a command prompt).
I've never used py2exe before. I installed the package, created a setup.py, ran it and it seemed to work without any obvious errors. I went into the dist folder it created and tried to run the exe but I'm getting this error:
Traceback (most recent call last):
File "rl2.py", line 3, in <module>
File "libtcodpy.pyc", line 41, in <module>
File "ctypes\__init__.pyc", line 428, in __getitem__
File "ctypes\__init__.pyc", line 423, in __getattr__
File "ctypes\__init__.pyc", line 353, in __init__
WindowsError: [Error 126] The specified module could not be found
How can I solve this?
What exact commands did you use to get this output?
I recommend using PyInstaller.
You can build an EXE with PyInstaller using this command:
"python pyinstaller.py -y -w -F [PATH\TO\SCRIPT.py] -n [PROJECT_NAME]"
-y: answer yes to every question
-w: windowed (does not show commandline when running)
-F: Compile into one EXE instead of a directory with .dll files
-n: assign project name
Hope I helped
You probably need to manually specify ctypes in the packages section of your setup.py.
See here
I had same issue today. HEre is the solution will work 100%
The meaning of error is the imported dll file in your code is misssng in created dist folder. so you have to copy and paste dll file in dist folder, and it will work.
I used logimove.dll file and same issue. so i put logimove.dll file in dist file and it works. :)