When I run the fully_connected_feed.py code:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/tutorials/mnist/fully_connected_feed.py
I get an error:
Traceback (most recent call last):
File "C:/Users/AppData/Local/Continuum/Anaconda3/Lib/site-packages/tensorflow/examples/tutorials/mnist/fully_connected_feed.py", line 277, in <module>
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "C:\Users\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\platform\app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "C:/Users/AppData/Local/Continuum/Anaconda3/Lib/site-packages/tensorflow/examples/tutorials/mnist/fully_connected_feed.py", line 222, in main
run_training()
File "C:/Users/AppData/Local/Continuum/Anaconda3/Lib/site-packages/tensorflow/examples/tutorials/mnist/fully_connected_feed.py", line 120, in run_training
data_sets = input_data.read_data_sets(FLAGS.input_data_dir, FLAGS.fake_data)
File "C:\Users\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\contrib\learn\python\learn\datasets\mnist.py", line 211, in read_data_sets
SOURCE_URL + TRAIN_IMAGES)
File "C:\Users\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\contrib\learn\python\learn\datasets\base.py", line 142, in maybe_download
gfile.Copy(temp_file_name, filepath)
File "C:\Users\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 316, in copy
compat.as_bytes(oldpath), compat.as_bytes(newpath), overwrite, status)
File "C:\Users\AppData\Local\Continuum\Anaconda3\lib\contextlib.py", line 66, in __exit__
next(self.gen)
File "C:\Users\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 469, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.OutOfRangeError: Read fewer bytes than requested
How do I resolve this issue?
After doing the following, I was able to run the script without errors. The key for getting it to work for me, was the version of tensorflow installed has to match the tutorial code, otherwise there were exceptions. Although, I got a different exception than you, at first.
After installing tensorflow, check version. Details of this step may be different if you installed it pip or some other method:
$ conda list tensorflow
# packages in environment at /Users/agr/miniconda3/envs/tensorflow:
#
tensorflow 0.11.0 py35_0 conda-forge
Clone the git repo
$ git clone https://github.com/tensorflow/tensorflow.git
Inspect the tags available and checkout the release matching your install:
$ cd tensorflow
$ git tag -l -n1
...
$ git checkout v0.11.0
Run script!
$ cd examples/tutorials/mnist/
$ python fully_connected_feed.py
The key point being, run the script from here, not from the link you posted in the original question.
TL; DR
Something else is altering your files as you create them. Find the process and stop it.
Research
I've just run the demo with Windows 10, Python 3.5, tensorflow 0.12.0 with no errors. It is therefore something about your environment.
Looking at the actual line of the error, you are failing to read the required number of bytes from the open file. Going further up the stack you can see that CopyFile is actually trying to read all the bytes of a file into a string in this function. This starts by finding out the current file size and then trying to read all the bytes.
The problem is that the file size at the start of this process doesn't match the size by the end of the copy. In other words, something else has altered your file.
What next?
Your best bet is to try to find out what else is accessing your file. I suggest you use the techniques explained here to see what else has the file open as you are running the copy.
I encountered the same problem on Windows 2012 Server.
As suggested in the previous post, I downloaded and launched Process Monitor, then set the filter: "Path contains mnist" see the image. The datasets were downloaded and unpacked correctly, while running code both from Spyder and Jupyter.
I suspect that there is a race condition in the library code, i.e. missing synchronization between downloading and unpacking operations. As Process Monitor introduced additional delays, the datasets were sucessfully downloaded before the next operation started, hence the hazardous behavior was not observed.
Related
I installed openmdao via the documentation here (windows 10 plus anaconda): http://openmdao.org/twodocs/versions/latest/getting_started/getting_started.html
If I actually use the [all] flag it seems that pip tries to download every version of the packages so I just went with pip install openmdao
When I try to run the sample from the above link I get this error:
AttributeError: 'Problem' object has no attribute 'model'
I tried re-running in spyder with the same error then tried the first few lines just in the terminal to verify no model attribute existed.
I tried to skip further down into the code with terminal and got some more errors:
prob.driver=om.ScipyOptimizeDriver()
Traceback (most recent call last):
File "<ipython-input-6-8ea598efdab2>", line 1, in <module>
prob.driver=om.ScipyOptimizeDriver()
AttributeError: module 'openmdao.api' has no attribute 'ScipyOptimizeDriver'
I assumed that maybe there is a disconnect with the different versions floating around with openmdao so I then installed the latest non-dev version and tried to run a few included files in that master folder. All the examples I ran had the same error though:
runfile('C:/Users/Vicconius/Anaconda3/OpenMDAO1-master/examples/beam_tutorial.py', wdir='C:/Users/Vicconius/Anaconda3/OpenMDAO1-master/examples')
Traceback (most recent call last):
File "<ipython-input-7-7e855a208cb8>", line 1, in <module>
runfile('C:/Users/Vicconius/Anaconda3/OpenMDAO1-master/examples/beam_tutorial.py', wdir='C:/Users/Vicconius/Anaconda3/OpenMDAO1-master/examples')
File "C:\Users\Vicconius\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 710, in runfile
execfile(filename, namespace)
File "C:\Users\Vicconius\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 101, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/Vicconius/Anaconda3/OpenMDAO1-master/examples/beam_tutorial.py", line 218, in <module>
top.setup()
File "C:\Users\Vicconius\Anaconda3\OpenMDAO1-master\openmdao\core\problem.py", line 456, in setup
connections = self._setup_connections(params_dict, unknowns_dict)
File "C:\Users\Vicconius\Anaconda3\OpenMDAO1-master\openmdao\core\problem.py", line 234, in _setup_connections
for node in input_graph.nodes_iter():
AttributeError: 'DiGraph' object has no attribute 'nodes_iter'
Any ideas? THANK YOU!!
I can imagine two scenarios:
you happened to create an openmdao folder in your current local directory. You're opening the interpreter or otherwise running python scripts in that same directory. So when you try import openmdao it picks the local folder first --- instead of the installed package. If thats the case, either cd to a different directory or rename/remove that folder.
Something has gone wrong with your install. You somehow have multiple conflicting versions sitting around.
To test, first uninstall with pip:
pip uninstall OpenMDAO.
Then open an interpreter and try
import openmdao
If that somehow works, after you just uninstalled it, that confirms that you somehow have multiple installs.
you need to manually clean up your environment.
inside the interpreter, openmdao.__file__ should tell you where this wonky second install is hiding. You can go delete it manually, and then repeat this import test till you get an ImportError. At that point you know you have found all the rouge installs and you can safely retry a new pip install.
I am trying to train a pretrained "faster_rcnn_resnet101_kitti" model for the tensorflow object detection API.
But everytime I try to run
python3 train.py --logtostderr --train_dir='/training/' --pipeline_config_path='/training/faster_rcnn_resnet101_kitti.config'
I receive the following error
Traceback (most recent call last):
File "train.py", line 167, in <module>
tf.app.run()
File "/usr/local/lib/python3.5/dist- packages/tensorflow/python/platform/app.py", line 126, in run
_sys.exit(main(argv))
File "train.py", line 163, in main
worker_job_name, is_chief, FLAGS.train_dir)
File "/usr/local/lib/python3.5/dist-packages/object_detection-0.1-py3.5.egg/object_detection/trainer.py", line 211, in train
detection_model = create_model_fn()
File "/usr/local/lib/python3.5/dist-packages/object_detection-0.1-py3.5.egg/object_detection/builders/model_builder.py", line 96, in build
add_summaries)
File "/usr/local/lib/python3.5/dist-packages/object_detection-0.1-py3.5.egg/object_detection/builders/model_builder.py", line 272, in _build_faster_rcnn_model
frcnn_config.inplace_batchnorm_update)
AttributeError: 'FasterRcnn' object has no attribute 'inplace_batchnorm_update'
I had this error too, and for me it was because I had not re-compiled my .proto-files after I pulled the last updates from the TF models repository.
To recompile (on Linux):
# From tensorflow/models/research/ folder
protoc object_detection/protos/*.proto --python_out=.
I assume that the failing code tries to read the attribute/field inplace_batchnorm_update from the faster rcnn config, which (assumable) does not exist in the older versions. I hope this helps you too.
My versions are: tensorflow-gpu 1.7.0 and have the TF models commit hash 77d3bbefeb33e89bfa1eee707151e5d794d1222b with message "Merge pull request #3888 from hsm207/patch-3 Fix typo".
Recompiling on Windows
I know from own experience that, compared to Windows, compiling many files as above is easy in Linux as a one-liner. For Windows, here is something to make the process less cumbersome:
In this issue, davemers0160 has shared
a script for compiling on Windows.
Just save this file as a .bat-file:
#echo off
setlocal
echo Searching for new .proto files...
for %%F in (object_detection\protos\*.proto) do (
echo %%F
protoc %%F --python_out=.
)
echo Complete!
Run that file from the same folder as mentioned above. As the question was in Linux, I've just added this to the bottom in case a Windows user come along to read this too.
I had the same error after I updated the models repository.
I re-compiled .proto files, but it still has the error.
According to the log:
File "/home/duane/anaconda3/lib/python3.6/site-packages/object_detection-0.1-py3.6.egg/object_detection/builders/model_builder.py", line 164, in _build_ssd_model
inplace_batchnorm_update=ssd_config.inplace_batchnorm_update)
I think maybe it caused by the version of object_detection-0.1-py3.6.egg is too old , So I re-installed models/research/setup.py:
# Form /models/research/
python setup.py build
python setup.py install
Then it has no error.
NOTE: I did re-compile .proto files before I re-install setup.py.
The more details you can see #3968
Hope this can help you.
I am trying to intall antiSMASH on my research group's server but hitting an issue with the final stage of downloading the relevant databases. The makers provide a script "download_databases.py" to do this for you (see code at https://bitbucket.org/antismash/antismash/src/718da23d059742048bf044a1ed663806051eb0b2/download_databases.py?at=master&fileviewer=file-view-default).
Sadly there seems to be some kind of access issue (I'm not root and sudo doesn't help).
Server is RedHat CentOS 7.2.1511
Command run is "python download_databases.py" in antismash directory. Output is below:
Creating checksum of Pfam-A.hmm.gz
Extraction of Pfam-A.hmm.gz finished successfully.
Traceback (most recent call last):
File "download_databases.py", line 221, in <module>
main()
File "download_databases.py", line 198, in main
compile_pfam(filename)
File "download_databases.py", line 161, in compile_pfam
execute(command)
File "download_databases.py", line 51, in execute
stderr=subprocess.PIPE)
File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Does anyone with experience of this have an idea of what I need to do? I've tried a local install of Python (2.7) where I have read/write access but that doesn't seem to help as the script is still looking elsewhere for the subprocess script (my own install is at the top of the PATH). Do I possibly need to install something extra with the local install or customise the install locations to include the subprocess etc stuff?
Thanks in advance. I'm no expert and really appreciate the help.
EDIT: Well I feel quite stupid now. Thanks to Hannu for suggesting I check $PATH and what binary the script was trying to execute. It turned out to be one of antiSMASH's dependencies (hmmpress) which I hadn't yet added to the $PATH on this machine. It hadn't occurred to me that it would be required prior to actually running antiSMASH.
Add print commands or something like that to your execute function and see what it tries to execute. The only explanation to get the error in that command is that the script or binary it tries to execute cannot be found.
Try adding a full path to your binary. That alone might solve the issue. Check that there are no typos in the command. Remember commands needs to be a list if it contains parameters, for example ["/usr/bin/foo/myprogram", "-a", "42", "-b", "/tmp/outputfile"] or whatever it is you need to pass to the program.
I am attempting to use MoviePy with Python 3.2.3 on Raspian.
I have installed it (for Python 2.7, 3.2 and 3.5... long story) and the line
from moviepy.editor import *
works fine.
When I try
clip = VideoFileClip("vid.mov")
which is the most basic command, it gives the error
Traceback (most recent call last):
File "/home/pi/QuickFlicsPics/moviepytest.py", line 8, in <module>
clip = VideoFileClip("vid.mov")
File "/usr/local/lib/python3.2/distpackages/moviepy/video/io/VideoFileClip.py", line 55, in __init__
reader = FFMPEG_VideoReader(filename, pix_fmt=pix_fmt)
File "/usr/local/lib/python3.2/dist-packages/moviepy/video/io/ffmpeg_reader.py", line 32, in __init__
infos = ffmpeg_parse_infos(filename, print_infos, check_duration)
File "/usr/local/lib/python3.2/dist-packages/moviepy/video/io/ffmpeg_reader.py", line 237, in ffmpeg_parse_infos
proc = sp.Popen(cmd, **popen_params)
File "/usr/lib/python3.2/subprocess.py", line 745, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.2/subprocess.py", line 1371, in _execute_child
raise child_exception_type(errno_num, err_msg)
OSError: [Errno 8] Exec format error
I have researched this error, and it appears to be something to do with a shebang line missing somewhere. Is this correct, if so, how do I go about finding where it is missing, and what do I add?
Thanks
Edit:
As per cxw's comment, I installed moviepy using the command
pip-3.2 install moviepy
(I may have used 'sudo' as well)
FFMPEG was supposed to download automatically when I first used moviepy:
MoviePy depends on the software FFMPEG for video reading and writing. > You don’t need to worry about that, as FFMPEG should be automatically > downloaded/installed by ImageIO during your first use of MoviePy (it takes a few seconds). If you want to use a specific version of FFMPEG, follow the instructions in file config_defaults.py.
[Quote from installation guide here]
Manually download ffmpeg, then before running your Python code, do
export FFMPEG_BINARY=path/to/ffmpeg
at the shell/terminal prompt.
As far as I can tell from the source, the automatic download of ffmpeg does not know about Raspberry Pis. The auto-download code pulls from the imageio github repo, which only knows "linux32" vs. "linux64". It doesn't look like it has an ARM-linux option. When the ARM kernel sees a non-ARM image, it throws the error you see.
Rather than using the environment variable, you can edit your moviepy config-defaults.py file to specify FFMPEG_BINARY = r"/path/to/ffmpeg".
Edit to find the path/to/ffmpeg after installing it with apt-get, do
dpkg -L ffmpeg | grep bin
at the shell/terminal prompt. It will probably be in /bin or /usr/bin, and will probably be called ffmpeg or ffmpeg-x.xx (with some version number).
Thanks to this answer for dpkg
I don't have experience in downloading libraries, so any help is appreciated. I've got a fresh install of Python3.3 and am trying to get IPython for 64bit Windows 7. The IPython .exe installer ran fine and completed normally, but I can't access the program. I looked through their documentation, and tried the commands there, but
$ python setup.py install
returns invalid syntax, highlighting "setup". I thought the fix might be in the distribute library mentioned on the page, so I went to go try and get that first. However, using the script from distribute's install page revealed it's own errors, namely
Traceback (most recent call last):
File "C:\Python33\distribute_setup.py", line 541, in <module>
sys.exit(main())
File "C:\Python33\distribute_setup.py", line 537, in main
tarball = download_setuptools(download_base=options.download_base)
File "C:\Python33\distribute_setup.py", line 200, in download_setuptools
log.warn("Downloading %s", url)
File "C:\Python33\lib\distutils\log.py", line 47, in warn
self._log(WARN, msg, args)
File "C:\Python33\lib\distutils\log.py", line 30, in _log
if stream.errors == 'strict':
AttributeError: errors
I've been at it for a couple hours now, and I'm fresh out of ideas. What next?
It looks like you ran into Python bug 12967. You need to run distribute_setup.py outside of IDLE (i.e. in a command prompt).