Unable to view a file using win32 dir command and Python subprocess - python

I'm attempting to read in a file and produce a hash of said file. For the most part, this has gone smoothly. However, there are a few files that I am unable to get a handle to. For instance:
file("c:\\windows\\system32\\nvvsvc.exe")
Produces the error: IOError: [Errno 2] No such file or directory: 'c:\\windows\\system32\\nvvsvc.exe
Similarly, the command:
subprocess.check_output(r"dir c:\windows\system32\nvvsvc.exe", shell = True)
Produces the error:
File Not Found
Traceback (most recent call last):
File "< stdin >", line 1, in < module >
File "C:\Python27\lib\subprocess.py", line 544, in check_output
raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command 'dir c:\windows\system32\nvvsvc.exe' returned non-zero exit status 1
I tried many variations thereof, including trying to even get cacls.exe to admit there's a file there. However, when I back out of the python instance (exit()) and perform a dir c:\windows\system32\nvvsvc.exe, it returns that there is indeed a file there.
A similar test using calc.exe instead allows me to view it and hash it both just fine from inside python.
I'm running Python 2.7. I've attempted kicking off python with administrative privs (run as administrator for the command prompt), but that doesn't change anything. There appears to be some change in privileges when running from inside python to the extent that the file system is essentially pretending that file doesn't exist to python.
For reference, the owner of the file in question (there are others that are similar), is system, and the cacls on the file allow reading of it from user, admin and system.

Simple answer! (but not necessarily easy to find) In regards to the IOError, it's because you don't need to escape backslashes.
The second part looks like a permissions error, as mentioned in the comments above.

If this is the 64-bit system try this:
subprocess.check_output(r"c:\windows\SysNative\nvvsvc.exe", shell = True)
The key here is "SysNative"

Related

Rendering a tree in python with Anytree and graphviz. Can't open file

This question is based on Tarun's Answer for rendering a tree in python using anytree and graphviz: https://stackoverflow.com/a/49442969/2139007
After installing graphviz and adding it to my PATH variables i'am trying to run the following python code:
DotExporter(nodes[0]["a"],
nodeattrfunc=lambda node: 'label="{}"'.format(node.display_name)).to_picture("tree.png")
The above code generates the following error:
Error: dot: can't open C:\Users\username\AppData\Local\Temp\tmpa7t554le
Traceback (most recent call last):
File "D:\mypath\tree.py", line 34, in <module>
dot.to_picture('tree.png')
File "C:\Users\username\AppData\Local\Programs\Python\Python36-32\lib\site-packages\anytree\exporter\dotexporter.py", line 229, in to_picture
check_call(cmd)
File "C:\Users\username\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 291, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['dot', 'C:\\Users\\username\\AppData\\Local\\Temp\\tmpa7t554le', '-T', 'png', '-o', 'tree.png']' returned non-zero exit status 2.
The problem does not occure if I run the dot command 'dot tree.dot -T png -o tree.png' manually from the console after generating a .dot file
DotExporter(nodes[0]["a"],
nodeattrfunc=lambda node: 'label="{}"'.format(node.display_name)).to_dotfile('tree.dot')
Is there a way to solve this?
I just ran into this problem. After 2 hours of debugging, I found out for Windows, you have to set the delete flag to False in dotexporter.py. Go to line 224 in "dotexporter.py" and make the following change. It seems like a bug in the source file. This worked for me:
with NamedTemporaryFile("wb", delete=False) as dotfile:
This is somehow an unintended behavior of windows and the python temporary file handle. Please just use the latest anytree version (https://anytree.readthedocs.io/en/latest/). It includes a fix.

os.execv returns OSError: [Errno 8] Exec format error

I am trying to have a Python script automatically download an updated version of itself, replace the existing version, then restart automatically so that it loads the new version.
I'm currently using the following code to restart it:
os.execv(__file__, sys.argv)
However, this isn't working. Whenever Python tries to run this line, it returns the following error:
Traceback (most recent call last):
File "N:\CardDB\Station\Read.py", line 195, in <module>
else:
File "N:\CardDB\Station\Read.py", line 187, in run_update
print("\n\nWould you like to install this update?")
File "N:\CardDB\Station\Read.py", line 144, in update
f.write(version)
OSError: [Errno 8] Exec format error
Other questions on StackOverflow suggest that it's due to a missing shebang line, but I've made sure that it isn't missing - the first line of my script is:
#!/usr/bin/env python3
Unlike unix OSes, Windows does not seem to have native support for interpreted executables. os.execv requires a binary and fails because it is given a text file. So instead of calling the script directly, call the python interpreter
os.execv(sys.executable, [sys.executable, __file__] + sys.argv)
On, e.g., Linux, you can use os.execv(__file__, [__file__] + sys.argv) if the script is marked executable and contains a shebang line. For nonexecutable scripts you have to call the python executable as above.

antiSMASH download_databases.py error in subprocess call

I am trying to intall antiSMASH on my research group's server but hitting an issue with the final stage of downloading the relevant databases. The makers provide a script "download_databases.py" to do this for you (see code at https://bitbucket.org/antismash/antismash/src/718da23d059742048bf044a1ed663806051eb0b2/download_databases.py?at=master&fileviewer=file-view-default).
Sadly there seems to be some kind of access issue (I'm not root and sudo doesn't help).
Server is RedHat CentOS 7.2.1511
Command run is "python download_databases.py" in antismash directory. Output is below:
Creating checksum of Pfam-A.hmm.gz
Extraction of Pfam-A.hmm.gz finished successfully.
Traceback (most recent call last):
File "download_databases.py", line 221, in <module>
main()
File "download_databases.py", line 198, in main
compile_pfam(filename)
File "download_databases.py", line 161, in compile_pfam
execute(command)
File "download_databases.py", line 51, in execute
stderr=subprocess.PIPE)
File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Does anyone with experience of this have an idea of what I need to do? I've tried a local install of Python (2.7) where I have read/write access but that doesn't seem to help as the script is still looking elsewhere for the subprocess script (my own install is at the top of the PATH). Do I possibly need to install something extra with the local install or customise the install locations to include the subprocess etc stuff?
Thanks in advance. I'm no expert and really appreciate the help.
EDIT: Well I feel quite stupid now. Thanks to Hannu for suggesting I check $PATH and what binary the script was trying to execute. It turned out to be one of antiSMASH's dependencies (hmmpress) which I hadn't yet added to the $PATH on this machine. It hadn't occurred to me that it would be required prior to actually running antiSMASH.
Add print commands or something like that to your execute function and see what it tries to execute. The only explanation to get the error in that command is that the script or binary it tries to execute cannot be found.
Try adding a full path to your binary. That alone might solve the issue. Check that there are no typos in the command. Remember commands needs to be a list if it contains parameters, for example ["/usr/bin/foo/myprogram", "-a", "42", "-b", "/tmp/outputfile"] or whatever it is you need to pass to the program.

Python Script can see the Y:/ Drive when run from IDLE but not the command line

I have a python script that builds a file by searching through folders and pulling in a list of files. This file runs fine and works as expected when I open and run it in IDLE, but if when I run the script in a commandline window I get this error:
C:\Windows\system32>python "C:\Users\ntreanor\Documents\RV Scripts\Server RV Sequence.py"
Traceback (most recent call last):
File "C:\Users\ntreanor\Documents\RV Scripts\Server RV Sequence.py", line 69,
in <module>
for foldername in os.listdir(pngFolders):
WindowsError: [Error 3] The system cannot find the path specified:
'Y:/20_temp_script_testing/pr126 movs\\04_comp_pngs/*.*'
In case it's not obvious, yes the path does exist. It not only works in IDLE but I double checked and the path definitely exists.
I also tried to create folders with a script that runs as a daemon and got a similar result
Traceback (most recent call last):
File "D:\shotgun\shotgunEventDaemon.py", line 888, in process
self._callback(self._shotgun, self._logger, event, self._args)
File "D:\shotgun\plugins\CreateAssetFolders.py", line 72, in createAssetFolders
os.makedirs(folder)
File "D:\Python27\Lib\os.py", line 150, in makedirs
makedirs(head, mode)
File "D:\Python27\Lib\os.py", line 150, in makedirs
makedirs(head, mode)
File "D:\Python27\Lib\os.py", line 150, in makedirs
makedirs(head, mode)
File "D:\Python27\Lib\os.py", line 157, in makedirs
mkdir(name, mode)
WindowsError: [Error 3] The system cannot find the path specified: 'Y:/'
This is what the script logged as a folder right before that:
Making folder:
Y:/07_design/04_environmental_elements\eec005-08_insect_ladybird_red_7_spots_wide
(the reason it's saying Y and not the whole path is that it attempts to make each folder back until it can't go back any further and that's when the exception is thrown)
Are the environment variables of the commandline window somehow affecting the drive mapping that should be pointing the script to the right location?
The issue is likely because IDLE and your command line are running with a different level of privileges. Mapped network drives are not automatically available to all user contexts. There is a superuser question on this here and plenty of other resources cover this topic. In short, the mapped network drive is only available to programs running at the level the mapping was made at.
If you have mapped the network drive through the windows UI, then it will be mapped for un-elevated programs. However if it was mapped with net use then it depends on the level of the command prompt when the mapping was made!
Disabling UAC will also affect change this behaviour, as will use of an elevated (or not) command prompt, which may explain why some PCs display different behaviour.
I think your problem is that you're trying to open the * file, which of course doesn't exist. open(path) takes path as a literal string and doesn't translate it in anyway, so it expects that value to be a valid filename. You should change your code to get a directory instead of a file, and then walking that directory.

Running python scripts with subprocess in windows. Python code checker wrappers from the emacswiki yield the same error

So i'm trying to setup the python code checkers suggested in the emacs wiki. However, I'm unable to run those scripts in my command shell let alone emacs.
The section is found here:
http://www.emacswiki.org/emacs/PythonProgrammingInEmacs#toc7
And I tried the script located here and here
In both cases I changed the first line from #!usr/bin python with the full path of my python executable and when I run the scripts via
python pylint_etc_wrappers.py someModule.py
or
python pycheckers.py soemModule.py
both boil down to the same error, most likely because they try to open a subprocess. Here's the trace:
Traceback (most recent call last):
File "pycheckers.py", line 254, in <module>
runner.run(source_file)
File "pycheckers.py", line 91, in run
process = Popen(args, stdout=PIPE, stderr=PIPE)
File "C:\devel\Python\Python-2.7\Lib\subprocess.py", line 672, in __init__
errread, errwrite)
File "C:\devel\Python\Python-2.7\Lib\subprocess.py", line 882, in _execute_child
startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
The second script suggests to change the first line to the path of the interpreter (which I did) and to change the path in the main function which looks something like :
os.environ['PATH'] = \
path.dirname(sys.executable) + ':' + os.environ['PATH']
which was a bit unclear to me. Any ideas?
I have pylint 0.25.1, installed using easy_install (Python 2.7, Win XP). Both pylint and pylint.bat were installed in Python27/Scripts (this directory is in my PATH).
I too get the "The system cannot find the file specified" error when running the pylint_etc_wrapper.py script unchanged.
Running pylint from the script does work if
command = 'pylint'
is changed to
command = 'pylint.bat'
Another way to make it work is to add shell=True to the Popen() call.
I can't really explain all this, but there is an unresolved Python bug that looks like it might be relevant: http://bugs.python.org/issue8557.

Categories