I'm using a python script which uses subprocess to pass a commmand to the terminal. Part of the command that I'm passing involves paths which contain parentheses. Python handles strings with parentheses fine, but of course terminal does not handle these without escape characters.
I'm trying to pass variables to a command line program by feeding a string into subprocess, but here's simple example to reproduce the error:
import subprocess
path = '/home/user/Desktop/directory(2018)'
command_str = 'rmdir ' + path
print (subprocess.run(command_str))
which gives me this error:
Traceback (most recent call last):
File "ex.py", line 7, in <module>
print (subprocess.run(command_str))
File "/usr/lib/python3.6/subprocess.py", line 403, in run
with Popen(*popenargs, **kwargs) as process:
File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'rmdir /home/user/Desktop/directory(2018)': 'rmdir /home/user/Desktop/directory(2018)'
When I write it directly into the terminal with escape characters it works great.
$ rmdir /home/user/Desktop/directory\(2018\)
But in Python when I try to add escape characters to the strings before calling subprocess:
command_str = command_str.replace('(','\(')
command_str = command_str.replace(')','\)')
I get the same error as before because, unlike print, the subprocess string adds a second escape character which gets passed to the terminal.
Traceback (most recent call last):
File "ex.py", line 7, in <module>
print (subprocess.run(command_str))
File "/usr/lib/python3.6/subprocess.py", line 403, in run
with Popen(*popenargs, **kwargs) as process:
File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'rmdir /home/user/Desktop/directory\\(2018\\)': 'rmdir /home/user/Desktop/directory\\(2018\\)'
Is there a way to fix this particular error? Either by doing something different with replace or subprocess.run? (I'm not looking for a better way to remove directories.) Thanks.
Python implements rm and rmdir so no need to call a process. In general, if you want to skip shell processing on a command in subprocess, don't use the shell.
import subprocess
path = '/home/user/Desktop/directory(2018)'
command = ['rmdir', path]
print (subprocess.run(command, shell=False))
The shell breaks a command line into a list of arguments. You can build that list yourself and skip the shell completely.
Do not use subprocess, and you don't have to worry about shell escaping. Use the high-level file operation APIs provided in stdlib's shutil:
import shutil
shutil.rmtree('/home/user/Desktop/directory(2018)')
Related
I am trying to follow this blog on how to execute an R script from Python. I have the R script working fine from the command line using Rscript.
Here's my Python code:
import subprocess
import os
command = "C:\Program Files\R\R-3.4.4\bin\Rscript"
path2script = os.getcwd() + "\max.R" # gives me the absolute path to the R script
args = ["11", "3", "9", "42"]
cmd = [command, path2script] + args
x = subprocess.check_output(cmd, universal_newlines = True)
Which gives me this error:
FileNotFoundError: [WinError 2] The system cannot find the file specified
I've read a lot of SO posts on this error and in most cases it seems to be a problem with trying to invoke system commands like dir or passing arguments to check_output in the wrong order but in my case I really don't see what should be going wrong.
Following some of the advice I've tried building a string for cmd instead of a list, and then passing it to check_output using the argument shell = True - when I do that I get a CalledProcessError: returned non-zero exit status 1.
I'm assuming this code, which is exactly as it appeared on the blog other than adding the absolute path to the file, is failing now because the behaviour of check_output has changed since 2015...
Can anyone help?
Here's the stack trace:
Traceback (most recent call last):
File "<ipython-input-2-3a0151808726>", line 1, in <module>
runfile('C:/Users/TomWagstaff/Documents/Raising IT/Projects/15 AdWords/Python_R_test/run_max.py', wdir='C:/Users/TomWagstaff/Documents/Raising IT/Projects/15 AdWords/Python_R_test')
File "C:\Users\TomWagstaff\Anaconda3\envs\adwords\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
execfile(filename, namespace)
File "C:\Users\TomWagstaff\Anaconda3\envs\adwords\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/TomWagstaff/Documents/Raising IT/Projects/15 AdWords/Python_R_test/run_max.py", line 31, in <module>
x = subprocess.check_output(cmd, universal_newlines = True)
File "C:\Users\TomWagstaff\Anaconda3\envs\adwords\lib\subprocess.py", line 336, in check_output
**kwargs).stdout
File "C:\Users\TomWagstaff\Anaconda3\envs\adwords\lib\subprocess.py", line 403, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\Users\TomWagstaff\Anaconda3\envs\adwords\lib\site-packages\spyder\utils\site\sitecustomize.py", line 210, in __init__
super(SubprocessPopen, self).__init__(*args, **kwargs)
File "C:\Users\TomWagstaff\Anaconda3\envs\adwords\lib\subprocess.py", line 709, in __init__
restore_signals, start_new_session)
File "C:\Users\TomWagstaff\Anaconda3\envs\adwords\lib\subprocess.py", line 997, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
check that you have a right path for command and script
print(os.path.exists(command))
print(os.path.exists(path2script))
note that writing path with backslashes may be dangerous as you can create escape sequence that way which will be interpreted in different way. You can write windows paths with forward slashes and then call os.path.normpath on them, turning them into safe form
(also in command you can use forward slashes only, Python interpret doesn't really care. In path to your R script that would be probably problem though)
I made an attempt at parsing running programs in my computer (Debian OS) with the Subprocess python module. Here is my code:
import subprocess
cmd = "ps -A" # Unix command to get running processes
runningprox = subprocess.check_output(cmd) #returns output as byte string
rpstring = runningprox.decode("utf-8")
#converts byte string to string and puts it in a variable
print(rpstring)
However, when I run the code, I get the following error message:
Traceback (most recent call last): File "ratalert.py", line 6, in
runningprox = subprocess.check_output(cmd) #returns output as byte string File "/usr/local/lib/python3.6/subprocess.py", line 336, in
check_output
**kwargs).stdout File "/usr/local/lib/python3.6/subprocess.py", line 403, in run
with Popen(*popenargs, **kwargs) as process: File "/usr/local/lib/python3.6/subprocess.py", line 707, in init
restore_signals, start_new_session) File "/usr/local/lib/python3.6/subprocess.py", line 1333, in _execute_child
raise child_exception_type(errno_num, err_msg) FileNotFoundError: [Errno 2] No such file or directory: 'ps -A'
I don't understand why I get this error message. Considering 'ps -A' is neither a file nor a directory, but just the Unix command I put in a variable as a string.
How can I fix this? Thank you.
The subprocess.check_output function expects a list with the command and its arguments, like so:
runningprox = subprocess.check_output(['ps', '-A'])
Otherwise, it will treat the string you passed as a single command, and will look up for an executable file with name ps -A, space included and all.
You can use shlex to do the splitting as the shell would do:
import shlex, subprocess
cmd = 'ps -A'
runningprox = subprocess.check_output(shlex.split(cmd))
In my program I call the command:
command_two = 'sfit4Layer0.py -bv5 -fs'
subprocess.call(command_two.split(), shell=False)
I am using PyCharm and I get the error message:
Traceback (most recent call last):
File "part2test.py", line 5, in <module>
subprocess.call(command_two.split(), shell=False) #writes the summary file
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 522, in call
return Popen(*popenargs, **kwargs).wait()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
When walking through my program, it never gets to the program I want it to sfit4Layer0.py, it is getting stuck in subprocess but I am not sure why. Changing the shell=True doesn't do anything helpful either - I don't get these error messages but it does not execute my code properly. Any suggestions would be helpful.
My bash profile:
PATH="~/bin:/usr/bin:${PATH}"
export PATH PYTHONPATH="/Users/nataliekille/Documents/sfit4/pbin/Layer0:/Users/nataliekille/Documents/sfit4/pbin/Layer1:/Users/nataliekille/Documents/sfit4/pbin/ModLib:/Users/nataliekille/Documents/sfit4/SpectralDB"
export PYTHONPATH
PATH=${PATH}:${PYTHONPATH}
export PATH
You've missed an important part of the subprocess documentation. "If passing a single string [at the command, rather than a list of strings], either shell must be True (see below) or else the string must simply name the program to be executed without specifying any arguments."
So the kernel is compaining because there is not executable with the name 'sfit4Layer0.py -bv5 -fs'. Should work if you replace the string with (for example) 'sfit4Layer0.py -bv5 -fs'.split(), or ['sfit4Layer0.py', '-bv5', '-fs'].
When I try to shell out of my Python 3.51 program to run the Popen command I get the following errors. Yet when I copy the exact string I'm passing to Popen to the Terminal command line it works fine and opens the file in Adobe Reader which is my default app for the .pdf files.
Here is the Code:
finalCall = r'open /Users/gbarnabic/Documents/1111/combined.pdf'
print(finalCall)
pid_id = subprocess.Popen(finalCall).pid
Here is the error:
open /Users/gbarnabic/Documents/1111/combined.pdf
Exception in Tkinter callback
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/tkinter/init.py", line 1549, in call
return self.func(*args)
File "pdfcomb2.py", line 212, in change_dir
self.openPDF(outFileName, pageNum)
File "pdfcomb2.py", line 426, in openPDF
subprocess.run(finalCall)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/subprocess.py", line 696, in run
with Popen(*popenargs, **kwargs) as process:
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/subprocess.py", line 950, in init
restore_signals, start_new_session)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/subprocess.py", line 1544, in _execute_child
raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'open /Users/gb/Documents/1111/combined.pdf'
Georges-MBP:filepicktest gb$ open /Users/gb/Documents/1111/combined.pdf
Georges-MBP:filepicktest gb$
With Popen you need to set shell=True to pass command as a string or split command in a list of arguments. Could be done with shlex
import shlex
import subprocess
subprocess.Popen(shlex.split('open ....'))
You could check example in documentation:
https://docs.python.org/2/library/subprocess.html#subprocess.Popen
So the error here means that Python try to run file with name open /Users/gb/Documents/1111/combined.pdf. Obviously it doesn't exist
I am trying to access the wifi interface through python:
In bash I can use the following
/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport /usr/sbin/airport -I
-s can also be passed.
I have tried using the following in python:
from subprocess import call
call(['/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport /usr/sbin/airport', '-I'])
something is definitely not correct - as I get as a reply:
Traceback (most recent call last):
File "ip3.py", line 5, in <module>
call(['/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport /usr/sbin/airport', '-I'])
File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/subprocess.py", line 467, in call
return Popen(*popenargs, **kwargs).wait()
File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/subprocess.py", line 741, in __init__
restore_signals, start_new_session)
File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/subprocess.py", line 1356, in _execute_child
raise child_exception_type(errno_num, err_msg)
OSError: [Errno 2] No such file or directory: '/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport /usr/sbin/airport'
Any ideas would be welcome... I just want to begin by printing this to screen, saving as an array etc...
I dont have a high enough rating to answer my own question yet, so Ill say it here!
so I was being stupid!
from subprocess import call
call(['/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport', '-I'])
Works fine. Just needed to remove /usr/sbin/airport
call take first argument as command and subsequent arguments to that command.
In your case
command is,
/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport
and command's two arguments are,
/usr/sbin/airport
-I
So, you need to call it as,
from subprocess import call
call(['/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport' '/usr/sbin/airport', '-I'])
Try like this
from subprocess import call
call(['/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport', '/usr/sbin/airport', '-I'])
Otherwise it thinks /usr/sbin/airport is part of the first path.