I'm running into a weird issue with subprocess.call() function. I am trying to execute Java's 'jar' command using subprocess.call(). Here's the code:
import os
import subprocess
def read_war():
war_file_path = "jackrabbit-webapp-2.6.5.war"
java_home = os.environ['JAVA_HOME']
jar_path = os.path.join(java_home, 'bin', 'jar')
jar_cmd = jar_path + ' tvf ' + war_file_path
print "command to be executed is : " + jar_cmd
subprocess.call(jar_cmd)
read_war()
I'm using Python v2.7.3 on both Windows and Linux (Oracle Enterprise Linux).
On Windows 7, I see the contents of the war file being displayed. On Linux, however, I see a 'no such file or directory' error.:
$ python example.py
command to be executed is : /usr/local/tools/jdk1.7.0_15/bin/jar tvf jackrabbit-webapp-2.6.5.war
Traceback (most recent call last):
File "example.py", line 24, in <module>
read_war()
File "example.py", line 23, in read_war
subprocess.call(jar_cmd)
File "/usr/local/tools/Python-2.7.3/Lib/subprocess.py", line 493, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/local/tools/Python-2.7.3/Lib/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/local/tools/Python-2.7.3/Lib/subprocess.py", line 1249, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
$
I've tried the command '/usr/local/tools/jdk1.7.0_15/bin/jar tvf jackrabbit-webapp-2.6.5.war' from command prompt and it works fine. So, nothing's wrong with the command.
I've tried various combinations of subprocess.call() - passing a string, passing a list etc. None of them worked. Any help at all would be appreciated.
Add shell=True to the call. On windows, the CreateProcess command does string parsing to separate the command and its various arguments. On linux, you only get string processing if you specifically tell subprocess to call the shell. Otherwise, it treats that entire string you handed in as the command and you don't get very far.
subprocess.call(jar_cmd, shell=True)
Use a list (sequence) argument instead of a string as the docs say:
args is required for all calls and should be a string, or a sequence
of program arguments. Providing a sequence of arguments is generally
preferred, as it allows the module to take care of any required
escaping and quoting of arguments (e.g. to permit spaces in file
names). If passing a single string, either shell must be True (see
below) or else the string must simply name the program to be executed
without specifying any arguments.
Example:
import os
import subprocess
def read_war():
war_file_path = "jackrabbit-webapp-2.6.5.war"
jar_path = os.path.join(os.environ['JAVA_HOME'], 'bin', 'jar')
jar_cmd = [jar_path, 'tvf', war_file_path]
print("command to be executed is: %s" % jar_cmd)
subprocess.check_call(jar_cmd)
read_war()
I've used check_call to raise an exception if the command returns non-zero exit status.
Related
I'm trying to pass a string parameter that has Korean characters. This causes an error, because Korean characters are apparently not properly encoded/decoded before it is passed to open() built-in function.
I wrote a command then executed it with os.system() which is equivalent to running it on the command prompt.
command = 'hwp5txt "C:\\Users\\username\\VSCodeProjects\\myproject\\data_files\\some_folder\\hwp\\2020-01-17_-_한국어가포함된 파일명(2020년도 제1차).hwp" > testdoc.txt'
os.system(command)
This throws an error because Korean characters are not properly decoded.
Traceback (most recent call last): File
"C:\Users\username\AppData\Local\pypoetry\Cache\virtualenvs\asiae-bok-nlp-xpMr0EW7-py3.7\Scripts\hwp5txt-script.py",
line 11, in
load_entry_point('pyhwp==0.1b12', 'console_scripts', 'hwp5txt')() File
"c:\users\username\appdata\local\pypoetry\cache\virtualenvs\asiae-bok-nlp-xpmr0ew7-py3.7\lib\site-packages\hwp5\hwp5txt.py",
line 102, in main
with closing(Hwp5File(hwp5path)) as hwp5file: File "c:\users\username\appdata\local\pypoetry\cache\virtualenvs\asiae-bok-nlp-xpmr0ew7-py3.7\lib\site-packages\hwp5\filestructure.py",
line 537, in init
stg = Hwp5FileBase(stg) File "c:\users\username\appdata\local\pypoetry\cache\virtualenvs\asiae-bok-nlp-xpmr0ew7-py3.7\lib\site-packages\hwp5\filestructure.py",
line 188, in init
stg = OleStorage(stg) File "c:\users\username\appdata\local\pypoetry\cache\virtualenvs\asiae-bok-nlp-xpmr0ew7-py3.7\lib\site-packages\hwp5\storage\ole.py",
line 35, in init
self.impl = impl_class(*args, **kwargs) File "c:\users\uesrname\appdata\local\pypoetry\cache\virtualenvs\asiae-bok-nlp-xpmr0ew7-py3.7\lib\site-packages\hwp5\plat\olefileio.py",
line 112, in init
if not isOleFile(olefile): File "c:\users\username\appdata\local\pypoetry\cache\virtualenvs\asiae-bok-nlp-xpmr0ew7-py3.7\lib\site-packages\olefile\olefile.py",
line 309, in isOleFile
with open(filename, 'rb') as fp: OSError: [Errno 22] Invalid argument:
'C:\Users\username\VSCodeProjects\asiae-BOK-nlp\data_files\BOK_minutes\hwp\2020-01-17_-_???????
???(2020?? ?1?).hwp'
As you can see, OS Error was raised because the command I sent to the prompt somehow didn't manage to pass the right Korean characters, which is now ????? instead of its proper name.
I tried it on the terminal manually but it also fails.
How do I pass string characters that is not properly passed to the module?
I'm using the latest version of VSCode with Git Bash terminal.
Also, I can check this information. If you need further information, please comment.
sys.stdout.encoding
>> 'UTF-8'
sys.stdin.encoding
>> 'cp1252'
sys.getfilesystemencoding
>> 'UTF-8'
Turned out, this wasn't Python's problem nor VSCode. It was just hwp5txt's issue where hwp5txt.exe won't digest Korean sys.argv. It works by trying:
$ hwp5txt-script.py 'C:\\...\\한국어가포함된파일.hwp'
However, one thing that bugs me is that this script would work on the terminal but not on Jupyter Lab or .py script.
i.e,
os.system(command) or subprocess.run(command, shell=True) won't run.
Instead, they will raise an error popup that says:
"This file does not have an app associated with it for performing this
action. Please install an app or, if one is already installed, create
an association in the Default Apps Settings page."
I am trying to run a program to make some system calls inside Python code using subprocess.call() which throws the following error:
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/lib/python2.7/subprocess.py", line 493, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
My actual Python code is as follows:
url = "/media/videos/3cf02324-43e5-4996-bbdf-6377df448ae4.mp4"
real_path = "/home/chanceapp/webapps/chanceapp/chanceapp"+url
fake_crop_path = "/home/chanceapp/webapps/chanceapp/chanceapp/fake1"+url
fake_rotate_path = "/home/chanceapp/webapps/chanceapp.chanceapp/fake2"+url
crop = "ffmpeg -i %s -vf "%(real_path)+"crop=400:400:0:0 "+ "-strict -2 %s"%(fake_crop_path)
rotate = "ffmpeg -i %s -vf "%(fake_crop_path)+"transpose=1 "+"%s"%(fake_rotate_path)
move_rotated = "mv"+" %s"%(fake_rotate_path)+" %s"%(real_path)
delete_cropped = "rm "+"%s"%(fake_crop_path)
#system calls:
subprocess.call(crop)
Can I get some relevant advice on how to solve this?
Use shell=True if you're passing a string to subprocess.call.
From docs:
If passing a single string, either shell must be True or
else the string must simply name the program to be executed without
specifying any arguments.
subprocess.call(crop, shell=True)
or:
import shlex
subprocess.call(shlex.split(crop))
No such file or directory can be also raised if you are trying to put a file argument to Popen with double-quotes.
For example:
call_args = ['mv', '"path/to/file with spaces.txt"', 'somewhere']
In this case, you need to remove double-quotes.
call_args = ['mv', 'path/to/file with spaces.txt', 'somewhere']
Can't upvote so I'll repost #jfs comment cause I think it should be more visible.
#AnneTheAgile: shell=True is not required. Moreover you should not use
it unless it is necessary (see # valid's comment). You should pass
each command-line argument as a separate list item instead e.g., use
['command', 'arg 1', 'arg 2'] instead of "command 'arg 1' 'arg 2'". –
jfs Mar 3 '15 at 10:02
I have some python code, from which I want to call another program. This program will
Print some output to STDOUT
Write a file to disk
Using call I get the following behavior;
from subprocess import call
call(['./tango_x86_64_release', 'VTS1 ct="N" nt="N" ph="7.2" te="303" io="0.02" seq="MKHPYEEFPTGSKSPYNMSRGAHPGAV"'])
34, File not properly written, try writing it up again,
1
This happens regardless if if the arguments are split into a list or not;
call(['./tango_x86_64_release', 'VTS1', 'ct="N"', 'nt="N"', 'ph="7.2"', 'te="303"', 'io="0.02"', 'seq="MKHPYEEFPTGSKSPYNMSRGAHPGAV"'])
34, File not properly written, try writing it up again,
1
I can call this same command from the my terminal
./tango_x86_64_release VTS1 ct="N" nt="N" ph="7.2" te="303" io="0.02" seq="MKHPYEEFPTGSKSPYNMSRGAHPGAV"
Which works and gives an exit status of 0.
It seems like its the writing to disk which is causing issues, if I break the command then I get the appropriate warning message (i.e. remove an argument, it warns me that the argument is missing).
Using subprocess.Popen() gives an OSError;
import subprocess as sub
output = sub.Popen('./tango_x86_64_release VTS1 ct="N" nt="N" ph="7.2" te="303" io="0.02" seq="MKHPYEEFPTGSKSPYNMSRGAHPGAV"', stdout=sub.PIPE, stderr=sub.PIPE)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/lib64/python2.7/subprocess.py", line 1249, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Any help greatly appreciated
Use shlex.split to split the command for you:
import shlex
call(shlex.split('./tango_x86_64_release VTS1 ct="N" nt="N" ph="7.2" te="303" io="0.02" seq="MKHPYEEFPTGSKSPYNMSRGAHPGAV"'))
Note that although you might be able to solve your problem by adding shell=True, you should avoid it if possible, since it can be a security risk (search for "shell injection").
Try to add shell=True to the Popen call.
Also see:
Why does subprocess.Popen() with shell=True work differently on Linux vs Windows?
Popen error: [Errno 2] No such file or directory
Documentation (and why Passing shell=True can be a security hazard)
I want to delete a number of jobs from a q. The command to delete the job is qdel JOBid.
Initially, I tried to use the subprocess module, but I got an error:
#!/usr/bin/env python
import sys, os, subprocess as sp
lo = sys.argv[1]
hi = sys.argv[2]
lo = int(lo)
hi = int(hi)
for i in range(lo,hi):
print "i is %d"%i
p=sp.Popen(['qdel %d'%i],stdout=sp.PIPE)
#os.system('qdel %d'%i)
So this did not work. The error I got was
Traceback (most recent call last):
File "del.py", line 14, in <module>
p=sp.Popen(['qdel %d'%i],stdout=sp.PIPE)
File "/usr/lib64/python2.6/subprocess.py", line 639, in __init__
errread, errwrite)
File "/usr/lib64/python2.6/subprocess.py", line 1228, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Then I commented out the subprocess line and used os and it worked immediately. I think I don't fully understand the subprocess module
#!/usr/bin/env python
import sys, os, subprocess as sp
lo = sys.argv[1]
hi = sys.argv[2]
lo = int(lo)
hi = int(hi)
for i in range(lo,hi):
print "i is %d"%i
#p=sp.Popen(['qdel %d'%i],stdout=sp.PIPE)
os.system('qdel %d'%i)
The above code worked flawlessly. I just want to know why and what the advantages are of the subprocess module. Also, I am using a unix shell
If you read manual, you can see that your call to Popen is wrong: you should pass not a single command, but an array of arguments:
p=sp.Popen(['qdel', '%d'%i],stdout=sp.PIPE)
Alternatively, as sc0tt's answer points out, you can use shell=True, but this has some disadvantages in more complex cases: you would have to manually escape all the variable data in the command in case it contains, for example, filenames with spaces or anything much more potentially harmful (like ;)
You want to use shell=True on your Popen call.
p=sp.Popen(['qdel %d'%i], shell=True, stdout=sp.PIPE)
I was also getting same issue. Using shell=True as one of the parameter solved my problem.
I am using the subprocess module to run a command in python. But the problem is that I also want to include a string (for a file name) in the command.
An example of what I want to do:
from subprocess import call
command = "cd/DirectoryName"
call = [(command)]
In this specific example I want DirectoryName to be a variable determined by the user.
What I have tried to no avail:
Desktop=raw_input()
cmd="'cd %s'(Desktop/)"
call([cmd])
Here's the error I get when I try to run these commands in the python shell.
Chicken='Chicken'
command = 'say %s' % (Chicken)
print command
say Chicken
call([command])
Traceback (most recent call last):
File "/Applications/WingIDE.app/Contents/MacOS/src/debug/tserver/_sandbox.py", line 1, in <module>
# Used internally for debug sandbox under external interpreter
File "/Library/Frameworks/EPD64.framework/Versions/7.2/lib/python2.7/subprocess.py", line 493, in call
return Popen(*popenargs, **kwargs).wait()
File "/Library/Frameworks/EPD64.framework/Versions/7.2/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/Library/Frameworks/EPD64.framework/Versions/7.2/lib/python2.7/subprocess.py", line 1228, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Just tried this and it made the shell crash.
Chicken="Chicken"
print Chicken
Chicken
call[("say %s" % (Chicken)]
That's not how string interpolation works.
cmd='cd %s' % (Desktop,)
First off,
cmd="'cd %s'(Desktop/)"
Doesn't seem like it would "printf" the %s.
Maybe
cmd="'cd %s/'%(Desktop)"
But I still don't know if that will interpolate since it's inside a string can using the "call" function and a python command -- wouldn't that call it on the command line?