test1.py:
process = Popen(["python","test2.py"])
time.sleep(3)
alive = process.poll()
if alive is None:
print "Still running"
else:
print "Not running\r\n"
print "%r" % alive
test1.py Output:
Not running
2
test2.py:
time.sleep(30)
print "done"
What is going on? Shouldn't this return "Still running"?
Because of a contradicting result here's the full test1.py code:
import cStringIO
import os
import cgi
import time
from subprocess import Popen
def application(environ, start_response):
headers = []
headers.append(('Content-Type', 'text/plain'))
write = start_response('200 OK', headers)
input = environ['wsgi.input']
output = cStringIO.StringIO()
process = Popen(["python","test2.py"])
time.sleep(3)
alive = process.poll()
if alive is None:
print >> output, "Still running"
else:
print >> output, "Not running\r\n"
print >> output, "%r" % alive
output.write(input.read(int(environ.get('CONTENT_LENGTH', '0'))))
return [output.getvalue()]
Updated test1.py:
process = Popen(["python","C:/wamp/www/python/popen/test2.py"], shell=True)
time.sleep(5)
alive = process.poll()
if alive is None:
#print >> output, "%r" % alive
print >> output, "Still running"
else:
print >> output, "Not running"
print >> output, "%r" % alive
print >> output, "Current working dir : %s" % os.getcwd()
print >> output, os.strerror(0)
Updated Output:
Not running
0
Current working dir : C:\wamp\bin\apache\apache2.2.22
No error
If Popen() cannot find test2.py, it produces the error "No such file or directory", with errno 2. This error number is returned by poll(). Since you seem to be running this script through wsgi, something seems to be gulping your stderr and you don't see the error message:
$ cat test1.py
from subprocess import Popen
import time
process = Popen(["python","doesnotexist.py"])
time.sleep(3)
alive = process.poll()
if alive is None:
print "Still running"
else:
print "Not running\r\n"
print "%r" % alive
$ python test1.py
python: can't open file 'doesnotexist.py': [Errno 2] No such file or directory
Not running
2
Now, the issue is probably because your current working directory of your script is not set to the script's directory by the front end server, try printing os.getcwd() to see if it's what you're expecting.
According to this
An exit status of 2 indicates an issue with the shell executing the command. Have you tried running test2.py directly in the shell to verify there aren't issues with it? As Lie pointed out it could be that the shell can't find the file you're trying to execute, though there could be another issue causing it to break.
Related
I have written a Python code to generate a shell script and then run the script using subprocess.
The script file is created, but when I try to run from within the code, it is not doing anything. If I try to run the same script with a file that I have created outside the script, it is working as expected.
Here is my code :
import subprocess
import os
cwd = os.getcwd()
file_name = cwd + "/cmd_file_from_python"
fd = open(file_name,"w")
fd.write("#!/usr/local/bin/tcsh -f\n")
fd.write("echo 'PRINT FROM CMD_FILE_FROM_PYTHON'\n")
fd.close
os.chmod(file_name, 0o777)
cmd=file_name
p = subprocess.Popen(cmd,executable='/bin/ksh', shell=True, stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
(stdout,stderr) = p.communicate()
p_status = p.wait()
print "Command output : ", stdout
print "Command outerr : ", stderr
print "Command exit status/return code : ", p_status
print "================================================================"
file_name = cwd + "/cmd_file"
cmd = file_name
p = subprocess.Popen(cmd,executable='/bin/ksh', shell=True, stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
(stdout,stderr) = p.communicate()
p_status = p.wait()
print "Command output : ", stdout
print "Command outerr : ", stderr
print "Command exit status/return code : ", p_status
and the output :
Command output :
Command outerr :
Command exit status/return code : 0
================================================================
Command output : PRINT FROM CMD_FILE
Command outerr :
Command exit status/return code : 0
Here is the code of the script which I created outside the code:
$ cat cmd_file
#!/usr/local/bin/tcsh -f
echo 'PRINT FROM CMD_FILE'
IF I check both files, they only difference is the print :
$ diff cmd_file_from_python cmd_file
2c2
< echo 'PRINT FROM CMD_FILE_FROM_PYTHON'
---
> echo 'PRINT FROM CMD_FILE'
Your file is empty while the program is running:
fd = open(file_name,"w")
fd.write("#!/usr/local/bin/tcsh -f\n")
fd.write("echo 'PRINT FROM CMD_FILE_FROM_PYTHON'\n")
fd.close
Note the lack of call parens on fd.close; you never actually closed the file, so the entire contents of the file were likely sitting in Python's buffer, and never go to disk until the program ends (when the CPython reference interpreter, as an implementation detail, goes through and cleans up globals, closing the open files for you by side-effect; it might never reach disk in another interpreter).
To fix, actually call close. Or even better, switch to the much safer with statement approach, where the close is implicit, and automatic, occurring even if an exception or return causes you to exit the code early:
with open(file_name, "w") as fd:
fd.write("#!/usr/local/bin/tcsh -f\n")
fd.write("echo 'PRINT FROM CMD_FILE_FROM_PYTHON'\n")
# No need to call close; file is guaranteed closed when you exit with block
I am tring to handle status exit with popen but it gives a error, the code is:
import os
try:
res = os.popen("ping -c 4 www.google.com")
except IOError:
print "ISPerror: popen"
try:
#wait = [0,0]
wait = os.wait()
except IOError:
print "ISPerror:os.wait"
if wait[1] != 0:
print(" os.wait:exit status != 0\n")
else:
print ("os.wait:"+str(wait))
print("before read")
result = res.read()
print ("after read:")
print ("exiting")
But it if giving the following error:
close failed in file object destructor:
IOError: [Errno 10] No child processes
Error Explanation
It looks like this error is occurring because upon exiting, the program tries to destroy res, which involves calling the res.close() method. But somehow invoking os.wait() has already closed the object. So it's trying to close res twice, resulting in the error. If the call to os.wait() is removed, the error no longer occurs.
import os
try:
res = os.popen("ping -c 4 www.google.com")
except IOError:
print "ISPerror: popen"
print("before read")
result = res.read()
res.close() # explicitly close the object
print ("after read: {}".format(result)
print ("exiting")
But this leaves you with the problem of knowing when the process has finished. And since res just has type file, your options are limited. I would instead move to using subprocess.Popen
Using subprocess.Popen
To use subprocess.Popen, you pass your command in as a list of strings. To be able to access the output of the process, you set the stdout argument to subprocess.PIPE, which allows you to access stdout later on using file operations. Then, instead of using the regular os.wait() method, subprocess.Popen objects have their own wait methods you call directly on the object, this also sets the returncode value which represents the exit status.
import os
import subprocess
# list of strings representing the command
args = ['ping', '-c', '4', 'www.google.com']
try:
# stdout = subprocess.PIPE lets you redirect the output
res = subprocess.Popen(args, stdout=subprocess.PIPE)
except OSError:
print "error: popen"
exit(-1) # if the subprocess call failed, there's not much point in continuing
res.wait() # wait for process to finish; this also sets the returncode variable inside 'res'
if res.returncode != 0:
print(" os.wait:exit status != 0\n")
else:
print ("os.wait:({},{})".format(res.pid, res.returncode)
# access the output from stdout
result = res.stdout.read()
print ("after read: {}".format(result))
print ("exiting")
im faily new to python. need to understand more about subprocess.popen.
i have a script that executes another python script. below is the part where my script will try to execute another script.
cmd = ['python %s %s %s %s %s'%(runscript, steps, part_number, serial_number, self.operation)]
p = subprocess.Popen(cmd, shell = True, stdout=subprocess.PIPE)
p.wait()
result = p.stdout.readline()
the problem is, the script that gets executed, i have to print out the result in order to read results through "result = p.stdout.readline()". below is the script that gets executed
def Main():
if sys.argv[1] == "Initiate" :
doFunc = Functions_obj.Initiate()
if doFunc != 0 :
print doFunc
else :
print "Initiate PASS"
elif sys.argv[1] == "Check" :
getDrive = Functions_obj.initialize()
if getDrive == "NoDevice" :
print getDrive
sys.exit()
doFunc = Functions_obj.Identify_Drive()
if doFunc != 0 :
print doFunc
else :
print "Check PASS"
my question is, i want to "return" results from the script that gets executed and not print. how do i do this with subprocess.popen? and how do i use subprocess to get whats returned rather than whats printed
You can't return data through a pipe like a function call. You have to use one of the many IPC mechanisms like pipes, shared memory, message passing, or sockets. Pipes are generally the simplest, and that's what you're doing here. You can send binary data through the pipe, though. You could try pickling your data, assuming it's pickleable.
I have a simple Python script which will execute a shell script using subprocess mdoule in Python.
Below is my Python shell script which is calling testing.sh shell script and it works fine.
import os
import json
import subprocess
jsonData = '{"pp": [0,3,5,7,9], "sp": [1,2,4,6,8]}'
jj = json.loads(jsonData)
print jj['pp']
print jj['sp']
os.putenv( 'jj1', 'Hello World 1')
os.putenv( 'jj2', 'Hello World 2')
os.putenv( 'jj3', ' '.join( str(v) for v in jj['pp'] ) )
os.putenv( 'jj4', ' '.join( str(v) for v in jj['sp'] ) )
print "start"
subprocess.call(['./testing.sh'])
print "end"
And below is my shell script -
#!/bin/bash
for el1 in $jj3
do
echo "$el1"
done
for el2 in $jj4
do
echo "$el2"
done
for i in $( david ); do
echo item: $i
done
Now the question I have is -
if you see my Python script, I am printing start, then executing shell script and then printing end.. So suppose for whatever reason that shell script which I am executing has any problem, then I don't want to print out end.
So in the above example, shell script will not run properly as david is not a linux command so it will throw an error. So how should I see the status of entire bash shell script and then decide whether I need to print end or not?
I have just added a for loop example, it can be any shell script..
Is it possible to do?
You can check stderr of the bash script rather than return code.
proc = subprocess.Popen('testing.sh', stdout=subprocess.PIPE, stderr=subprocess.PIPE)
(stdout, stderr) = proc.communicate()
if stderr:
print "Shell script gave some error"
else:
print "end" # Shell script ran fine.
Just use the returned value from call():
import subprocess
rc = subprocess.call("true")
assert rc == 0 # zero exit status means success
rc = subprocess.call("false")
assert rc != 0 # non-zero means failure
You could use check_call() to raise an exception automatically if the command fails instead of checking the returned code manually:
rc = subprocess.check_call("true") # <-- no exception
assert rc == 0
try:
subprocess.check_call("false") # raises an exception
except subprocess.CalledProcessError as e:
assert e.returncode == 1
else:
assert 0, "never happens"
Well, according to the docs, .call will return the exit code back to you. You may want to check that you actually get an error return code, though. (I think the for loop will still return a 0 code since it more-or-less finished.)
I have a python script 'b.py' which prints out time ever 5 sec.
while (1):
print "Start : %s" % time.ctime()
time.sleep( 5 )
print "End : %s" % time.ctime()
time.sleep( 5 )
And in my a.py, I call b.py by:
def run_b():
print "Calling run b"
try:
cmd = ["./b.py"]
p = subprocess.Popen(cmd,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
for line in iter(p.stdout.readline, b''):
print (">>>" + line.rstrip())
except OSError as e:
print >>sys.stderr, "fcs Execution failed:", e
return None
and later on, I kill 'b.py' by:
PS_PATH = "/usr/bin/ps -efW"
def kill_b(program):
try:
cmd = shlex.split(PS_PATH)
retval = subprocess.check_output(cmd).rstrip()
for line in retval.splitlines():
if program in line:
print "line =" + line
pid = line.split(None)[1]
os.kill(int(pid), signal.SIGKILL)
except OSError as e:
print >>sys.stderr, "kill_all Execution failed:", e
except subprocess.CalledProcessError as e:
print >>sys.stderr, "kill_all Execution failed:", e
run_b()
time.sleep(600)
kill_b("b.py")
I have 2 questions.
1. why I don't see any prints out from 'b.py' and when I do 'ps -efW' I don't see a process named 'b.py'?
2. Why when I kill a process like above, I see 'permission declined'?
I am running above script on cygwin under windows.
Thank you.
Why I don't see any prints out from 'b.py' and when I do 'ps -efW' I don't see a process named 'b.py'?
Change run_b() lines:
p = subprocess.Popen(cmd,
stdout=sys.stdout,
stderr=sys.stderr)
You will not see a process named "b.py" but something like "python b.py" which is little different. You should use pid instead of name to find it (in your code "p.pid" has the pid).
Why when I kill a process like above, I see 'permission declined'?
os.kill is supported under Windows only 2.7+ and acts a little bit different than posix version. However you can use "p.pid". Best way to kill a process in a cross platform way is:
if platform.system() == "Windows":
subprocess.Popen("taskkill /F /T /PID %i" % p.pid, shell=True)
else:
os.killpg(p.pid, signal.SIGKILL)
killpg works also on OS X and other Unixy operating systems.