I am trying to use popen to kick off a subprocess that calls two commands (with multiple arguements) one after the other. The second command relies on the first command running, so I was hoping to use a single subprocess to run both rather than spawning two processes and wait on the first.
But I am running into issues because I am not sure how to give two command inputs or to seperate the command as one single object.
Also, I am trying to avoid setting shell to true if possible.
This is essentially, what I am trying to do:
for test in resources:
command = [
'pgh',
'resource',
'create',
'--name', test['name'],
'--description', test['description'],
]
command2 = [
'pgh',
'assignment',
'create',
'--name', test['name'],
'--user', test['user'],
]
p = Popen(command, stdout=PIPE, stderr=PIPE)
stdout, stderr = p.communicate()
print(stdout)
print(stderr)
As per my understanding the following should work for you.
To chain the execution once the previous completes use.
p1 = subprocess.Popen(command, stdout=subprocess.PIPE)
p2 = subprocess.Popen(command2, stdin=p1.stdout, stdout=subprocess.PIPE)
print p2.communicate()
You will have to launch command and wait for completion before launching another command. You should do this repeatedly for each command.
This can be done as
ps = [ Popen(c, stdout=PIPE, stderr=PIPE).communicate()
for c in command]
Note that this launches the next command irrespective of weather the first command succeeded or failed. If you want to launch the next command only if the previous command succeds then use
def check_execute(commands):
return_code = 0
for c in commands:
p = Popen(c, stdout=PIPE, stderr=PIPE)
result = p.communicate()
yield result
return_code = p.returncode
if return_code != 0:
break
Related
How to execute multiple commands to a command line program(.exe) one by one using python. I need to send commands and need to read the reply on the command line for each command. All in one session since login and settings has to be done.
I tried the below code(python 3.6) but it's not working
from subprocess import Popen, PIPE
process = Popen( "cmd.exe", shell=False, universal_newlines=True,
stdin=PIPE, stdout=PIPE, stderr=PIPE )
cmd= "\"C:\temp\demo.exe\"\n"
out, err = process.communicate(cmd)
print(out)
out, err = process.communicate( 'login\n' )
print(out)
this way you can execute multiple commands
import subprocess
command_list=["C:\\temp\\demo.exe","echo hello"]
for command in command_list:
proc = subprocess.Popen(command, shell=True,
stdin=subprocess.PIPE, stdout=subprocess.PIPE)
out, err = proc.communicate()
print("{} : {}".format(command,out.decode()))
Because shell=True is not the need for cmd.exe
I have a python (v3.3) script that runs other shell scripts. My python script also prints message like "About to run script X" and "Done running script X".
When I run my script I'm getting all the output of the shell scripts separate from my print statements. I see something like this:
All of script X's output
All of script Y's output
All of script Z's output
About to run script X
Done running script X
About to run script Y
Done running script Y
About to run script Z
Done running script Z
My code that runs the shell scripts looks like this:
print( "running command: " + cmnd )
ret_code = subprocess.call( cmnd, shell=True )
print( "done running command")
I wrote a basic test script and do *not* see this behaviour. This code does what I would expect:
print("calling")
ret_code = subprocess.call("/bin/ls -la", shell=True )
print("back")
Any idea on why the output is not interleaved?
Thanks. This works but has one limitation - you can't see any output until after the command completes. I found an answer from another question (here) that uses popen but also lets me see the output in real time. Here's what I ended up with this:
import subprocess
import sys
cmd = ['/media/sf_git/test-automation/src/SalesVision/mswm/shell_test.sh', '4', '2']
print('running command: "{0}"'.format(cmd)) # output the command.
# Here, we join the STDERR of the application with the STDOUT of the application.
process = subprocess.Popen(cmd, bufsize=1, universal_newlines=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
for line in iter(process.stdout.readline, ''):
line = line.replace('\n', '')
print(line)
sys.stdout.flush()
process.wait() # Wait for the underlying process to complete.
errcode = process.returncode # Harvest its returncode, if needed.
print( 'Script ended with return code of: ' + str(errcode) )
This uses Popen and allows me to see the progress of the called script.
It has to do with STDOUT and STDERR buffering. You should be using subprocess.Popen to redirect STDOUT and STDERR from your child process into your application. Then, as needed, output them. Example:
import subprocess
cmd = ['ls', '-la']
print('running command: "{0}"'.format(cmd)) # output the command.
# Here, we join the STDERR of the application with the STDOUT of the application.
process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
process.wait() # Wait for the underlying process to complete.
out, err = process.communicate() # Capture what it outputted on STDOUT and STDERR
errcode = process.returncode # Harvest its returncode, if needed.
print(out)
print('done running command')
Additionally, I wouldn't use shell = True unless it's really required. It forces subprocess to fire up a whole shell environment just to run a command. It's usually better to inject directly into the env parameter of Popen.
Have been trying to get something like this to work for a while, the below doesn't seem to be sending the correct arg to the c program arg_count, which outputs argc = 1. When I'm pretty sure I would like it to be 2. ./arg_count -arg from the shell outputs 2...
I have tried with another arg (so it would output 3 in the shell) and it still outputs 1 when calling via subprocess.
import subprocess
pipe = subprocess.Popen(["./args/Release/arg_count", "-arg"], shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = pipe.communicate()
result = out.decode()
print "Result : ",result
print "Error : ",err
Any idea where im falling over? I'm running linux btw.
From the documentation:
The shell argument (which defaults to False) specifies whether to use
the shell as the program to execute. If shell is True, it is
recommended to pass args as a string rather than as a sequence.
Thus,
pipe = subprocess.Popen("./args/Release/arg_count -arg", shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
should give you what you want.
If shell=True then your call is equivalent to:
from subprocess import Popen, PIPE
proc = Popen(['/bin/sh', '-c', "./args/Release/arg_count", "-arg"],
stdout=PIPE, stderr=PIPE)
i.e., -arg is passed to the shell itself and not your program. Drop shell=True to pass -arg to the program:
proc = Popen(["./args/Release/arg_count", "-arg"],
stdout=PIPE, stderr=PIPE)
If you don't need to capture stderr separately from stdout then you could use check_output():
from subprocess import check_output, STDOUT
output = check_output(["./args/Release/arg_count", "-arg"]) # or
output_and_errors = check_output(["./args/Release/arg_count", "-arg"],
stderr=STDOUT)
I'm trying to run a program from python and print the output based on its exit status. The code below is outputting directly to my vim screen (messing it) instead of opening a shell with the output.
python << EOF
import subprocess
import vim
cmd = "BAD_COMMAND"
p = subprocess.Popen(cmd, shell=True)
retcode = p.poll()
if retcode > 0:
output_of_error = p.communicate()[0]
vim.command("!echo show errors here")
EOF
You want to redirect your subprocess to a pipe:
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
communicate() will then return a 2-tuple containing the contents of stdout and stderr of the process.
There's a file named startup.cmd that sets some environment variables, runs some preparation commands, then does:
start "startup" cmd /k
Which opens a command shell named startup. The manual process I'm trying to automate is to then enter the following command into this shell: get startup.xml. I thought the correct way to do this in Python would be something like this:
import subprocess
p = subprocess.Popen('startup.cmd', shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
getcommand = 'get startup.xml'
servercommand = 'startserver'
p.stdin.write(getcommand)
p.stdin.write(startserver)
(stdoutdata, stderrdata) = p.communicate()
print stdoutdata
print stderrdata
But those commands don't seem to be executing in the shell. What am I missing? Also, the command shell appears regardless of whether shell is set to True or False.
I found this warning in subprocess's document,
Warning Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.
So my suggestion is to use communicate to send your command.
import subprocess
p = subprocess.Popen('startup.cmd', shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
command = 'get startup.xml\n'
command += 'startserver\n'
(stdoutdata, stderrdata) = p.communicate(command)
print stdoutdata
print stderrdata
This is a new process, so one cannot communicate directly with Popen.