I want to store into a variable the last output of a subprocess after the user performs a Keyboard Interrupt. My problem is mainly with a subprocess without end, i.e. tail in my exemple below. Here is my code:
class Testclass:
def Testdef(self):
try:
global out
print "Tail running"
tail_cmd='tail -f log.Reconnaissance'
proc = subprocess.Popen([tail_cmd], stdout=subprocess.PIPE, shell=True)
(out, err) = proc.communicate()
except KeyboardInterrupt:
print("KeyboardInterrupt received, stopping…")
finally:
print "program output:", out
if __name__ == "__main__":
app = Testclass()
app.Testdef()
Below is its output, which I don't understand at this moment.
Tail running
program output:
Traceback (most recent call last):
File "./2Test.py", line 19, in <module>
app.Testdef()
File "./2Test.py", line 15, in Testdef
print "program output:", out
NameError: global name 'out' is not defined
out not being defined indicates that the process proc.communicate() did not return any values, otherwise it would have populated your tuple (out, err). Now to find out whether the communicate() method was supposed to return or whether, more likely, your keyboard interrupt simply killed it, thus preventing out from being defined.
I assume you imported the subprocess module, but make sure you do that first. I rewrote your program without using global out or the try statements.
import subprocess
class Testclass:
def __init__(self,out): # allows you to pass in the value of out
self.out = out # makes out a member of this class
def Testdef(self):
print("Tail running")
tail_cmd='tail -f log.Reconnaissance'
proc = subprocess.Popen([tail_cmd], stdout=subprocess.PIPE, shell=True)
# Perhaps this is where you want to implement the try:
(self.out, err) = proc.communicate()
# and here the except:
# and here the finally:
if __name__ == "__main__":
app = Testclass(1) # pass 1 (or anything for testing) to the out variable
app.Testdef()
print('%r' % app.out) # print the contents of the out variable
# i get an empty string, ''
So as-is this program runs once. There is nothing in out. I believe to create a meaningful example of the user doing a keyboard interrupt, we need the program to be doing something that can be interrupted. Maybe I can provide an example in the future...
Related
I want code like this:
if True:
run('ABC.PY')
else:
if ScriptRunning('ABC.PY):
stop('ABC.PY')
run('ABC.PY'):
Basically, I want to run a file, let's say abc.py, and based on some conditions. I want to stop it, and run it again from another python script. Is it possible?
I am using Windows.
You can use python Popen objects for running processes in a child process
So run('ABC.PY') would be p = Popen("python 'ABC.PY'")
if ScriptRunning('ABC.PY) would be if p.poll() == None
stop('ABC.PY') would be p.kill()
This is a very basic example for what you are trying to achieve
Please checkout subprocess.Popen docs to fine tune your logic for running the script
import subprocess
import shlex
import time
def run(script):
scriptArgs = shlex.split(script)
commandArgs = ["python"]
commandArgs.extend(scriptArgs)
procHandle = subprocess.Popen(commandArgs, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
return procHandle
def isScriptRunning(procHandle):
return procHandle.poll() is None
def stopScript(procHandle):
procHandle.terminate()
time.sleep(5)
# Forcefully terminate the script
if isScriptRunning(procHandle):
procHandle.kill()
def getOutput(procHandle):
# stderr will be redirected to stdout due "stderr=subprocess.STDOUT" argument in Popen call
stdout, _ = procHandle.communicate()
returncode = procHandle.returncode
return returncode, stdout
def main():
procHandle = run("main.py --arg 123")
time.sleep(5)
isScriptRunning(procHandle)
stopScript(procHandle)
print getOutput(procHandle)
if __name__ == "__main__":
main()
One thing that you should be aware about is stdout=subprocess.PIPE.
If your python script has a very large output, the pipes may overflow causing your script to block until .communicate is called over the handle.
To avoid this, pass a file handle to stdout, like this
fileHandle = open("main_output.txt", "w")
subprocess.Popen(..., stdout=fileHandle)
In this way, the output of the python process will be dumped into the file.(You will have to modily the getOutput() function too for this)
import subprocess
process = None
def run_or_rerun(flag):
global process
if flag:
assert(process is None)
process = subprocess.Popen(['python', 'ABC.PY'])
process.wait() # must wait or caller will hang
else:
if process.poll() is None: # it is still running
process.terminate() # terminate process
process = subprocess.Popen(['python', 'ABC.PY']) # rerun
process.wait() # must wait or caller will hang
I am running a python progam from another python program through subprocess, I am calling it like this.
try:
subproc = subprocess.Popen(cmd, stderr=subprocess.PIPE, stdout=subprocess.PIPE,
shell=True)
o, e = subproc.communicate()
sys.stdout.write(o)
sys.stderr.write(e)
except:
subproc.terminate()
In the called program I have registered signal handlers like shown below. However, this never gets called on exception in the above program, despite calling the terminate function. But If I run the child program separately, the handle_exit function gets called fine. What mistake am I doing here?
def handle_exit(sig, frame):
print('\nClean up code here)
....
signal.signal(signal.SIGTERM, handle_exit)
signal.signal(signal.SIGINT, handle_exit)
UPDATE:
OK, I got it working by replacing subproc.terminate with the following.
subproc.send_signal(signal.SIGINT)
subproc.wait()
That is good, but I would also want to get the output of the child process on exception. How can I get that?
I've found the solution and here it is.
try:
subproc = subprocess.Popen(cmd, stderr=subprocess.PIPE, stdout=subprocess.PIPE,
shell=True)
o, e = subproc.communicate()
except:
subproc.send_signal(signal.SIGINT)
o, e = subproc.communicate()
sys.stdout.write(o)
sys.stderr.write(e)
How do I write a function that can start and kill a subrocess in python??
this is my code so far:
import subprocess
import signal
import time
def myfunction(action):
if action == 'start':
print 'Start subrocess'
process = subprocess.Popen("ping google.com", shell=True)
if action == 'stop':
print 'Stop subrocess'
process.send_signal(signal.SIGINT)
myfunction('start')
time.sleep(10)
myfunction('stop')
When I run this code I get this error:
Traceback (most recent call last):
File "test.py", line 15, in <module>
myfunction('stop')
File "test.py", line 11, in myfunction
process.send_signal(signal.SIGINT)
UnboundLocalError: local variable 'process' referenced before assignment
You need to save your subprocess variable and pass it into the function. when you call myfunction('stop'), there's nowhere for the function scope to get process from (thus the UnboundLocalError).
Without the function scope, this should work fine - which shows that your issue is with function scope and not really with process handling:
print 'Start subrocess'
process = subprocess.Popen("ping google.com", shell=True)
time.sleep(10)
print 'Stop subprocess'
process.send_signal(signal.SIGINT)
You need to learn OOP and define MyClass with constructor and destructor.
Assuming you do not need run many copies of process, and to make it more exotic we can use class methods
class MyClass(object):
#classmethod
def start(self)
print 'Start subrocess'
self.process = subprocess.Popen("ping google.com", shell=True)
#classmethod
def stop(self)
self.process.send_signal(signal.SIGINT)
MyClass.start()
MyClass.stop()
This not ideal as it allows you to create several new processes.
Quite often in such cases singleton pattern is used, that insures there is only one process is running yet it is a bit out of fashion.
The minimal fix (keeping myfunction) is to save process in a variable:
import subprocess
import signal
import time
def myfunction(action, process=None):
if action == 'start':
print 'Start subrocess'
process = subprocess.Popen("ping google.com", shell=True)
return process
if action == 'stop':
print 'Stop subrocess'
process.send_signal(signal.SIGINT)
process = myfunction('start')
time.sleep(10)
myfunction('stop', process)
It seems that the problem you are having is due to the fact that process is declared as a local variable within myfunction, and in particular only within the 'start' if statement. This small scope means that when you call myfunction('stop'), the function has no notion of the 'process' variable.
There are several ways around this, but the most intuitive would be for myfunction to return process when one was made, and take one as a parameter when you want to close it. The code would look something like:
import subprocess
import signal
import time
def myfunction(action, process=None):
if action == 'start':
print 'Start subrocess'
process = subprocess.Popen("ping google.com", shell=True)
return process
if action == 'stop':
print 'Stop subrocess'
process.send_signal(signal.SIGTERM)
process = myfunction('start')
time.sleep(10)
myfunction('stop', process)
I have just ran this in 2.7.13 and it works fine
I'm trying to invoke a command line utility from Python. The code is as follows
import subprocess
import sys
class Executor :
def executeEXE(self,executable ) :
CREATE_NO_WINDOW = 0x08000000
process = subprocess.Popen(executable, stdout=subprocess.PIPE,
creationflags=CREATE_NO_WINDOW )
while True:
line = process.stdout.readline()
if line == '' and process.poll() != None:
break
print line
The problem with above code is I want the real-time output of above process which I'm not getting. What I'm doing wrong here.
there are 2 problems in your code:
first of all, readline() will block untill when a new line is printed out and flushed.
That means you should execute the code
while True:
...
in a new Thread and call a callback function when the output is ready.
Since the readline is waiting for a new line, you must use
print 'Hello World'
sys.stdout.flush()
everytime in your executable.
You can see some code and example on my git:
pyCommunicator
Instead, if your external tool is buffered, the only thing you can try is to use stderr as PIPE:
https://stackoverflow.com/a/11902799/2054758
I'm working on application in wxPython which is a GUI for a command line utility. In the GUI there is a text control which should display the output from the application. I'm launching the shell command using subprocess, but I don't get any output from it until it has completed.
I have tried several solutions but none of them seems to work. Below is the code I'm using at the moment (updated):
def onOk(self,event):
self.getControl('infotxt').Clear()
try:
thread = threading.Thread(target=self.run)
thread.setDaemon(True)
thread.start()
except Exception:
print 'Error starting thread'
def run(self):
args = dict()
# creating a command to execute...
cmd = ["aplcorr", "-vvfile", args['vvfile'], "-navfile", args['navfile'], "-lev1file", args['lev1file'], "-dem", args['dem'], "-igmfile", args['outfile']]
proc = subprocess.Popen(' '.join(cmd), shell=True, stdout=subprocess.PIPE, stderr.subprocess.PIPE)
print
while True:
line = proc.stdout.readline()
wx.Yield()
if line.strip() == "":
pass
else:
print line.strip()
if not line: break
proc.wait()
class RedirectInfoText:
""" Class to redirect stdout text """
def __init__(self,wxTextCtrl):
self.out=wxTextCtrl
def write(self,string):
self.out.WriteText(string)
class RedirectErrorText:
""" Class to redirect stderr text """
def __init__(self,wxTextCtrl):
self.out.SetDefailtStyle(wx.TextAttr())
self.out=wxTextCtrl
def write(self,string):
self.out.SetDefaultStyle(wx.TextAttr(wx.RED))
self.out.WriteText(string)
In particular I'm going to need the output in real-time to create a progress-bar.
Edit: I changed my code, based on Mike Driscoll's suggestion. It seems to work sometimes, but most of the time I'm getting one of the following errors:
(python:7698): Gtk-CRITICAL **: gtk_text_layout_real_invalidate:
assertion `layout->wrap_loop_count == 0' failed
or
(python:7893): Gtk-WARNING **: Invalid text buffer iterator: either
the iterator is uninitialized, or the characters/pixbufs/widgets in
the buffer have been modified since the iterator was created. You must
use marks, character numbers, or line numbers to preserve a position
across buffer modifications. You can apply tags and insert marks
without invalidating your iterators, but any mutation that affects
'indexable' buffer contents (contents that can be referred to by
character offset) will invalidate all outstanding iterators
Segmentation fault (core dumped)
Any clues?
The problem is because you are trying to wx.Yield and to update the output widgets from the context of the thread running the process, instead of doing the update from the GUI thread.
Since you are running the process from a thread there should be no need to call wx.Yield, because you are not blocking the GUI thread, and so any pending UI events should be processed normally anyway.
Take a look at the wx.PyOnDemandOutputWindow class for an example of how to handle prints or other output that originate from a non-GUI thread.
This can be a little tricky, but I figured out one way to do it which I wrote about here: http://www.blog.pythonlibrary.org/2010/06/05/python-running-ping-traceroute-and-more/
After you have set up the redirection of the text, you just need to do something like this:
def pingIP(self, ip):
proc = subprocess.Popen("ping %s" % ip, shell=True,
stdout=subprocess.PIPE)
print
while True:
line = proc.stdout.readline()
wx.Yield()
if line.strip() == "":
pass
else:
print line.strip()
if not line: break
proc.wait()
The article shows how to redirect the text too. Hopefully that will help!