Is there a proper way to create a script that loops through files in a folder and executes a subprocess that can be externally killed with Ctrl C? I have something like the following embedded in a pipeline and cannot Ctrl C it from the command line when the main process is killed.
Example script:
import subprocess
import os
import sys
input_directory = sys.argv[1]
for file in os.listdir(os.path.abspath(input_directory)):
output = file + "_out.out"
command = ['somescript.py', file, output]
try:
subprocess.check_call(command)
except:
print "Command Failed"
I would then execute program:
Example_script.py /path/to/some/directory/containing/files/
While it is looping, if I see the command failed, I want use Ctrl C. However, it fails and continues to run additional subprocesses despite the main script has been destroyer with Ctrl C. Is there a proper way to write something like this that can kill the childs (additional subprocess) with Ctrl C?
Any help, or pointing me in the direction is appreciated greatly. I am currently looking for a good method to do.
What you have in your try/except block is too permissive, such that when Ctrl+C is pressed, the KeyboardInterrupt exception is also handled by that same exception handler as the one that print "Command Failed", and as that is now properly handled there, the flow of the program is continued through the for loop. What you should do is:
Replace except: with except Exception: so that the KeyboardInterrupt exception will not be trapped, such that any time Ctrl+C is pressed the program will terminate (including subprocesses that isn't stuck in some non-terminatable state);
After the print statement, break out of the loop to prevent further execution from happening, if that is the intended behavior that you want this program to do.
You can catch KeyboardInterrupt, that way you can deal with Ctrl+C in whatever manner you want.
import subprocess
import os
import sys
input_directory = sys.argv[1]
for file in os.listdir(os.path.abspath(input_directory)):
output = file + "_out.out"
command = ['somescript.py', file, output]
try:
subprocess.check_call(command)
except KeyboardInterrupt as e:
print "Interrupted"
sys.exit(1)
except:
print "Command Failed"
However I agree with the other posters in that your exception is too vague, and you should be more specific in what can and can't fail.
I think Ctrl + Z can also help you to push the execution to background and suspended.
Related
Context:
I have a running python script. It contains many os.system("./executableNane") calls in a loop.
If I press ctrl + C, it just stops the execution of the current ./executableNane and passes to the next one.
Question:
How to stop the execution of the whole script and not only the execution of the current executable called?
Please note that I have read carefully the question/answer here but even with kill I can kill the executable executableNane but not the whole script (that I cannot find using top).
The only way I have to stop the script (without reboot the system) is to continue to press ctrl + C in a loop as well until all the tests are completed.
You can use subprocess and signal handlers to do this. You can also use subprocess to receive and send information via subprocess.PIPE, which you can read more about in the documentation.
The following should be a basic example of what you are looking to do:
import subprocess
import signal
import sys
def signal_handler(sig, frame):
print("You pressed Ctrl+C, stopping.")
print("Signal: {}".format(sig))
print("Frame: {}".format(frame))
sys.exit(123)
# Set up signal handler
signal.signal(signal.SIGINT, signal_handler)
print("Starting.")
while True:
cmd = ['sleep', '10']
p = subprocess.Popen(cmd)
p.wait()
if p.returncode != 0:
print("Command failed.")
else:
print("Command worked.")
The other solution to the question (by #Quillan Kaseman) is much more elegant compared with the solution I have found. All my problems are solved when I press Ctrl + Z instead of Ctrl + C.
Indeed, I have no idea why with Z works and with C does not. (I will try to look for some details later on).
I have created a watchdog timer for my script (Python 3), which allows me to halt execution if anything goes wrong (not shown in code below). However, I would like to have the ability to restart the script automatically using only Python (no external scripts). The code needs to be cross platform compatible.
I have tried subprocess and execv (os.execv(sys.executable, ['python'] + sys.argv)), however I am seeing very weird functionality on Windows. I open the command line, and run the script ("python myscript.py"). The script stops but does not exit (verified through Task Manager), and it will not restart itself unless I press enter twice. I would like it to work automatically.
Any suggestions? Thanks for your help!
import threading
import time
import subprocess
import os
import sys
if __name__ == '__main__':
print("Starting thread list: " + str(threading.enumerate()))
for _ in range(3):
time.sleep(1)
print("Sleeping")
''' Attempt 1 with subprocess.Popen '''
# child = subprocess.Popen(['python',__file__], shell=True)
''' Attempt 2 with os.execv '''
args = sys.argv[:]
args.insert(0, sys.executable)
if sys.platform == 'win32':
args = ['"%s"' % arg for arg in args]
os.execv(sys.executable, args)
sys.exit()
Sounds like you are using threading in your original script, which explains why your can't break your original script when simply pressing Ctrl+C. In that case, you might want to add a KeyboardInterrupt exception to your script, like this:
from time import sleep
def interrupt_this()
try:
while True:
sleep(0.02)
except KeyboardInterrupt as ex:
# handle all exit procedures and data cleaning
print("[*] Handling all exit procedures...")
After this, you should be able to automatically restart your relevant procedure (even from within the script itself, without any external scripts). Anyway, it's a bit hard to know without seeing the relevant script, so maybe I can be of more help if you share some of it.
I have a bit of python code to to try and make raw_input catch keyboard interrupts. If I run the code in this function it works perfectly fine. But if I run it in my program, the print statement is never made, indicating that the keyboard interrupt is not caught. The program attempts to exit and fails until it escalates to SIGKILL, which of course works fine. My guess is somewhere else the keyboard interrupt is being caught, preventing the exception from running at all. My question is, where would such an interrupt likely occur, and how can I prevent it from blocking this one. My plan has been to add a slight delay between the program catching a keyboard interrupt and killing itself to give excepting here a moment to catch.
Any ideas appreciated
Thanks!
import sys
def interruptable_input(text=''):
'''Takes raw input, but accepts keyboard interrupt'''
try:
return raw_input(text)
except KeyboardInterrupt:
print "Interrupted by user"
sys.exit()
I have narrowed it down to the following:
import sys
text=''
try:
print raw_input(text)
except KeyboardInterrupt:
print "Interrupted by user"
sys.exit()
Which works perfectly when i run it on the command line using python 2.7.
It lets me type an input on the console and when I hit ctrl+c it prints intterupted by user
Edit:
I misread your question at first, however when i use the method from your example and call it from another method the result is the same
I have determined the reason for my issue was another interrupt handler killing the script before the KeyboardInterrupt was hit. I solved it by setting my own interrupt handler for signal.SIGINT like so:
import sys
import signal
signal.signal(signal.SIGINT, signal_term_handler)
def signal_term_handler(signal, frame):
'''Handles KeyboardInterrupts to ensure smooth exit'''
rospy.logerr('User Keyboard interrupt')
sys.exit(0)
it's slightly less direct but it get's the job done. Now raw_input() will simply die when told to.
I have written a programme by python which is successfully tested under eclipse.Then I used pyinstaller to excute it as a .exe file. When the programme raise the exception ,the cmd window will quit immediately. I want to stay in this window to take a good look at this exception. How can I do it ? Thank you.
As Ms Turdy mentioned, you should run it in a command prompt or terminal first, if it will have the same behavior as the exe.
You can execute a python script with python -m pdb script.py and it will enter into the debugger. You run it by pressing C for continue, then it will break when it raises the exception.
That is because the python script has finished its job. You can do this:
import time
# your code
...
time.sleep(20)
This will give you 20 seconds to see the result. And after 20 s, the cmd window will remain 20 s for you to see the result. You can change the time for your requirement.
You can try raw_input to hold the screen:
import traceback
try:
# do something dangerous
except Exception, e:
print 'Error:', e
print traceback.format_exc()
raw_input('Input anything to end...')
I have the following python code:
os.system("C:/Python27/python.exe C:/GUI/TestGUI.py")
sys.exit(0)
It runs the command fine, and a window pops up. However, it doesn't exit the first script. It just stays there, and I eventually have to force kill the process. No errors are produced. What's going on?
instead of os.system use subprocess.Popen
this runs a command and doesn't wait for it and then exits:
import subprocess
import sys
subprocess.Popen(["mupdf", "/home/dan/Desktop/Sieve-JFP.pdf"])
sys.exit(0)
note that os.system(command) like:
p = subprocess.Popen(command)
p.wait()
KeyboardInterrupts and signals are only seen by the process (ie the main thread). If your nested command hangs due to some kind of file read or write block, you won't be able to quit the program using any keyboard commands.
Why does a read-only open of a named pipe block?
If you can't eliminate the source of the disk block, then one way is to wrap the process in the thread so you can force kill it. But if you do this, you leave opportunity for half-written and corrupted files on disk.
I suggest using os._exit instead of sys.exit, as sys.exit doesnt quit a program but raises exception level, or exits a thread. os._exit(-1) quits the entire program
import sys ,subprocess
subprocess.Popen(["C:/Python27/python.exe", "C:/GUI/TestGUI.py"])
sys.exit(0)
Popen from subprocess module what you are looking for.