I am somewhat new to Python, so I imagine this question has a simple answer. But I cannot seem to find a solution anywhere.
I have a Python script that continually accepts input from a streaming API and saves the data out to a file.
My problem when I need to stop the script to modify the code. If I use ctrl-f2, I sometime catch the script while it is in the process of writing to the output file, and the file ends up corrupted.
Is there a simple way to stop Python manually that allows it to finish executing the current line of code?
You can catch the SIGTERM or SIGINT signal and set a global variable that your script routinely checks to see if it should exit. It may mean you need to break your operations up into smaller chunks so that you can check the exit variable more frequently
import signal
EXIT = False
def handler(signum, frame):
global EXIT
EXIT = True
signal.signal(signal.SIGINT, handler)
def long_running_operation():
for i in range(1000000):
if EXIT:
# do cleanup or raise exception so that cleanup
# can be done higher up.
return
# Normal operation.
Related
While making an HTML help tool in Python, I wanted to exit the program smoothly, like clicking the X button on Google Chrome. But, I encountered an issue. It asks me if I want to kill the program, instead of doing it automatically.
I tried using quit(), exit() and sys.exit(). All do the same thing. How can I get the program to exit smoothly?
As it was suggested in the comments, your problem should only be noticed inside Python's IDLE, but should run just fine when executed inside a terminal. However, this code should also kill your program in IDLE:
import os, signal
from time import sleep
print("I will sleep for 3 secs and shut down!")
sleep(3)
os.kill(os.getpid(), signal.SIGTERM)
This sends a signal to your application to terminate.
Or alternatively you could call os' _exit function.
From the docs:
Exit the process with status n, without calling cleanup handlers, flushing stdio buffers, etc.
I know it's late but I took a different way.
It might be a dirty way of doing it. I don't like the accepted
answer because you have to hardcode those lines in every script.
I just comment out the lines, so that the dialog never shows on screen.
You can find the file /usr/lib/python3.9/idlelib/pyshell.py
aprx: line 1007
def close(self):
"Extend EditorWindow.close()"
#if self.executing:
#response = messagebox.askokcancel(
#"Kill?",
#"Your program is still running!\n Do you want to kill it?",
#default="ok",
#parent=self.text)
#if response is False:
#return "cancel"
self.stop_readline()
self.canceled = True
self.closing = True
return EditorWindow.close(self)
That way it will never ask you this silly question "Your program is still running!\n Do you want to kill it?"
So I have this part of code which does a simple thing : it launches a script and while the script is processing, a throbber is set on.
def go(self):
if ui.chk.isChecked():
self.startThrobber()
script = subprocess.check_call(r'"C:\Program Files\FME\fme.exe"', shell=False)
if script == 0:
self.stopThrobber() # opens a QMessageBox and stops throbber
else:
QMessageBox.information(self.popup(), "Errpr", "Error !")
After trying different methods (QThread, subprocess.Popen ...) this is the closest i got to make it work.
The only thing that doesn't work is that the throbber doesn't start right before the subprocess is executed, it starts after and thus it never stops.
So why is the throbber not ending when stopThrobber() is executed ?
And why is startThrobber not being executed before the subprocess (i'm pretty sure it's a subprocess thing, but i'm pretty new to all this, never heard about thread until yesterday)
EDIT :
The single quote was just a typing error, sorry. Still doesn't fix the problem.
Any call to a subprocess, from your main thread, that is blocking (waits for return) is going to stop your throbber from working properly. My answer to your other SO question on this topic outlines an approach that does not cause the subprocess call to block the main thread. I should point out that solution is not the only way to create a non-blocking call to a subprocess (for instance see here. You could create a QTimer to poll the subprocess poll() method periodically so that you can check the returncode to see if the subprocess has finished.)
The key theme is that you need your methods that run in the main thread to return quickly in order to keep the GUI responsive and allow your throbber to run/animate. So choose a way to launch the subprocess that meets this requirement.
Your single quotes denoting the raw string enclose the 'shell' argument.
def go(self):
if ui.chk.isChecked():
self.startThrobber()
script = subprocess.check_call(r"C:\Program Files\FME\fme.exe", shell=False)
if script == 0:
self.stopThrobber() # opens a QMessageBox and stops throbber
else:
QMessageBox.information(self.popup(), "Errpr", "Error !")
So I've tried another thing (unsuccessfully..)
When I click on a button, it executes startThrobber() and sends a signal to the following function :
def go(self):
self.startThrobber()
script = subprocess.Popen(r'"C:\Program Files\FME\fme.exe" ', shell=False)
while script.poll() == None:
time.sleep(1)
else:
p.stopThrobber()
But still doesn't working.. startThrobber is executed but nothing appears on the GUI... I thougth that the point of subprocess was to allow to do multiple tasks simultaneously so why isn't the throbber appearing ?
UPDATE : if i erase the while loop, startThrobber works : it appears while the subprocess is turning. So why when there is the while loop it doesn't work ?!
I am trying to write a python script that also deals killing/stopping its own process with the signals.
It runs each files one at a time, sleep at specific time and run again until it finished the whole directory with files. The processing time of each file is around 5 to 10 minutes depending on the size.
However, I want my program to stop when I give the signal. It should not kill it right away. It should run the current file and stop afterwards.
So I cannot use CTRL Z because it suspends the pid right away.
stop = False
def handler(number, frame):
global stop
stop = True
signal.signal(signal.SIGUSR1, handler)
while not stop:
# Do things
Above is what I tried, but it kills it right away when I signal. Also it goes into an infinite loop even after it finishes working on all the files.
What can I do to stop the process when I signal, allowing it to finish processing the current file first?
Just install a signal handler for signal.SIGTERM - and within it setup a state variable in your program that you check when finishing processing each file.
It is actually quite simple - see the documentation at: https://docs.python.org/2/library/signal.html .
import os
import signal
terminate = False
for filename in os.listdir("<your dir>"):
if terminate:
break
process_next_file(filename)
def handler(signum, frame):
global terminate
print("Termination requested")
terminate = True
signal.signal(signal.SIGTERM, handler)
(Also, you can use other signals - SIGINT is the one used when the user press ctrl+C for example)
You can create a command listener thread. Main thread still does file processing. For example, the listener thread waits a command from standard input. When you send "stop" command, it sets a varible. The file processor thread checks the variable before processing a file. So, it can stop when you want to stop processing.
Please don't consider it a duplicate before reading, There are a lot of questions about multithreading and keyboard interrupt, but i didn't find any considering os.system and it looks like it's important.
I have a python script which makes some external calls in worker threads.
I want it to exit if I press ctrl+c But it look like the main thread ignores it.
Something like this:
from threading import Thread
import sys
import os
def run(i):
while True:
os.system("sleep 10")
print i
def main():
threads=[]
try:
for i in range(0, 3):
threads.append(Thread(target=run, args=(i,)))
threads[i].daemon=True
threads[i].start()
for i in range(0, 3):
while True:
threads[i].join(10)
if not threads[i].isAlive():
break
except(KeyboardInterrupt, SystemExit):
sys.exit("Interrupted by ctrl+c\n")
if __name__ == '__main__':
main()
Surprisingly, it works fine if I change os.system("sleep 10") to time.sleep(10).
I'm not sure what operating system and shell you are using. I describe Mac OS X and Linux with zsh (bash/sh should act similar).
When you hit Ctrl+C, all programs running in the foreground in your current terminal receive the signal SIGINT. In your case it's your main python process and all processes spawned by os.system.
Processes spawned by os.system then terminate their execution. Usually when python script receives SIGINT, it raises KeyboardInterrupt exception, but your main process ignores SIGINT, because of os.system(). Python os.system() calls the Standard C function system(), that makes calling process ignore SIGINT (man Linux / man Mac OS X).
So neither of your python threads receives SIGINT, it's only children processes who get it.
When you remove os.system() call, your python process stops ignoring SIGINT, and you get KeyboardInterrupt.
You can replace os.system("sleep 10") with subprocess.call(["sleep", "10"]). subprocess.call() doesn't make your process ignore SIGINT.
I've had this same problem more times than I could count back when i was first learning python multithreading.
Adding the sleep call within the loop makes your main thread block, which will allow it to still hear and honor exceptions. What you want to do is utilize the Event class to set an event in your child threads that will serve as an exit flag to break execution upon. You can set this flag in your KeyboardInterrupt exception, just put the except clause for that in your main thread.
I'm not entirely certain what is going on with the different behaviors between the python specific sleep and the os called one, but the remedy I am offering should work for what your desired end result is. Just offering a guess, the os called one probably blocks the interpreter itself in a different way?
Keep in mind that generally in most situations where threads are required the main thread is going to keep executing something, in which case the "sleeping" in your simple example would be implied.
http://docs.python.org/2/library/threading.html#event-objects
I have a script that runs as a
while True:
doStuff()
What is the best way to communicate with this script if I need to stop it but I don't want to kill it if it is in the middle of an operation?
And I'm assuming you mean killing from outside the python script.
The way I've found easiest is
#atexit.register
def cleanup()
sys.unlink("myfile.%d" % os.getpid() )
f = open("myfile.%d" % os.getpid(), "w" )
f.write("Nothing")
f.close()
while os.path.exists("myfile.%d" % os.getpid() ):
doSomething()
Then to terminate the script just remove the myfile.xxx and the application should quit for you. You can use this even with multiple instances of the same script running at once if you only need to shut one down. And it tries to clean up after itself....
The best way is to rewrite the script so it doesn't use while True:.
Sadly, it's impossible to conjecture a good way to terminate this.
You could use the Linux signals.
You could use a timer and stop after a while.
You could have dostuff return a value and stop if the value is False.
You could check for a local file and stop if the file exists.
You could check an FTP site for a remote file and stop of the file exists.
You could check an HTTP web page for information that indicates if your loop should stop or not stop.
You could use OS-specific things like semaphores or shared memory.
I think the most elegant would be:
keep_running = true
while keep_running:
dostufF()
and then dostuff() can set keep_running = false whenever in no longer wants to keep running, then the while loop ends, and everything cleans up nicely.
If that's a console aplication and exiting by pressing Ctrl+C is ok, could that solve your problem?
try:
while True:
doStuff()
except KeyboardInterrupt:
doOtherStuff()
I guess the problem with that approach is that you wouldn't have any control exactly when and where in doStuff the execution is terminated.
Long time ago I've implemented such a thing. It catches Ctrl+C (or keyboard interrupt). It uses my package snuff-utils.
To install:
pip install snuff-utils
from snuff_utils.graceful_exit import graceful_exit
while True:
do_task_until_complete()
if graceful_exit:
do_stuff_before_exit()
break
On Ctrl+C it will log:
An interrupt signal has been received. The signal will be processed according to the logic of the application.
The goal I was after is to exit program but only after finishing already running task.
Be careful with multiprocessing/multithreading. It is not tested.
The signal module can trap signals and react accordingly?