Is there a way to pause a process (running from an executable) so that it stops the cpu load while it's paused, and waits till it's unpaused to go on with its work? Possibly in python, or in some way accessible by python.
By using psutil ( https://github.com/giampaolo/psutil ):
>>> import psutil
>>> somepid = 1023
>>> p = psutil.Process(somepid)
>>> p.suspend()
>>> p.resume()
you are thinking of SIGTSTP -- the same signal that happens when you push CTRL-Z. This suspends the process until it gets SIGCONT.
of course, some programs can just catch and ignore this signal, so it depends on the executable. however, if you can suspend and resume it manually, you can do it from a python program, too. use os.kill()
I just implemented this with signals in python something like this:
def mysignalhandler(sig, frame):
print "Got " + str(sig)
if sig == signal.SIGUSR1:
do_something()
signal.signal(signal.SIGUSR1, mysignalhandler)
signal.pause()
This will pause at the last line and call do_something() when it receives the signal USR1, for example through a
kill -USR1 <pid>
command.
This will only work in UNIX though.
There is a (almost) native way of doing this in Python, and it's quite simple :
import time
time.sleep(5)
In this snippet, 5 is the number of seconds you want to pause your program.
Related
How can I exit my entire Python application from one of its threads? sys.exit() only terminates the thread in which it is called, so that is no help.
I would not like to use an os.kill() solution, as this isn't very clean.
Short answer: use os._exit.
Long answer with example:
I yanked and slightly modified a simple threading example from a tutorial on DevShed:
import threading, sys, os
theVar = 1
class MyThread ( threading.Thread ):
def run ( self ):
global theVar
print 'This is thread ' + str ( theVar ) + ' speaking.'
print 'Hello and good bye.'
theVar = theVar + 1
if theVar == 4:
#sys.exit(1)
os._exit(1)
print '(done)'
for x in xrange ( 7 ):
MyThread().start()
If you keep sys.exit(1) commented out, the script will die after the third thread prints out. If you use sys.exit(1) and comment out os._exit(1), the third thread does not print (done), and the program runs through all seven threads.
os._exit "should normally only be used in the child process after a fork()" -- and a separate thread is close enough to that for your purpose. Also note that there are several enumerated values listed right after os._exit in that manual page, and you should prefer those as arguments to os._exit instead of simple numbers like I used in the example above.
If all your threads except the main ones are daemons, the best approach is generally thread.interrupt_main() -- any thread can use it to raise a KeyboardInterrupt in the main thread, which can normally lead to reasonably clean exit from the main thread (including finalizers in the main thread getting called, etc).
Of course, if this results in some non-daemon thread keeping the whole process alive, you need to followup with os._exit as Mark recommends -- but I'd see that as the last resort (kind of like a kill -9;-) because it terminates things quite brusquely (finalizers not run, including try/finally blocks, with blocks, atexit functions, etc).
Using thread.interrupt_main() may not help in some situation. KeyboardInterrupts are often used in command line applications to exit the current command or to clean the input line.
In addition, os._exit will kill the process immediately without running any finally blocks in your code, which may be dangerous (files and connections will not be closed for example).
The solution I've found is to register a signal handler in the main thread that raises a custom exception. Use the background thread to fire the signal.
import signal
import os
import threading
import time
class ExitCommand(Exception):
pass
def signal_handler(signal, frame):
raise ExitCommand()
def thread_job():
time.sleep(5)
os.kill(os.getpid(), signal.SIGUSR1)
signal.signal(signal.SIGUSR1, signal_handler)
threading.Thread(target=thread_job).start() # thread will fire in 5 seconds
try:
while True:
user_input = raw_input('Blocked by raw_input loop ')
# do something with 'user_input'
except ExitCommand:
pass
finally:
print('finally will still run')
Related questions:
Why does sys.exit() not exit when called inside a thread in Python?
Python: How to quit CLI when stuck in blocking raw_input?
The easiest way to exit the whole program is, we should terminate the program by using the process id (pid).
import os
import psutil
current_system_pid = os.getpid()
ThisSystem = psutil.Process(current_system_pid)
ThisSystem.terminate()
To install psutl:- "pip install psutil"
For Linux you can use the kill() command and pass the current process' ID and the SIGINT signal to start the steps to exit the app.
import signal
os.kill(os.getpid(), signal.SIGINT)
Please don't consider it a duplicate before reading, There are a lot of questions about multithreading and keyboard interrupt, but i didn't find any considering os.system and it looks like it's important.
I have a python script which makes some external calls in worker threads.
I want it to exit if I press ctrl+c But it look like the main thread ignores it.
Something like this:
from threading import Thread
import sys
import os
def run(i):
while True:
os.system("sleep 10")
print i
def main():
threads=[]
try:
for i in range(0, 3):
threads.append(Thread(target=run, args=(i,)))
threads[i].daemon=True
threads[i].start()
for i in range(0, 3):
while True:
threads[i].join(10)
if not threads[i].isAlive():
break
except(KeyboardInterrupt, SystemExit):
sys.exit("Interrupted by ctrl+c\n")
if __name__ == '__main__':
main()
Surprisingly, it works fine if I change os.system("sleep 10") to time.sleep(10).
I'm not sure what operating system and shell you are using. I describe Mac OS X and Linux with zsh (bash/sh should act similar).
When you hit Ctrl+C, all programs running in the foreground in your current terminal receive the signal SIGINT. In your case it's your main python process and all processes spawned by os.system.
Processes spawned by os.system then terminate their execution. Usually when python script receives SIGINT, it raises KeyboardInterrupt exception, but your main process ignores SIGINT, because of os.system(). Python os.system() calls the Standard C function system(), that makes calling process ignore SIGINT (man Linux / man Mac OS X).
So neither of your python threads receives SIGINT, it's only children processes who get it.
When you remove os.system() call, your python process stops ignoring SIGINT, and you get KeyboardInterrupt.
You can replace os.system("sleep 10") with subprocess.call(["sleep", "10"]). subprocess.call() doesn't make your process ignore SIGINT.
I've had this same problem more times than I could count back when i was first learning python multithreading.
Adding the sleep call within the loop makes your main thread block, which will allow it to still hear and honor exceptions. What you want to do is utilize the Event class to set an event in your child threads that will serve as an exit flag to break execution upon. You can set this flag in your KeyboardInterrupt exception, just put the except clause for that in your main thread.
I'm not entirely certain what is going on with the different behaviors between the python specific sleep and the os called one, but the remedy I am offering should work for what your desired end result is. Just offering a guess, the os called one probably blocks the interpreter itself in a different way?
Keep in mind that generally in most situations where threads are required the main thread is going to keep executing something, in which case the "sleeping" in your simple example would be implied.
http://docs.python.org/2/library/threading.html#event-objects
I'm trying to kill the notepad.exe process on windows using this function:
import thread, wmi, os
print 'CMD: Kill command called'
def kill():
c = wmi.WMI ()
Commands=['notepad.exe']
if Commands[0]!='All':
print 'CMD: Killing: ',Commands[0]
for process in c.Win32_Process ():
if process.Name==Commands[0]:
process.Terminate()
else:
print 'CMD: trying to kill all processes'
for process in c.Win32_Process ():
if process.executablepath!=inspect.getfile(inspect.currentframe()):
try:
process.Terminate()
except:
print 'CMD: Unable to kill: ',proc.name
kill() #Works
thread.start_new_thread( kill, () ) #Not working
It works like a charm when I'm calling the function like this:
kill()
But when running the function in a new thread it crashes and I have no idea why.
import thread, wmi, os
import pythoncom
print 'CMD: Kill command called'
def kill():
pythoncom.CoInitialize()
. . .
Running Windows functions in threads can be tricky since it often involves COM objects. Using pythoncom.CoInitialize() usually allows you do it. Also, you may want to take a look at the threading library. It's much easier to deal with than thread.
There are a couple of problems (EDIT: The second problem has been addressed since starting my answer, by "MikeHunter", so I will skip that):
Firstly, your program ends right after starting the thread, taking the thread with it. I will assume this is not a problem long-term because presumably this is going to be part of something bigger. To get around that, you can simulate something else keeping the program going by just adding a time.sleep() call at the end of the script with, say, 5 seconds as the sleep length.
This will allow the program to give us a useful error, which in your case is:
CMD: Kill command called
Unhandled exception in thread started by <function kill at 0x0223CF30>
Traceback (most recent call last):
File "killnotepad.py", line 4, in kill
c = wmi.WMI ()
File "C:\Python27\lib\site-packages\wmi.py", line 1293, in connect
raise x_wmi_uninitialised_thread ("WMI returned a syntax error: you're probably running inside a thread without first calling pythoncom.CoInitialize[Ex]")
wmi.x_wmi_uninitialised_thread: <x_wmi: WMI returned a syntax error: you're probably running inside a thread without first calling pythoncom.CoInitialize[Ex] (no underlying exception)>
As you can see, this reveals the real problem and leads us to the solution posted by MikeHunter.
An application that asks for a keyboard interrupt. How can I send for a keyboard interrupt programmatically? I need it for automation.
Like <C-c> or <C-x>
KeyboardInterrupt
Code running in a separate thread can cause a KeyboardInterrupt to be generated in the main thread by calling thread.interrupt_main().
See https://docs.python.org/2/library/thread.html#thread.interrupt_main
Since you mention automation I assume you want a SendKeys for Python. Try this: http://rutherfurd.net/python/sendkeys/
My suggestion to solve this problem is to use the following code pattern. I used it to programmatically start a tensorboard server and shut it down by sending a CTRL-C when the object it belongs to is deleted. Generally speaking, this should work for any example that provokes a subprocess that is supposed to be send a KeyBoardInterrupt:
Import signal and subprocess
import signal
import subprocess
Create the subprocess using subprocess.Popen. Important: set the creationflags parameter to subprocess.CREATE_NEW_PROCESS_GROUP. This is necessary to later be able to send the KeyboardInterrupt event.
command= <enter your command that is supposed to be run in different process as a string>
process = subprocess.Popen(command.split(),stdout=subprocess.PIPE,stdin=subprocess.PIPE,creationflags=subprocess.CREATE_NEW_PROCESS_GROUP)
Wherever you want to send the keyboardInterrupt, do the following:
process.send_signal(signal.CTRL_C_EVENT)
That is it! Please see the the official subprocess documentation for insights on why the creationflags parameter of popen has to be set that way.
This is how the code looks for my example in a less generic way:
import signal
import subprocess
import time
class ExperimentTracker():
def __init__(self):
self.tensorboard_process=None
def __del__(self):
#shutdown tensorboard server and terminate process
self.tensorboard_process.send_signal(signal.CTRL_C_EVENT)
time.sleep(0.2)
self.tensorboard_process.kill()
def launch_tensorboard(self):
#launch tensorboard
bashCommand= "tensorboard --logdir runs"
self.tensorboard_process = subprocess.Popen(bashCommand.split(),stdout=subprocess.PIPE,stdin=subprocess.PIPE,creationflags=subprocess.CREATE_NEW_PROCESS_GROUP)
time.sleep(2) #sleep for 2 seconds to give tensorboard time to be launched
If say you want to run a program via shell ./program. In Linux what you could do is:
# Made a function to handle more complex programs which require multiple inputs.
run(){
./program
}
# Making a child process that will run the program while you stop it later.
run &
childPid=($!)
# Process id of the program which you want to interrupt (via `run`).
# Time after which you want to interrupt
sleep 5
# Actual command to send the interrupt
kill -SIGINT $childPid
Let me know if it works in windows as well.
I have a script that runs as a
while True:
doStuff()
What is the best way to communicate with this script if I need to stop it but I don't want to kill it if it is in the middle of an operation?
And I'm assuming you mean killing from outside the python script.
The way I've found easiest is
#atexit.register
def cleanup()
sys.unlink("myfile.%d" % os.getpid() )
f = open("myfile.%d" % os.getpid(), "w" )
f.write("Nothing")
f.close()
while os.path.exists("myfile.%d" % os.getpid() ):
doSomething()
Then to terminate the script just remove the myfile.xxx and the application should quit for you. You can use this even with multiple instances of the same script running at once if you only need to shut one down. And it tries to clean up after itself....
The best way is to rewrite the script so it doesn't use while True:.
Sadly, it's impossible to conjecture a good way to terminate this.
You could use the Linux signals.
You could use a timer and stop after a while.
You could have dostuff return a value and stop if the value is False.
You could check for a local file and stop if the file exists.
You could check an FTP site for a remote file and stop of the file exists.
You could check an HTTP web page for information that indicates if your loop should stop or not stop.
You could use OS-specific things like semaphores or shared memory.
I think the most elegant would be:
keep_running = true
while keep_running:
dostufF()
and then dostuff() can set keep_running = false whenever in no longer wants to keep running, then the while loop ends, and everything cleans up nicely.
If that's a console aplication and exiting by pressing Ctrl+C is ok, could that solve your problem?
try:
while True:
doStuff()
except KeyboardInterrupt:
doOtherStuff()
I guess the problem with that approach is that you wouldn't have any control exactly when and where in doStuff the execution is terminated.
Long time ago I've implemented such a thing. It catches Ctrl+C (or keyboard interrupt). It uses my package snuff-utils.
To install:
pip install snuff-utils
from snuff_utils.graceful_exit import graceful_exit
while True:
do_task_until_complete()
if graceful_exit:
do_stuff_before_exit()
break
On Ctrl+C it will log:
An interrupt signal has been received. The signal will be processed according to the logic of the application.
The goal I was after is to exit program but only after finishing already running task.
Be careful with multiprocessing/multithreading. It is not tested.
The signal module can trap signals and react accordingly?