I want to submit my long running Python job using ampersand. I'm going to kick this process off from an interactive Python program by using a sub process call it.
How would I keep track of the submitted job programmatically in case I want to end the job from a menu option?
Example of interactive program:
Main Menu
1. Submit long running job &
2. End long running job
If you're using python's subprocess module, you don't really need to background it again with & do you? You can just keep your Popen object around to track the job, and it will run while the other python process continues.
If your "outer" python process is going to terminate what sort of track do you need to keep? Would pgrep/pkill be suitable? Alternately, you could have the long running job log its PID, often under /var/run somewhere, and use that to track if the process is still alive and/or signal it.
You could use Unix signals. Here we capture SIGUSR1 to tell the process to communicate some info to STDOUT.
#!/usr/bin/env python
import signal
import sys
def signal_handler(signal, frame):
print('Caught SIGUSR1!')
print("Current job status is " + get_job_status())
signal.signal(signal.SIGUSR1, signal_handler)
and then from the shell
kill <pid> --signal SIGUSR1
Related
I am trying to start and later kill a process that requires sudo via a python-script. Even if the python script itself is run with sudo and kill() does not give any permission errors the process is not killed (and never receives SIGKILL).
Investigating this, i found out that Popen() returns the the process id of the sudo process, i assume at least, rather than the process i want to control. So when i correctly kill it later the underlying process keeps running. (Although if i kill the python program before killing the sudo process in python code the underlying process is also killed, so i guess there must be a way to do this manually, too).
I know it might be an option to use pgrep or pidof to search for the correct process, but as the processes name might not be unique it seems unnescessarly error prone (it might also occur that a process with the same name is started around the same time, so taking the latest one might not help).
Is there any solution to get reliably the pid of the underlying process started with sudo in python?
Using Python3.
My code for conducting the tests, taken slightly modified from https://stackoverflow.com/a/43417395/1171541:
import subprocess, time
cmd = ["sudo", "testscript.sh"]
def myfunction(action, process=None):
if action === "start":
process = subprocess.Popen(cmd)
return process
if action === "stop"
# kill() and send_signal(signal.SIGTERM) do not work either
process.terminate()
process = myfunction("start")
time.sleep(5)
myfunction("stop", process);
Okay, i can answer my own question here (which i found on https://izziswift.com/how-to-terminate-a-python-subprocess-launched-with-shelltrue/). The trick was to open the process with:
subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True, preexec_fn=os.setsid)
and then kill it:
os.killpg(os.getpgid(process.pid), signal.SIGTERM)
This time i use a shell to open and use the os to kill all the processes in the process group.
OS: Windows 10
Python: 3.5.2
I am trying to open calc.exe do some actions and than close it.
Here is my code sample
import subprocess, os, time
p = subprocess.Popen('calc.exe')
#Some actions
time.sleep(2)
p.kill()
So this is not working for calc.exe, it just opens the calculator, but does not close it, But same code is working fine for "notepad.exe".
I am guessing that there is a bug in subprocess lib for process kill method. so the notepad.exe process name in task manager is notepad.exe, but the calc.exe process name is calculator.exe, so I am guessing it is trying to kill by name and do not find it.
There's no bug in subprocess.kill. If you're really worried about that, just check the source, which is linked from the docs. The kill method just calls send_signal, which just calls os.kill unless the process is already done, and you can see the Windows implementation for that function. In short: subprocess.Process.kill doesn't care what name the process has in the kernel's process table (or the Task Manager); it remembers the PID (process ID) of the process it started, and kills it that way.
The most likely problem is that, like many Windows apps, calc.exe has some special "single instance" code: when you launch it, if there's already a copy of calc.exe running in your session, it just tells that copy to come to the foreground (and open a window, if it doesn't have one), and then exits. So, by the time you try to kill it 2 seconds later, the process has already exited.
And if the actual running process is calculator.exe, that means calc.exe is just a launcher for the real program, so it always tells calculator.exe to come to the foreground, launching it if necessary, and then exits.
So, how can you kill the new calculator you started? Well, you can't, because you didn't start a new one. You can kill all calc.exe and/or calculator.exe processes (the easiest way to do this is with a third-party library like psutil—see the examples on filtering and then kill the process once you've found it), but that will kill any existing calculator process you had open before running your program, not just the new one you started. Since calc.exe makes it impossible to tell if you've started a new process or not, there's really no way around that.
This is one way to kill it, but it will close every open calculator.
It calls a no window command prompt and gives the command to close the Calculator.exe process.
import subprocess, os, time
p = subprocess.Popen('calc.exe')
print(p)
#Some actions
time.sleep(2)
CREATE_NO_WINDOW = 0x08000000
subprocess.call('taskkill /F /IM Calculator.exe', creationflags=CREATE_NO_WINDOW)
I'm trying to build a todo manager in python where I want to continuously run a process in the bg that will alert the user with a popup when the specified time comes. I'm wondering how I can achieve that.
I've looked at some of the answers on StackOverflow and on other sites but none of them really helped.
So, What I want to achieve is to start a bg process once the user enters a task and keep on running it in the background until the time comes. At the same time there might be other threads running for other tasks as well that will end at their end times.
So far, I've tried this:
t = Thread(target=bg_runner, kwargs={'task': task, 'lock_file': lock_file_path})
t.setName("Get Done " + task.
t.start()
t.join()
With this the thread is continuosly running but it runs in the foreground and only exits when the execution is done.
If I add t.daemon = True in the above code, the main thread immediately exits after start() and it looks like the daemon is also getting killed then.
Please let me know how this can be solved.
I'm guessing that you just don't want to see the terminal window after you launch the script. In this case, it is a matter of how you execute the script.
Try these things.
If you are using a windows computer you can try using pythonw.exe:
pythonw.exe example_script.py
If you are using linux (maybe OSx) you may want to use 'nohup' in the terminal.
nohup python example_script.py
More or less the reason you have to do this comes down to how the Operating system handles processes. I am not an expert on this subject matter, but generally if you launch a script from a terminal, that script becomes a child process of the terminal. So if you exit that terminal, it will also terminate any child processes. The only way to get around that is to either detach the process from the terminal with something like nohup.
Now if you end up adding the #!/usr/bin/env python shebang line, your os could possibly just run the script without a terminal window if you just double click the script. YMMV (Again depends on how your OS works)
The first thing you need to do is prevent your script from exiting by adding a while loop in the main thread:
import time
from threading import Thread
t = Thread(target=bg_runner, kwargs={'task': task, 'lock_file': lock_file_path})
t.setName("Get Done " + task)
t.start()
t.join()
while True:
time.sleep(1.0)
Then you need to put it in the background:
$ nohup python alert_popup.py >> /dev/null 2>&1 &
You can get more information on controlling a background process at this answer.
Python 3.4
OS - Raspbian Jessie running on a Raspberry Pi 3
NOTE: The program "gnutv" does NOT have a stop command. It only has an option of a timer to stop the recording.
My question:
I'm still fairly new to programming and Python (self/YouTube/books taught). I am writing a program that checks a system for alarms. When an alarm is present, it triggers the program "gnutv" to begin recording video to a file. That was the easy part. I can make the program start, and record video using
Popen(["gnutv", "-out", "file", str(Videofile), str(Channel)])
The program continues to monitor the alarm inputs while the video is recording so it will know when to stop recording. BUT I can't get it to stop recording when the alarm is no longer present. I've attempted to use kill(), terminate(), and others without success (all returned errors indicating I don't know how to use these more complex commands). HOWEVER, I CAN kill the process by switching to the terminal and finding the PID using
pidof 'gnutv'
and then killing it with
kill PID#
So how can I return the PID value I get from the terminal so I can send the kill command to the terminal (again using Popen)?
i.e. - Popen(['kill', 'PID#'])
You don't need to run the kill program itself, you can just call .kill() or .terminate() on the Popen object.
import subprocess
proc = subprocess.Popen(['gnutv', '-out', 'file', str(Videofile), str(Channel)])
# Some time later...
# This is equivalent to running "kill <pid>"
proc.terminate()
# This is equivalent to running "kill -9 <pid>"
proc.kill()
If you really need the pid (hint: you don't) you can get it from the object as well, it's stored in the pid attribute.
print('Spawned gnutv (pid={})'.format(proc.pid))
You really should not be running the kill program, since that program is just a wrapper around the kill() function in the first place. Just call the function directly, or through the wrapper provided by subprocess.
thanks for helping!
I want to start and stop a Python script from a shell script. The start works fine, but I want to stop / terminate the Python script after 10 seconds. (it's a counter that keeps counting). bud is won't stop.... I think it is hanging on the first line.
What is the right way to start wait for 10 seconds en stop?
Shell script:
python /home/pi/count1.py
sleep 10
kill /home/pi/count1.py
It's not working yet. I get the point of doing the script on the background. That's working!. But I get another comment form my raspberry after doing:
python /home/pi/count1.py &
sleep 10; kill /home/pi/count1.py
/home/pi/sebastiaan.sh: line 19: kill: /home/pi/count1.py: arguments must be process or job IDs
It's got to be in the: (but what? Thanks for helping out!)
sleep 10; kill /home/pi/count1.py
You're right, the shell script "hangs" on the first line until the python script finishes. If it doesn't, the shell script won't continue. Therefore you have to use & at the end of the shell command to run it in the background. This way, the python script starts and the shell script continues.
The kill command doesn't take a path, it takes a process id. After all, you might run the same program several times, and then try to kill the first, or last one.
The bash shell supports the $! variable, which is the pid of the last background process.
Your current example script is wrong, because it doesn't run the python job and the sleep job in parallel. Without adornment, the script will wait for the python job to finish, then sleep 10 seconds, then kill.
What you probably want is something like:
python myscript.py & # <-- Note '&' to run in background
LASTPID=$! # Save $! in case you do other background-y stuff
sleep 10; kill $LASTPID # Sleep then kill to set timeout.
You can terminate any process from any other if OS let you do it. I.e. if it isn't some critical process belonging to the OS itself.
The command kill uses PID to kill the process, not the process's name or command.
Use pkill for that.
You can also, send it a different signal instead of SIGTERM (request to terminate a program) that you may wish to detect inside your Python application and respond to it.
For instance you may wish to check if the process is alive and get some data from it.
To do this, choose one of the users custom signals and register them within your Python program using signal module.
To see why your script hangs, see Austin's answer.