I am working on a script(let's call it p1.py) which calls another python script(let's call it p2.py). This p2.py is called with subprocess run(cmd, shell=True) and at some certain point, p2.py opens up a window that needs to be canceled with ESC. In the meantime, p1.py is just waiting for p2.py to be over. Unfortunately, this window is not avoidable coming from the library which I don't want to touch for compatibility. The question is that any ways to avoid this window or anyways to close it programmatically?
Parameters:
Language: Python 3.7
OS: Ubuntu/Linux
You can use subprocess.Popen('python py2.py') instead of subprocess.run().
In py1.py this will return immediately and give you an object to kill the process of py2.py later. However, you need to know how long you have to wait or get some other signal for py1.py when py2.py is done and the window is showing.
Try something like this in py1.py:
import subprocess
import signal
import time
p = subprocess.Popen('python py2.py')
time.sleep(...) # wait until py2.py is done
p.send_signal(signal.SIGTERM)
Note: This does not work with the shell=True parameter.
I'm trying to port a shell script to the much more readable python version. The original shell script starts several processes (utilities, monitors, etc.) in the background with "&". How can I achieve the same effect in python? I'd like these processes not to die when the python scripts complete. I am sure it's related to the concept of a daemon somehow, but I couldn't find how to do this easily.
While jkp's solution works, the newer way of doing things (and the way the documentation recommends) is to use the subprocess module. For simple commands its equivalent, but it offers more options if you want to do something complicated.
Example for your case:
import subprocess
subprocess.Popen(["rm","-r","some.file"])
This will run rm -r some.file in the background. Note that calling .communicate() on the object returned from Popen will block until it completes, so don't do that if you want it to run in the background:
import subprocess
ls_output=subprocess.Popen(["sleep", "30"])
ls_output.communicate() # Will block for 30 seconds
See the documentation here.
Also, a point of clarification: "Background" as you use it here is purely a shell concept; technically, what you mean is that you want to spawn a process without blocking while you wait for it to complete. However, I've used "background" here to refer to shell-background-like behavior.
Note: This answer is less current than it was when posted in 2009. Using the subprocess module shown in other answers is now recommended in the docs
(Note that the subprocess module provides more powerful facilities for spawning new processes and retrieving their results; using that module is preferable to using these functions.)
If you want your process to start in the background you can either use system() and call it in the same way your shell script did, or you can spawn it:
import os
os.spawnl(os.P_DETACH, 'some_long_running_command')
(or, alternatively, you may try the less portable os.P_NOWAIT flag).
See the documentation here.
You probably want the answer to "How to call an external command in Python".
The simplest approach is to use the os.system function, e.g.:
import os
os.system("some_command &")
Basically, whatever you pass to the system function will be executed the same as if you'd passed it to the shell in a script.
I found this here:
On windows (win xp), the parent process will not finish until the longtask.py has finished its work. It is not what you want in CGI-script. The problem is not specific to Python, in PHP community the problems are the same.
The solution is to pass DETACHED_PROCESS Process Creation Flag to the underlying CreateProcess function in win API. If you happen to have installed pywin32 you can import the flag from the win32process module, otherwise you should define it yourself:
DETACHED_PROCESS = 0x00000008
pid = subprocess.Popen([sys.executable, "longtask.py"],
creationflags=DETACHED_PROCESS).pid
Use subprocess.Popen() with the close_fds=True parameter, which will allow the spawned subprocess to be detached from the Python process itself and continue running even after Python exits.
https://gist.github.com/yinjimmy/d6ad0742d03d54518e9f
import os, time, sys, subprocess
if len(sys.argv) == 2:
time.sleep(5)
print 'track end'
if sys.platform == 'darwin':
subprocess.Popen(['say', 'hello'])
else:
print 'main begin'
subprocess.Popen(['python', os.path.realpath(__file__), '0'], close_fds=True)
print 'main end'
Both capture output and run on background with threading
As mentioned on this answer, if you capture the output with stdout= and then try to read(), then the process blocks.
However, there are cases where you need this. For example, I wanted to launch two processes that talk over a port between them, and save their stdout to a log file and stdout.
The threading module allows us to do that.
First, have a look at how to do the output redirection part alone in this question: Python Popen: Write to stdout AND log file simultaneously
Then:
main.py
#!/usr/bin/env python3
import os
import subprocess
import sys
import threading
def output_reader(proc, file):
while True:
byte = proc.stdout.read(1)
if byte:
sys.stdout.buffer.write(byte)
sys.stdout.flush()
file.buffer.write(byte)
else:
break
with subprocess.Popen(['./sleep.py', '0'], stdout=subprocess.PIPE, stderr=subprocess.PIPE) as proc1, \
subprocess.Popen(['./sleep.py', '10'], stdout=subprocess.PIPE, stderr=subprocess.PIPE) as proc2, \
open('log1.log', 'w') as file1, \
open('log2.log', 'w') as file2:
t1 = threading.Thread(target=output_reader, args=(proc1, file1))
t2 = threading.Thread(target=output_reader, args=(proc2, file2))
t1.start()
t2.start()
t1.join()
t2.join()
sleep.py
#!/usr/bin/env python3
import sys
import time
for i in range(4):
print(i + int(sys.argv[1]))
sys.stdout.flush()
time.sleep(0.5)
After running:
./main.py
stdout get updated every 0.5 seconds for every two lines to contain:
0
10
1
11
2
12
3
13
and each log file contains the respective log for a given process.
Inspired by: https://eli.thegreenplace.net/2017/interacting-with-a-long-running-child-process-in-python/
Tested on Ubuntu 18.04, Python 3.6.7.
You probably want to start investigating the os module for forking different threads (by opening an interactive session and issuing help(os)). The relevant functions are fork and any of the exec ones. To give you an idea on how to start, put something like this in a function that performs the fork (the function needs to take a list or tuple 'args' as an argument that contains the program's name and its parameters; you may also want to define stdin, out and err for the new thread):
try:
pid = os.fork()
except OSError, e:
## some debug output
sys.exit(1)
if pid == 0:
## eventually use os.putenv(..) to set environment variables
## os.execv strips of args[0] for the arguments
os.execv(args[0], args)
You can use
import os
pid = os.fork()
if pid == 0:
Continue to other code ...
This will make the python process run in background.
I haven't tried this yet but using .pyw files instead of .py files should help. pyw files dosen't have a console so in theory it should not appear and work like a background process.
#!/usr/bin/python
import subprocess
import signal
import time
p=subprocess("adb devices")
p_id=p.pid
time.sleep
os.kill(p_pid,signal.SIGINT)
print "bye"
This is working fine and printing "bye", but the below code is not terminating, keeps on printing log content, and won't print "bye".
#!/usr/bin/python
import subprocess
import signal
import time
p=subprocess("adb logcat -v time event")
p_id=p.pid
time.sleep
os.kill(p_pid,signal.SIGINT)
print "bye"
kindly suggest
Thanks in advance
Bushra
#!/usr/bin/python
import subprocess
import time
p=subprocess.Popen(['adb', 'logcat', '-v', 'time', 'event'])
time.sleep(1) # wait for a second
p.kill()
print "bye"
Notable items:
If you use a shell, your signal is delivered to the shell, not the process it started. To ensure that the signal is delivered direct to the process, run the process directly.
time.sleep does absolutely nothing on its own; it needs to be called as a function, as in time.sleep(1), to have any effect.
Use p.kill() to let the subprocess module select the appropriate kill mechanism for the current platform, as opposed to os.kill().
After start.exe is executed by Python, the output of start.exe is displayed in the python stdout. However 5 seconds later, we do not see Quitting printed and the task is not killed.
Is there an easier way to terminate an exe that was originally started by Python? Like getting a handle for the exe executed and using that handle to kill the process.
import subprocess
import os
import time
subprocess.call(['.\\start.exe'])
time.sleep(5)
print("Quitting")
os.system("taskkill /im start.exe")
As you can see at the subprocess documentation, the call function blocks until the process completes.
I believe that you should use the Popen functions as it doesn't block, and provides you a process handle.
Then, you can kill it like this: p.kill(), where p is the result from the Popen function.
The reason is that subprocess.call will wait for your start.exe to complete before continuing your script.
You can get a popen object (with better functionality) to your process via:
import subprocess
import os
import time
process = subprocess.Popen(['.\\start.exe'])
time.sleep(5)
print("Quitting")
process.terminate()
An application that asks for a keyboard interrupt. How can I send for a keyboard interrupt programmatically? I need it for automation.
Like <C-c> or <C-x>
KeyboardInterrupt
Code running in a separate thread can cause a KeyboardInterrupt to be generated in the main thread by calling thread.interrupt_main().
See https://docs.python.org/2/library/thread.html#thread.interrupt_main
Since you mention automation I assume you want a SendKeys for Python. Try this: http://rutherfurd.net/python/sendkeys/
My suggestion to solve this problem is to use the following code pattern. I used it to programmatically start a tensorboard server and shut it down by sending a CTRL-C when the object it belongs to is deleted. Generally speaking, this should work for any example that provokes a subprocess that is supposed to be send a KeyBoardInterrupt:
Import signal and subprocess
import signal
import subprocess
Create the subprocess using subprocess.Popen. Important: set the creationflags parameter to subprocess.CREATE_NEW_PROCESS_GROUP. This is necessary to later be able to send the KeyboardInterrupt event.
command= <enter your command that is supposed to be run in different process as a string>
process = subprocess.Popen(command.split(),stdout=subprocess.PIPE,stdin=subprocess.PIPE,creationflags=subprocess.CREATE_NEW_PROCESS_GROUP)
Wherever you want to send the keyboardInterrupt, do the following:
process.send_signal(signal.CTRL_C_EVENT)
That is it! Please see the the official subprocess documentation for insights on why the creationflags parameter of popen has to be set that way.
This is how the code looks for my example in a less generic way:
import signal
import subprocess
import time
class ExperimentTracker():
def __init__(self):
self.tensorboard_process=None
def __del__(self):
#shutdown tensorboard server and terminate process
self.tensorboard_process.send_signal(signal.CTRL_C_EVENT)
time.sleep(0.2)
self.tensorboard_process.kill()
def launch_tensorboard(self):
#launch tensorboard
bashCommand= "tensorboard --logdir runs"
self.tensorboard_process = subprocess.Popen(bashCommand.split(),stdout=subprocess.PIPE,stdin=subprocess.PIPE,creationflags=subprocess.CREATE_NEW_PROCESS_GROUP)
time.sleep(2) #sleep for 2 seconds to give tensorboard time to be launched
If say you want to run a program via shell ./program. In Linux what you could do is:
# Made a function to handle more complex programs which require multiple inputs.
run(){
./program
}
# Making a child process that will run the program while you stop it later.
run &
childPid=($!)
# Process id of the program which you want to interrupt (via `run`).
# Time after which you want to interrupt
sleep 5
# Actual command to send the interrupt
kill -SIGINT $childPid
Let me know if it works in windows as well.