We Keep Coding

Closing a window opened by Subprocess

I am working on a script(let's call it p1.py) which calls another python script(let's call it p2.py). This p2.py is called with subprocess run(cmd, shell=True) and at some certain point, p2.py opens up a window that needs to be canceled with ESC. In the meantime, p1.py is just waiting for p2.py to be over. Unfortunately, this window is not avoidable coming from the library which I don't want to touch for compatibility. The question is that any ways to avoid this window or anyways to close it programmatically?
Parameters:
Language: Python 3.7
OS: Ubuntu/Linux

You can use subprocess.Popen('python py2.py') instead of subprocess.run().
In py1.py this will return immediately and give you an object to kill the process of py2.py later. However, you need to know how long you have to wait or get some other signal for py1.py when py2.py is done and the window is showing.
Try something like this in py1.py:
import subprocess
import signal
import time
p = subprocess.Popen('python py2.py')
time.sleep(...) # wait until py2.py is done
p.send_signal(signal.SIGTERM)
Note: This does not work with the shell=True parameter.

How to skip error message pop-up with subprocess in Python

I am using subprocess in Python to call an external program on WINDOWS. I control the process with ThreadPool so that I can limit it to max 6 processes at the same time, and new process continuously began when one was done.
Codes as below:
### some codes above
### Code of Subprocess Part
from multiprocessing.pool import ThreadPool as Pool
def FAST_worker(file):
p = subprocess.Popen([r'E:/pyworkspace/FAST/FAST_RV_W64.exe', file],
cwd = r'E:/pyworkspace/FAST/',
shell = True)
p.wait()
# List of *.in filenames
FAST_in_pathname_li = [
'334.in',
'893.in',
'9527.in',
...
'114514.in',
'1919810.in',
]
# Limit max 6 processes at same time
with Pool(processes = 6) as pool:
for result in pool.imap_unordered(FAST_worker, FAST_in_pathname_li):
pass
### some codes below
I got problem when the external program unexpectedly terminated and showed error message pop-up. Though the other 5 processes still kept going, the whole progress will finally get stuck at the "subprocess part" and couldn't go forward anymore. (unless I came to my desk and manually clicked "Shut down the program")
What I want to know is how can I avoid the pop-up and make the whole script process keep going, like bypass the error message or something, rather than manual click, in order to avoid wasting time.

Since we don't know enough about the program FAST_worker is calling, I'll assume you already checked there isn't any "kill on error" or "quiet" mode that would be more convenient to use in a script.
My two cents: maybe you can setup a timeout on the worker execution, so that a stuck process is killed automatically after a certain delay.
Building on the snippet provided here, here is a draft:
from threading import Timer
def FAST_worker(file, timeout_sec):
def kill_proc():
"""called by the Timer thread upon expiration"""
p.kill()
# maybe add task to list of failed task, for tracability
p = subprocess.Popen([r'E:/pyworkspace/FAST/FAST_RV_W64.exe', file],
cwd = r'E:/pyworkspace/FAST/',
shell = True)
# setup timer to kill the process after a timeout
timer = Timer(timeout_sec, kill_proc)
try:
timer.start()
stdout, stderr = p.wait()
finally:
timer.cancel()
Note that there are also GUI automation libraries in python that can do the clicking for you, but that is likely to be more tedious to program:
tutorial for pyAutoGui
SO question on the subject

Subprocess.check_call and throbber not working

So I have this part of code which does a simple thing : it launches a script and while the script is processing, a throbber is set on.
def go(self):
if ui.chk.isChecked():
self.startThrobber()
script = subprocess.check_call(r'"C:\Program Files\FME\fme.exe"', shell=False)
if script == 0:
self.stopThrobber() # opens a QMessageBox and stops throbber
else:
QMessageBox.information(self.popup(), "Errpr", "Error !")
After trying different methods (QThread, subprocess.Popen ...) this is the closest i got to make it work.
The only thing that doesn't work is that the throbber doesn't start right before the subprocess is executed, it starts after and thus it never stops.
So why is the throbber not ending when stopThrobber() is executed ?
And why is startThrobber not being executed before the subprocess (i'm pretty sure it's a subprocess thing, but i'm pretty new to all this, never heard about thread until yesterday)
EDIT :
The single quote was just a typing error, sorry. Still doesn't fix the problem.

Any call to a subprocess, from your main thread, that is blocking (waits for return) is going to stop your throbber from working properly. My answer to your other SO question on this topic outlines an approach that does not cause the subprocess call to block the main thread. I should point out that solution is not the only way to create a non-blocking call to a subprocess (for instance see here. You could create a QTimer to poll the subprocess poll() method periodically so that you can check the returncode to see if the subprocess has finished.)
The key theme is that you need your methods that run in the main thread to return quickly in order to keep the GUI responsive and allow your throbber to run/animate. So choose a way to launch the subprocess that meets this requirement.

Your single quotes denoting the raw string enclose the 'shell' argument.
def go(self):
if ui.chk.isChecked():
self.startThrobber()
script = subprocess.check_call(r"C:\Program Files\FME\fme.exe", shell=False)
if script == 0:
self.stopThrobber() # opens a QMessageBox and stops throbber
else:
QMessageBox.information(self.popup(), "Errpr", "Error !")

So I've tried another thing (unsuccessfully..)
When I click on a button, it executes startThrobber() and sends a signal to the following function :
def go(self):
self.startThrobber()
script = subprocess.Popen(r'"C:\Program Files\FME\fme.exe" ', shell=False)
while script.poll() == None:
time.sleep(1)
else:
p.stopThrobber()
But still doesn't working.. startThrobber is executed but nothing appears on the GUI... I thougth that the point of subprocess was to allow to do multiple tasks simultaneously so why isn't the throbber appearing ?
UPDATE : if i erase the while loop, startThrobber works : it appears while the subprocess is turning. So why when there is the while loop it doesn't work ?!

Python run system command and then exit... won't exit

I have the following python code:
os.system("C:/Python27/python.exe C:/GUI/TestGUI.py")
sys.exit(0)
It runs the command fine, and a window pops up. However, it doesn't exit the first script. It just stays there, and I eventually have to force kill the process. No errors are produced. What's going on?

instead of os.system use subprocess.Popen
this runs a command and doesn't wait for it and then exits:
import subprocess
import sys
subprocess.Popen(["mupdf", "/home/dan/Desktop/Sieve-JFP.pdf"])
sys.exit(0)
note that os.system(command) like:
p = subprocess.Popen(command)
p.wait()

KeyboardInterrupts and signals are only seen by the process (ie the main thread). If your nested command hangs due to some kind of file read or write block, you won't be able to quit the program using any keyboard commands.
Why does a read-only open of a named pipe block?
If you can't eliminate the source of the disk block, then one way is to wrap the process in the thread so you can force kill it. But if you do this, you leave opportunity for half-written and corrupted files on disk.

I suggest using os._exit instead of sys.exit, as sys.exit doesnt quit a program but raises exception level, or exits a thread. os._exit(-1) quits the entire program

import sys ,subprocess
subprocess.Popen(["C:/Python27/python.exe", "C:/GUI/TestGUI.py"])
sys.exit(0)
Popen from subprocess module what you are looking for.

Python: How to prevent subprocesses from receiving CTRL-C / Control-C / SIGINT

I am currently working on a wrapper for a dedicated server running in the shell. The wrapper spawns the server process via subprocess and observes and reacts to its output.
The dedicated server must be explicitly given a command to shut down gracefully. Thus, CTRL-C must not reach the server process.
If I capture the KeyboardInterrupt exception or overwrite the SIGINT-handler in python, the server process still receives the CTRL-C and stops immediately.
So my question is:
How to prevent subprocesses from receiving CTRL-C / Control-C / SIGINT?

Somebody in the #python IRC-Channel (Freenode) helped me by pointing out the preexec_fn parameter of subprocess.Popen(...):
If preexec_fn is set to a callable
object, this object will be called in
the child process just before the
child is executed. (Unix only)
Thus, the following code solves the problem (UNIX only):
import subprocess
import signal
def preexec_function():
# Ignore the SIGINT signal by setting the handler to the standard
# signal handler SIG_IGN.
signal.signal(signal.SIGINT, signal.SIG_IGN)
my_process = subprocess.Popen(
["my_executable"],
preexec_fn = preexec_function
)
Note: The signal is actually not prevented from reaching the subprocess. Instead, the preexec_fn above overwrites the signal's default handler so that the signal is ignored. Thus, this solution may not work if the subprocess overwrites the SIGINT handler again.
Another note: This solution works for all sorts of subprocesses, i.e. it is not restricted to subprocesses written in Python, too. For example the dedicated server I am writing my wrapper for is in fact written in Java.

Combining some of other answers that will do the trick - no signal sent to main app will be forwarded to the subprocess.
import os
from subprocess import Popen
def preexec(): # Don't forward signals.
os.setpgrp()
Popen('whatever', preexec_fn = preexec)

you can do something like this to make it work in windows and unix:
import subprocess
import sys
def pre_exec():
# To ignore CTRL+C signal in the new process
signal.signal(signal.SIGINT, signal.SIG_IGN)
if sys.platform.startswith('win'):
#https://msdn.microsoft.com/en-us/library/windows/desktop/ms684863(v=vs.85).aspx
#CREATE_NEW_PROCESS_GROUP=0x00000200 -> If this flag is specified, CTRL+C signals will be disabled
my_sub_process=subprocess.Popen(["executable"], creationflags=0x00000200)
else:
my_sub_process=subprocess.Popen(["executable"], preexec_fn = pre_exec)

After an hour of various attempts, this works for me:
process = subprocess.Popen(["someprocess"], creationflags=subprocess.DETACHED_PROCESS | subprocess.CREATE_NEW_PROCESS_GROUP)
It's solution for windows.

Try setting SIGINT to be ignored before spawning the subprocess (reset it to default behavior afterward).
If that doesn't work, you'll need to read up on job control and learn how to put a process in its own background process group, so that ^C doesn't even cause the kernel to send the signal to it in the first place. (May not be possible in Python without writing C helpers.)
See also this older question.