How to get subprocess default/preset values/behaviour - python

In this post I have explained how to check how many QGIS projects are open in a windows session.
Here is the shortened code:
import os
import subprocess
from os.path import basename
from PyQt4.QtGui import QMessageBox
def checkQgisProcesses(self):
try:
from subprocess import DEVNULL
except ImportError:
DEVNULL = os.open(os.devnull, os.O_RDWR)
res = subprocess.check_output('C:\Windows\System32\cmd.exe /c tasklist /FI "IMAGENAME eq qgis-bin.exe" -v 2>NUL', stdin=DEVNULL, stderr=DEVNULL, startupinfo = info)
...
def someOtherFunc(self):
self.checkQgisProcesses()
...
The code uses tasklist from Windows to get information about all open windows and filters the QGIS window title.
I use this code within a function in a QGIS plugin.
There is another function in the plugin where I call multiple subprocess another time to make some calculations with another programm (SAGA GIS):
curv_PATH = plugin_pth + 'dem_curvature.bat'
subprocess.call(["C:\Windows\System32\cmd.exe", "/c", script_PATH], startupinfo = info)
subprocess.call(['gdalwarp', raster_dest_data + 'dem_gaussian.sdat', '-tr', cellsize, cellsize, '-r', 'bilinear', raster_dest_data + 'dem_res.tif' ], startupinfo = info)
subprocess.call(["C:\Windows\System32\cmd.exe", "/c", curv_PATH], startupinfo = info)
The problem is that these subprocess calls doesn't work anymore. I don't get a specific error in the QGIS python console.
When I comment self.checkQgisProcesses() it works again.
I think the problem lies within the DEVNULL, stdin, stderr parameters.
How can I set them back to default?
UPDATE
I wrongly thought the problem could be the DEVNULL declaration. Obviously the problem is the use of subprocess.STARTUPINFO().
Here is a reproducible example:
import os
from os.path import basename
import subprocess
SW_HIDE = 0
info = subprocess.STARTUPINFO()
info.dwFlags = subprocess.STARTF_USESHOWWINDOW
info.wShowWindow = SW_HIDE
def checkFirefoxProcesses():
try:
from subprocess import DEVNULL
except ImportError:
DEVNULL = os.open(os.devnull, os.O_RDWR)
res = subprocess.check_output('C:\Windows\System32\cmd.exe /c tasklist /FI "IMAGENAME eq firefox.exe" -v 2>NUL', stdin=DEVNULL, stderr=DEVNULL, startupinfo = info)
print res
def helloWorldToText():
subprocess.call(["C:\Windows\System32\cmd.exe", "/c", r"d:\\hello_world.bat"], startupinfo = info)
checkFirefoxProcesses()
helloWorldToText()
This is the code from hello_world.bat:
REM #ECHO OFF
ECHO Hello World! > d:\\hello_world.txt
PAUSE
The function checkFirefoxProcesses() looks for open firefox processes. To run through the example you have to open a firefox session. The function helloWorldToText() creates a text file from ECHO with the help of a BAT file.
The example doesn't creates the text file with the declaration of startupinfo = info. When I run the subprocess of function helloWorldToText() without startupinfo = info it works.
The case: running checkFirefoxProcesses() without startupinfo = info and running helloWorldToText() with startupinfo = info works.

Related

how can i pass a file name in python tkinter using subprocess

below is my code this is a mini IDE for a class project I am stuck here am trying to build an IDE that compiles java. I downloaded JDK and am using subprocess to pipe cmd and communicate with javac but I need to pass the file name with extension so it just shows output and I also need help with outputting a console because it tends to only open in visual studio terminal please help me because I will be submitting on Thursday.
femi.femiii#gmail.com
from tkinter import filedialog
from tkinter import messagebox
import subprocess
import os
name_file = os.path.basename(__file__)
# run button that opens command line
def run(self, *args):
p1 = subprocess.Popen('cmd', shell=True, stdin=None, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
p2 = subprocess.Popen('javac name_file; java name_file', shell=True, stdin=p1.stdout)
p1.stdout.close()
out, err = p2.communicate()
if __name__ == "__main__":
master = tk.Tk()
pt = PyText(master)
master.mainloop()```
The easiest way I can think of would be to use sys argv's to pass the file name to a node or java routine
modify your python script to pass info to command line in argv
fname = file
#file you want to pass
pythonCall = 'node javascript.js -- '+fname
#the -- passes what follows into the sys argvs
#this is python notation, java/node may not need the '--' but just a space
runCommand = 'cmd.exe /c ' + pythonCall
subprocess.Popen(runCommand)
then at the start of your javascript, this will give you the file name that was passed into argv, its from the link I included below.
var args = process.argv.slice(2);
if you need more help accessing process.argv in java:
How do I pass command line arguments to a Node.js program?

python subprocess module stuck while executing a batch file

I am trying to learn to code using python on my own but I ran into a problem.
I am using python's subprocess module to execute a .bat file, but the process seems to get stuck at the bat file. The python code currently looks like this:
import getpass
username = getpass.getuser()
from subprocess import Popen
p = Popen("hidefolder.bat", cwd=r"C:\Users\%s\Desktop" % username)
stdout, stderr = p.communicate()
import sys
sys.exit()
And the .bat file looks like this:
if exist "C:\Users\%username%\Desktop\HiddenFolder\" (
attrib -s -h "HiddenFolder"
rename "HiddenFolder" "Projects"
exit
)
if exist "C:\Users\%username%\Desktop\Projects\" (
rename "Projects" "HiddenFolder"
attrib +s +h "HiddenFolder"
exit
)
if not exist "C:\Users\%username%\Desktop\HiddenFolder\" (
mkdir "C:\Users\%username%\Desktop\HiddenFolder\"
)
exit
Is there a way to kill the child process even if the python script is waiting for the child process to be terminated before continuing? Or is the problem in the child process to start with?
Thank you in advance.
You need to use subprocess.PIPE for stdout and stderr, or else they can't be fetched through Popen.communicate, and is the reason why your process is stuck.
from subprocess import Popen, PIPE
import getpass
username = getpass.getuser()
p = Popen("hidefolder.bat", cwd=r"C:\Users\%s\Desktop" % username, stdout=PIPE, stderr=PIPE)
stdout, stderr = p.communicate()
import sys
sys.exit()
I am a new programmer but i could solve my problem writting below code.
import subprocess
subprocess.call([r'ProcurementSoftwareRun.bat'])
print ('Software run successful')
My bat file was like:
#ECHO OFF
cmd /c start "" "C:\Program Files (x86)\UserName\ERPModule\PROCUREMENT.exe
exit

Multiple pipes in subprocess

I am trying to use Sailfish, which takes multiple fastq files as arguments, in a ruffus pipeline. I execute Sailfish using the subprocess module in python, but <() in the subprocess call does not work even when I set shell=True.
This is the command I want to execute using python:
sailfish quant [options] -1 <(cat sample1a.fastq sample1b.fastq) -2 <(cat sample2a.fastq sample2b.fastq) -o [output_file]
or (preferably):
sailfish quant [options] -1 <(gunzip sample1a.fastq.gz sample1b.fastq.gz) -2 <(gunzip sample2a.fastq.gz sample2b.fastq.gz) -o [output_file]
A generalization:
someprogram <(someprocess) <(someprocess)
How would I go about doing this in python? Is subprocess the right approach?
To emulate the bash process substitution:
#!/usr/bin/env python
from subprocess import check_call
check_call('someprogram <(someprocess) <(anotherprocess)',
shell=True, executable='/bin/bash')
In Python, you could use named pipes:
#!/usr/bin/env python
from subprocess import Popen
with named_pipes(n=2) as paths:
someprogram = Popen(['someprogram'] + paths)
processes = []
for path, command in zip(paths, ['someprocess', 'anotherprocess']):
with open(path, 'wb', 0) as pipe:
processes.append(Popen(command, stdout=pipe, close_fds=True))
for p in [someprogram] + processes:
p.wait()
where named_pipes(n) is:
import os
import shutil
import tempfile
from contextlib import contextmanager
#contextmanager
def named_pipes(n=1):
dirname = tempfile.mkdtemp()
try:
paths = [os.path.join(dirname, 'named_pipe' + str(i)) for i in range(n)]
for path in paths:
os.mkfifo(path)
yield paths
finally:
shutil.rmtree(dirname)
Another and more preferable way (no need to create a named entry on disk) to implement the bash process substitution is to use /dev/fd/N filenames (if they are available) as suggested by #Dunes. On FreeBSD, fdescfs(5) (/dev/fd/#) creates entries for all file descriptors opened by the process. To test availability, run:
$ test -r /dev/fd/3 3</dev/null && echo /dev/fd is available
If it fails; try to symlink /dev/fd to proc(5) as it is done on some Linuxes:
$ ln -s /proc/self/fd /dev/fd
Here's /dev/fd-based implementation of someprogram <(someprocess) <(anotherprocess) bash command:
#!/usr/bin/env python3
from contextlib import ExitStack
from subprocess import CalledProcessError, Popen, PIPE
def kill(process):
if process.poll() is None: # still running
process.kill()
with ExitStack() as stack: # for proper cleanup
processes = []
for command in [['someprocess'], ['anotherprocess']]: # start child processes
processes.append(stack.enter_context(Popen(command, stdout=PIPE)))
stack.callback(kill, processes[-1]) # kill on someprogram exit
fds = [p.stdout.fileno() for p in processes]
someprogram = stack.enter_context(
Popen(['someprogram'] + ['/dev/fd/%d' % fd for fd in fds], pass_fds=fds))
for p in processes: # close pipes in the parent
p.stdout.close()
# exit stack: wait for processes
if someprogram.returncode != 0: # errors shouldn't go unnoticed
raise CalledProcessError(someprogram.returncode, someprogram.args)
Note: on my Ubuntu machine, the subprocess code works only in Python 3.4+, despite pass_fds being available since Python 3.2.
Whilst J.F. Sebastian has provided an answer using named pipes it is possible to do this with anonymous pipes.
import shlex
from subprocess import Popen, PIPE
inputcmd0 = "zcat hello.gz" # gzipped file containing "hello"
inputcmd1 = "zcat world.gz" # gzipped file containing "world"
def get_filename(file_):
return "/dev/fd/{}".format(file_.fileno())
def get_stdout_fds(*processes):
return tuple(p.stdout.fileno() for p in processes)
# setup producer processes
inputproc0 = Popen(shlex.split(inputcmd0), stdout=PIPE)
inputproc1 = Popen(shlex.split(inputcmd1), stdout=PIPE)
# setup consumer process
# pass input processes pipes by "filename" eg. /dev/fd/5
cmd = "cat {file0} {file1}".format(file0=get_filename(inputproc0.stdout),
file1=get_filename(inputproc1.stdout))
print("command is:", cmd)
# pass_fds argument tells Popen to let the child process inherit the pipe's fds
someprogram = Popen(shlex.split(cmd), stdout=PIPE,
pass_fds=get_stdout_fds(inputproc0, inputproc1))
output, error = someprogram.communicate()
for p in [inputproc0, inputproc1, someprogram]:
p.wait()
assert output == b"hello\nworld\n"

python subprocess communicate freezes

I have the following python code that hangs :
cmd = ["ssh", "-tt", "-vvv"] + self.common_args
cmd += [self.host]
cmd += ["cat > %s" % (out_path)]
p = subprocess.Popen(cmd, stdin=subprocess.PIPE,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = p.communicate(in_string)
It is supposed to save a string (in_string) into a remote file over ssh.
The file is correctly saved but then the process hangs. If I use
cmd += ["echo"] instead of
cmd += ["cat > %s" % (out_path)]
the process does not hang so I am pretty sure that I misunderstand something about the way communicate considers that the process has exited.
do you know how I should write the command so the the "cat > file" does not make communicate hang ?
-tt option allocates tty that prevents the child process to exit when .communicate() closes p.stdin (EOF is ignored). This works:
import pipes
from subprocess import Popen, PIPE
cmd = ["ssh", self.host, "cat > " + pipes.quote(out_path)] # no '-tt'
p = Popen(cmd, stdin=PIPE, stdout=PIPE, stderr=PIPE)
stdout, stderr = p.communicate(in_string)
You could use paramiko -- pure Python ssh library, to write data to a remote file via ssh:
#!/usr/bin/env python
import os
import posixpath
import sys
from contextlib import closing
from paramiko import SSHConfig, SSHClient
hostname, out_path, in_string = sys.argv[1:] # get from command-line
# load parameters to setup ssh connection
config = SSHConfig()
with open(os.path.expanduser('~/.ssh/config')) as config_file:
config.parse(config_file)
d = config.lookup(hostname)
# connect
with closing(SSHClient()) as ssh:
ssh.load_system_host_keys()
ssh.connect(d['hostname'], username=d.get('user'))
with closing(ssh.open_sftp()) as sftp:
makedirs_exists_ok(sftp, posixpath.dirname(out_path))
with sftp.open(out_path, 'wb') as remote_file:
remote_file.write(in_string)
where makedirs_exists_ok() function mimics os.makedirs():
from functools import partial
from stat import S_ISDIR
def isdir(ftp, path):
try:
return S_ISDIR(ftp.stat(path).st_mode)
except EnvironmentError:
return None
def makedirs_exists_ok(ftp, path):
def exists_ok(mkdir, name):
"""Don't raise an error if name is already a directory."""
try:
mkdir(name)
except EnvironmentError:
if not isdir(ftp, name):
raise
# from os.makedirs()
head, tail = posixpath.split(path)
if not tail:
assert path.endswith(posixpath.sep)
head, tail = posixpath.split(head)
if head and tail and not isdir(ftp, head):
exists_ok(partial(makedirs_exists_ok, ftp), head) # recursive call
# do create directory
assert isdir(ftp, head)
exists_ok(ftp.mkdir, path)
It makes sense that the cat command hangs. It is waiting for an EOF. I tried sending an EOF in the string but couldn't get it to work. Upon researching this question, I found a great module for streamlining the use of SSH for command line tasks like your cat example. It might not be exactly what you need for your usecase, but it does do what your question asks.
Install fabric with
pip install fabric
Inside a file called fabfile.py put
from fabric.api import run
def write_file(in_string, path):
run('echo {} > {}'.format(in_string,path))
And then run this from the command prompt with,
fab -H username#host write_file:in_string=test,path=/path/to/file

Python 2.5 and Tskill

I had the tskill operation working fine through a python script when using Python 2.7 but when I switched to 2.5 this code fails.
import os
import time
import win32com.client
os.startfile('cmd')
shell = win32com.client.Dispatch("WScript.Shell")
shell.AppActivate('Administrator:C:\Windows\System32\cmd.exe')
time.sleep(2)
shell.SendKeys('tskill java + {ENTER}')
time.sleep(2)
shell.Sendkeys('tskill javaw + {ENTER}')
#shell.SendKeys('exit + {ENTER}')
I use os.startfile to run a python script that opens and runs the application(It's a graphics tool).
Note: The application is already running by the time this script is called, hence I don't need to open it here.
Can some one please tell me how to close an application successfully using this or similar method on Python 2.5. Thanks for the help!
I don't have tskill on my system, but I think it's similar to taskkill /F /FI "IMAGENAME eq java*". Anyway, here's how to call a normal console application in the background (i.e. a hidden window) using the subprocess module (tested in Python 2.5.2, but using taskkill instead):
import subprocess
def tskill(*args):
si = subprocess.STARTUPINFO()
si.dwFlags |= subprocess.STARTF_USESHOWWINDOW
command = ('tskill',) + args
rc = subprocess.call(command, startupinfo=si)
return rc == 0
if tskill('java') and tskill('javaw'):
#success
I used call because it only matters whether the call succeeds or fails, but in general you can use subprocess.PIPE to capture stdout and stderr using subprocess.Popen.
Edit: How to kill a process started with the subprocess module
command = ['path/to/executable', 'arg0', 'arg1', '...']
p = subprocess.Popen(command)
#do something
#kill if it's still alive
if p.poll() is None:
p.kill()

Categories