We Keep Coding

Running .exe without a console - OSError: [WinError 6] The handle is invalid

EDIT: Please note, I have seen other topics concerning this issue and already tried most of the suggestions there
I use pyinstaller to run an .exe file and now I am tryng to run it without a console (using the -w command).
One of my key libraries, patool, uses subprocesses, which gives the following error:
Traceback (most recent call last):
File "apscheduler\executors\base.py", line 125, in run_job
File "script.py", line 478, in Archiver
File "patoolib\__init__.py", line 521, in _create_archive
File "patoolib\__init__.py", line 421, in run_archive_cmdlist
File "patoolib\util.py", line 227, in run_checked
File "patoolib\util.py", line 219, in run
File "subprocess.py", line 339, in call
File "subprocess.py", line 753, in __init__
File "subprocess.py", line 1090, in _get_handles
OSError: [WinError 6] The handle is invalid
Here is the part of the patool util.py code which has the subprocesses.call() that gives the error:
def run (cmd, verbosity=0, **kwargs):
"""Run command without error checking.
#return: command return code"""
# Note that shell_quote_nt() result is not suitable for copy-paste
# (especially on Unix systems), but it looks nicer than shell_quote().
if verbosity >= 0:
log_info("running %s" % " ".join(map(shell_quote_nt, cmd)))
if kwargs:
if verbosity >= 0:
log_info(" with %s" % ", ".join("%s=%s" % (k, shell_quote(str(v)))\
for k, v in kwargs.items()))
if kwargs.get("shell"):
# for shell calls the command must be a string
cmd = " ".join(cmd)
if verbosity < 1:
# hide command output on stdout
with open(os.devnull, 'wb') as devnull:
kwargs['stdout'] = devnull
res = subprocess.call(cmd, **kwargs) <------------- ERROR
else:
res = subprocess.call(cmd, **kwargs)
return res
This is a common error, so I tried reading about subprocesses module and also dug out every single possible suggestion online, including:
adding kwargs['stdin'] = devnull, as suggested here: Python running as Windows Service: OSError: [WinError 6] The handle is invalid
adding shell=True to the call() method
adding subprocess._cleanup() at the start of the run() function
Neither of these works, the handler is still invalid. The programme works fine with the console active.
I am using Python 3.7, Anaconda3, 64-bit Windows 10 OS.
Later in the util.py there is a subprocess.popen() that I suspect will cause my the same problem.
I was trying to run the .exe by having the console active and then hiding it, but then I encounter other problems (it does not start upon system start up). I guess the console being present is pretty important, but I would love to get rid off it for better user experience.

Fixed it by adding: stdin=subprocess.PIPE in this file for python 3.8:
C:\Program Files (x86)\Python38-32\Lib*os.py* line 983 to 991
proc = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, **stdin=subprocess.PIPE**, stderr=subprocess.STDOUT,bufsize=buffering)

Error while trying to parse running processes through Python

I made an attempt at parsing running programs in my computer (Debian OS) with the Subprocess python module. Here is my code:
import subprocess
cmd = "ps -A" # Unix command to get running processes
runningprox = subprocess.check_output(cmd) #returns output as byte string
rpstring = runningprox.decode("utf-8")
#converts byte string to string and puts it in a variable
print(rpstring)
However, when I run the code, I get the following error message:
Traceback (most recent call last): File "ratalert.py", line 6, in
runningprox = subprocess.check_output(cmd) #returns output as byte string File "/usr/local/lib/python3.6/subprocess.py", line 336, in
check_output
**kwargs).stdout File "/usr/local/lib/python3.6/subprocess.py", line 403, in run
with Popen(*popenargs, **kwargs) as process: File "/usr/local/lib/python3.6/subprocess.py", line 707, in init
restore_signals, start_new_session) File "/usr/local/lib/python3.6/subprocess.py", line 1333, in _execute_child
raise child_exception_type(errno_num, err_msg) FileNotFoundError: [Errno 2] No such file or directory: 'ps -A'
I don't understand why I get this error message. Considering 'ps -A' is neither a file nor a directory, but just the Unix command I put in a variable as a string.
How can I fix this? Thank you.

The subprocess.check_output function expects a list with the command and its arguments, like so:
runningprox = subprocess.check_output(['ps', '-A'])
Otherwise, it will treat the string you passed as a single command, and will look up for an executable file with name ps -A, space included and all.
You can use shlex to do the splitting as the shell would do:
import shlex, subprocess
cmd = 'ps -A'
runningprox = subprocess.check_output(shlex.split(cmd))

using Python to send subprocess commands with escape characters

I'm using a python script which uses subprocess to pass a commmand to the terminal. Part of the command that I'm passing involves paths which contain parentheses. Python handles strings with parentheses fine, but of course terminal does not handle these without escape characters.
I'm trying to pass variables to a command line program by feeding a string into subprocess, but here's simple example to reproduce the error:
import subprocess
path = '/home/user/Desktop/directory(2018)'
command_str = 'rmdir ' + path
print (subprocess.run(command_str))
which gives me this error:
Traceback (most recent call last):
File "ex.py", line 7, in <module>
print (subprocess.run(command_str))
File "/usr/lib/python3.6/subprocess.py", line 403, in run
with Popen(*popenargs, **kwargs) as process:
File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'rmdir /home/user/Desktop/directory(2018)': 'rmdir /home/user/Desktop/directory(2018)'
When I write it directly into the terminal with escape characters it works great.
$ rmdir /home/user/Desktop/directory\(2018\)
But in Python when I try to add escape characters to the strings before calling subprocess:
command_str = command_str.replace('(','\(')
command_str = command_str.replace(')','\)')
I get the same error as before because, unlike print, the subprocess string adds a second escape character which gets passed to the terminal.
Traceback (most recent call last):
File "ex.py", line 7, in <module>
print (subprocess.run(command_str))
File "/usr/lib/python3.6/subprocess.py", line 403, in run
with Popen(*popenargs, **kwargs) as process:
File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'rmdir /home/user/Desktop/directory\\(2018\\)': 'rmdir /home/user/Desktop/directory\\(2018\\)'
Is there a way to fix this particular error? Either by doing something different with replace or subprocess.run? (I'm not looking for a better way to remove directories.) Thanks.

Python implements rm and rmdir so no need to call a process. In general, if you want to skip shell processing on a command in subprocess, don't use the shell.
import subprocess
path = '/home/user/Desktop/directory(2018)'
command = ['rmdir', path]
print (subprocess.run(command, shell=False))
The shell breaks a command line into a list of arguments. You can build that list yourself and skip the shell completely.

Do not use subprocess, and you don't have to worry about shell escaping. Use the high-level file operation APIs provided in stdlib's shutil:
import shutil
shutil.rmtree('/home/user/Desktop/directory(2018)')

subprocess.call() fails on Mac and Linux

I'm running into a weird issue with subprocess.call() function. I am trying to execute Java's 'jar' command using subprocess.call(). Here's the code:
import os
import subprocess
def read_war():
war_file_path = "jackrabbit-webapp-2.6.5.war"
java_home = os.environ['JAVA_HOME']
jar_path = os.path.join(java_home, 'bin', 'jar')
jar_cmd = jar_path + ' tvf ' + war_file_path
print "command to be executed is : " + jar_cmd
subprocess.call(jar_cmd)
read_war()
I'm using Python v2.7.3 on both Windows and Linux (Oracle Enterprise Linux).
On Windows 7, I see the contents of the war file being displayed. On Linux, however, I see a 'no such file or directory' error.:
$ python example.py
command to be executed is : /usr/local/tools/jdk1.7.0_15/bin/jar tvf jackrabbit-webapp-2.6.5.war
Traceback (most recent call last):
File "example.py", line 24, in <module>
read_war()
File "example.py", line 23, in read_war
subprocess.call(jar_cmd)
File "/usr/local/tools/Python-2.7.3/Lib/subprocess.py", line 493, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/local/tools/Python-2.7.3/Lib/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/local/tools/Python-2.7.3/Lib/subprocess.py", line 1249, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
$
I've tried the command '/usr/local/tools/jdk1.7.0_15/bin/jar tvf jackrabbit-webapp-2.6.5.war' from command prompt and it works fine. So, nothing's wrong with the command.
I've tried various combinations of subprocess.call() - passing a string, passing a list etc. None of them worked. Any help at all would be appreciated.

Add shell=True to the call. On windows, the CreateProcess command does string parsing to separate the command and its various arguments. On linux, you only get string processing if you specifically tell subprocess to call the shell. Otherwise, it treats that entire string you handed in as the command and you don't get very far.
subprocess.call(jar_cmd, shell=True)

Use a list (sequence) argument instead of a string as the docs say:
args is required for all calls and should be a string, or a sequence
of program arguments. Providing a sequence of arguments is generally
preferred, as it allows the module to take care of any required
escaping and quoting of arguments (e.g. to permit spaces in file
names). If passing a single string, either shell must be True (see
below) or else the string must simply name the program to be executed
without specifying any arguments.
Example:
import os
import subprocess
def read_war():
war_file_path = "jackrabbit-webapp-2.6.5.war"
jar_path = os.path.join(os.environ['JAVA_HOME'], 'bin', 'jar')
jar_cmd = [jar_path, 'tvf', war_file_path]
print("command to be executed is: %s" % jar_cmd)
subprocess.check_call(jar_cmd)
read_war()
I've used check_call to raise an exception if the command returns non-zero exit status.

Python - unable to call system command

I have some python code, from which I want to call another program. This program will
Print some output to STDOUT
Write a file to disk
Using call I get the following behavior;
from subprocess import call
call(['./tango_x86_64_release', 'VTS1 ct="N" nt="N" ph="7.2" te="303" io="0.02" seq="MKHPYEEFPTGSKSPYNMSRGAHPGAV"'])
34, File not properly written, try writing it up again,
1
This happens regardless if if the arguments are split into a list or not;
call(['./tango_x86_64_release', 'VTS1', 'ct="N"', 'nt="N"', 'ph="7.2"', 'te="303"', 'io="0.02"', 'seq="MKHPYEEFPTGSKSPYNMSRGAHPGAV"'])
34, File not properly written, try writing it up again,
1
I can call this same command from the my terminal
./tango_x86_64_release VTS1 ct="N" nt="N" ph="7.2" te="303" io="0.02" seq="MKHPYEEFPTGSKSPYNMSRGAHPGAV"
Which works and gives an exit status of 0.
It seems like its the writing to disk which is causing issues, if I break the command then I get the appropriate warning message (i.e. remove an argument, it warns me that the argument is missing).
Using subprocess.Popen() gives an OSError;
import subprocess as sub
output = sub.Popen('./tango_x86_64_release VTS1 ct="N" nt="N" ph="7.2" te="303" io="0.02" seq="MKHPYEEFPTGSKSPYNMSRGAHPGAV"', stdout=sub.PIPE, stderr=sub.PIPE)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/lib64/python2.7/subprocess.py", line 1249, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Any help greatly appreciated

Use shlex.split to split the command for you:
import shlex
call(shlex.split('./tango_x86_64_release VTS1 ct="N" nt="N" ph="7.2" te="303" io="0.02" seq="MKHPYEEFPTGSKSPYNMSRGAHPGAV"'))
Note that although you might be able to solve your problem by adding shell=True, you should avoid it if possible, since it can be a security risk (search for "shell injection").

Try to add shell=True to the Popen call.
Also see:
Why does subprocess.Popen() with shell=True work differently on Linux vs Windows?
Popen error: [Errno 2] No such file or directory
Documentation (and why Passing shell=True can be a security hazard)