We Keep Coding

Running .exe without a console - OSError: [WinError 6] The handle is invalid

EDIT: Please note, I have seen other topics concerning this issue and already tried most of the suggestions there
I use pyinstaller to run an .exe file and now I am tryng to run it without a console (using the -w command).
One of my key libraries, patool, uses subprocesses, which gives the following error:
Traceback (most recent call last):
File "apscheduler\executors\base.py", line 125, in run_job
File "script.py", line 478, in Archiver
File "patoolib\__init__.py", line 521, in _create_archive
File "patoolib\__init__.py", line 421, in run_archive_cmdlist
File "patoolib\util.py", line 227, in run_checked
File "patoolib\util.py", line 219, in run
File "subprocess.py", line 339, in call
File "subprocess.py", line 753, in __init__
File "subprocess.py", line 1090, in _get_handles
OSError: [WinError 6] The handle is invalid
Here is the part of the patool util.py code which has the subprocesses.call() that gives the error:
def run (cmd, verbosity=0, **kwargs):
"""Run command without error checking.
#return: command return code"""
# Note that shell_quote_nt() result is not suitable for copy-paste
# (especially on Unix systems), but it looks nicer than shell_quote().
if verbosity >= 0:
log_info("running %s" % " ".join(map(shell_quote_nt, cmd)))
if kwargs:
if verbosity >= 0:
log_info(" with %s" % ", ".join("%s=%s" % (k, shell_quote(str(v)))\
for k, v in kwargs.items()))
if kwargs.get("shell"):
# for shell calls the command must be a string
cmd = " ".join(cmd)
if verbosity < 1:
# hide command output on stdout
with open(os.devnull, 'wb') as devnull:
kwargs['stdout'] = devnull
res = subprocess.call(cmd, **kwargs) <------------- ERROR
else:
res = subprocess.call(cmd, **kwargs)
return res
This is a common error, so I tried reading about subprocesses module and also dug out every single possible suggestion online, including:
adding kwargs['stdin'] = devnull, as suggested here: Python running as Windows Service: OSError: [WinError 6] The handle is invalid
adding shell=True to the call() method
adding subprocess._cleanup() at the start of the run() function
Neither of these works, the handler is still invalid. The programme works fine with the console active.
I am using Python 3.7, Anaconda3, 64-bit Windows 10 OS.
Later in the util.py there is a subprocess.popen() that I suspect will cause my the same problem.
I was trying to run the .exe by having the console active and then hiding it, but then I encounter other problems (it does not start upon system start up). I guess the console being present is pretty important, but I would love to get rid off it for better user experience.

Fixed it by adding: stdin=subprocess.PIPE in this file for python 3.8:
C:\Program Files (x86)\Python38-32\Lib*os.py* line 983 to 991
proc = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, **stdin=subprocess.PIPE**, stderr=subprocess.STDOUT,bufsize=buffering)

Exception str() failed

I'm starting to create a new 3D scanner with a Raspberry Pi 3B + and a Canon 6D. I have a part of the Python code to recover the images thanks to the gphoto2 library but I can't put my ISO configuration on the reflex.
I have already done several tests but nothing works. I always have the same error:
I use the gp command to send all parameters to the Canon reflex.
Import :
import time
from datetime import datetime
from sh import gphoto2 as gp
import signal, os, subprocess, shutil
Gp command example (all works) :
CaptureImageDownload = ["--capture-image-and-download"]
CaptureImage = ["--capture-image"]
But this line don't work :
ValueISO = ["--set-config iso=0"]
Here is the error displayed in the command terminal
File "CameraShot.py", line 124, in <module>
gp(ValueISO)
File "/usr/local/lib/python2.7/dist-packages/sh.py", line 1427, in __call__
return RunningCommand(cmd, call_args, stdin, stdout, stderr)
File "/usr/local/lib/python2.7/dist-packages/sh.py", line 774, in __init__
self.wait()
File "/usr/local/lib/python2.7/dist-packages/sh.py", line 792, in wait
self.handle_command_exit_code(exit_code)
File "/usr/local/lib/python2.7/dist-packages/sh.py", line 815, in handle_command_exit_code
raise exc
sh.ErrorReturnCode_1: <exception str() failed>
I can not write this command line otherwise my camera does not understand the order.

From the sh documentation on passing in arguments:
When passing multiple arguments to a command, each argument must be a separate string[.]
Yours are not separate strings. Split out the different parts (on spaces not surrounded by quotes):
ValueISO = ["--set-config", "iso=0"]
Also see the project's detailed explanation on why this is; but the short answer is that sh is not parsing arguments into separate strings like a shell would.
You can also use the shlex.split() function to handle the splitting for you:
ValueISO = shlex.split("--set-config iso=0")
Note that sh also supports using keyword arguments, where set_config="iso=0" is translated to ["--set-config", "iso=0"] for you. You could use that as:
value_iso = dict(set_config="iso=0")
then
gp(**value_iso)
That you get sh.ErrorReturnCode_1: <exception str() failed> is probably a bug in sh. Python uses type(exception).__name__: str(exception) as the last line of a traceback, and the str() call fails on the sh.ErrorReturnCode exception (sh.ErrorReturnCode_1 is a subclass of sh.ErrorReturnCode). I can see from the sh source code for the exception class that the error message is decoded from bytes to Unicode text, and Python 2 can't actually handle Unicode objects returned from a __str__ method. I've filed a bug report with sh to have that fixed.

UnicodeEncodeError in a Debian terminal

guys! I'm in a Debian 7 terminal trying to run a python script that sends a message via telegram-cli. The following line gives me an error:
check_call(["/usr/local/tg/bin/telegram-cli", "-W", "-k", "/usr/local/tg/tg-server.pub", "-e", msg])
At the end of this line, "msg" is a variable...
The error is this:
Traceback (most recent call last):
File "LogServicos.py", line 60, in <module>
msg_telegram()
File "LogServicos.py", line 17, in msg_telegram
check_call(["/usr/local/tg/bin/telegram-cli", "-W", "-k", "/usr/local/tg/tg-server.pub", "-e", msg])
File "/usr/local/lib/python3.5/subprocess.py", line 579, in check_call
retcode = call(*popenargs, **kwargs)
File "/usr/local/lib/python3.5/subprocess.py", line 560, in call
with Popen(*popenargs, **kwargs) as p:
File "/usr/local/lib/python3.5/subprocess.py", line 950, in __init__
restore_signals, start_new_session)
File "/usr/local/lib/python3.5/subprocess.py", line 1483, in _execute_child
restore_signals, start_new_session, preexec_fn)
UnicodeEncodeError: 'ascii' codec can't encode character '\xe7' in position 25: ordinal not in range(128)
The script is fully fuctional on my debian 15.10 using python 3.5.1, the same as in Debian.
Any help? Thanks!

The command line arguments passed to telegram-cli contain non-ascii characters, in this case msg contains the character ç (\xe7). On *nix, file names and command line arguments are just bytes in the end, so unicode strings need to be converted to bytes to be used as such.
python uses sys.getfilesystemencoding() for such conversions, which usually depends on the LANG and LC_* environment variables.
In your case on the computer where this program fails, the default locale only supports ascii (possibly the C locale), you can use the locale command to see which one is exactly used.
To fix this error, you can either:
encode msg when passing it to the command line, but that will only work correctly if telepathy-client doesn't also rely on the current locale to interpret it, otherwise you may end up with garbage characters
make sure the correct locale is set in the environment where you're running your script. You can use locale -a to check which locales are supported on your system, if no suitable locale shows up in the list, you may need to install a suitable new locale (like described here or here)

Why path is not found in python?

I am trying to open a file that exists using python, and opens perfectly if I open it using gedit in command line.
However, I get the following error message:
andreas#ubuntu:~/Desktop/Thesis/Codes/ModifiedFiles$ python vis.py -f myoutputcsv.csv
Matplotlib version 1.3.1
Traceback (most recent call last):
File "vis.py", line 1082, in <module>
reliability_table = ReliabilityTable(reliability_table_file)
File "vis.py", line 112, in __init__
self.read(filename)
File "vis.py", line 139, in read
self.data = genfromtxt(filename, delimiter=',',comments='#', dtype=float)
File "/usr/lib/python2.7/dist-packages/numpy/lib/npyio.py", line 1344, in genfromtxt
fhd = iter(np.lib._datasource.open(fname, 'rbU'))
File "/usr/lib/python2.7/dist-packages/numpy/lib/_datasource.py", line 147, in open
return ds.open(path, mode)
File "/usr/lib/python2.7/dist-packages/numpy/lib/_datasource.py", line 496, in open
raise IOError("%s not found." % path)
IOError: ~/Desktop/Thesis/Codes/ModifiedFiles/reliability_table_2.csv not found.
Do you know what I may be doing wrong? I have very little experience with python and I cannot find the reason the file opens at command line but not using python.

The ~ (tilde) is a shell expansion, not a special "filesystem expansion".
So ~ expands to the current user directly only when found in a shell command:
$echo ~
/home/username
But not if used in the filename passed to python's file objects.
The python code:
open('some/file/name')
is equivalent to opening the file 'some/file/name' in the shell, and I mean literally with the single quotes that prevent expansions included.
So:
open('~/file.txt')
Tries to open:
$echo '~/file.txt'
~/file.txt
And not:
$echo ~/file.txt
/home/username/file.txt
This is stated at the top of the documentation of the os.path module too:
Unlike a unix shell, Python does not do any automatic path expansions.
Functions such as expanduser() and expandvars() can be invoked
explicitly when an application desires shell-like path expansion. (See
also the glob module.)
In fact you can create a file called ~:
$touch '~'
$ls | grep '~'
~
The single quotes are necessary, because touch ~ would simply execute touch on /home/username and no file would be created.
Now if you try to delete it you must escape its name, otherwise the shell will expand it into /home/username:
$echo ~
/home/username
$rm ~ # translation: "rm: cannot remove "/home/username": It's a directory"
rm: impossibile rimuovere "/home/username": È una directory
$rm '~' # proper way to delete it
If you want to expand the ~ in filenames use the os.path.expanduser function:
>>> import os.path
>>> os.path.expanduser('~/file.txt')
'/home/username/file.txt'
Note that realpath and abspath do not expand the ~:
>>> os.path.realpath('~/file.txt')
'/home/username/~/file.txt'
>>> os.path.abspath('~/file.txt')
'/home/username/~/file.txt'
So, if you want to be sure to convert a pathname given by the user in "shell language"1 into an absolute path usable with python's file objects you should do:
os.path.abspath(os.path.expanduser(path))
1 Not saying sh/bash because they are cross platform.

subprocess.call() fails on Mac and Linux

I'm running into a weird issue with subprocess.call() function. I am trying to execute Java's 'jar' command using subprocess.call(). Here's the code:
import os
import subprocess
def read_war():
war_file_path = "jackrabbit-webapp-2.6.5.war"
java_home = os.environ['JAVA_HOME']
jar_path = os.path.join(java_home, 'bin', 'jar')
jar_cmd = jar_path + ' tvf ' + war_file_path
print "command to be executed is : " + jar_cmd
subprocess.call(jar_cmd)
read_war()
I'm using Python v2.7.3 on both Windows and Linux (Oracle Enterprise Linux).
On Windows 7, I see the contents of the war file being displayed. On Linux, however, I see a 'no such file or directory' error.:
$ python example.py
command to be executed is : /usr/local/tools/jdk1.7.0_15/bin/jar tvf jackrabbit-webapp-2.6.5.war
Traceback (most recent call last):
File "example.py", line 24, in <module>
read_war()
File "example.py", line 23, in read_war
subprocess.call(jar_cmd)
File "/usr/local/tools/Python-2.7.3/Lib/subprocess.py", line 493, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/local/tools/Python-2.7.3/Lib/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/local/tools/Python-2.7.3/Lib/subprocess.py", line 1249, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
$
I've tried the command '/usr/local/tools/jdk1.7.0_15/bin/jar tvf jackrabbit-webapp-2.6.5.war' from command prompt and it works fine. So, nothing's wrong with the command.
I've tried various combinations of subprocess.call() - passing a string, passing a list etc. None of them worked. Any help at all would be appreciated.

Add shell=True to the call. On windows, the CreateProcess command does string parsing to separate the command and its various arguments. On linux, you only get string processing if you specifically tell subprocess to call the shell. Otherwise, it treats that entire string you handed in as the command and you don't get very far.
subprocess.call(jar_cmd, shell=True)

Use a list (sequence) argument instead of a string as the docs say:
args is required for all calls and should be a string, or a sequence
of program arguments. Providing a sequence of arguments is generally
preferred, as it allows the module to take care of any required
escaping and quoting of arguments (e.g. to permit spaces in file
names). If passing a single string, either shell must be True (see
below) or else the string must simply name the program to be executed
without specifying any arguments.
Example:
import os
import subprocess
def read_war():
war_file_path = "jackrabbit-webapp-2.6.5.war"
jar_path = os.path.join(os.environ['JAVA_HOME'], 'bin', 'jar')
jar_cmd = [jar_path, 'tvf', war_file_path]
print("command to be executed is: %s" % jar_cmd)
subprocess.check_call(jar_cmd)
read_war()
I've used check_call to raise an exception if the command returns non-zero exit status.