Python IDLE shows an error when I am trying to extract files using winrar(UnRAR.exe):
"Traceback (most recent call last):
File "<pyshell#32>", line 1, in <module>
response=subprocess.check_output(['"C:\\Users\\B74Z3\\Desktop\\Test\\UnRAR.exe" e -p123 "C:\\Users\\B74Z3\\Desktop\\Test\\Test.rar"'], shell=True)
File "C:\Program Files\Python 3.5\lib\subprocess.py", line 629, in check_output
**kwargs).stdout
File "C:\Program Files\Python 3.5\lib\subprocess.py", line 711, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['"C:\\Users\\B74Z3\\Desktop\\Test\\UnRAR.exe" e -p123 "C:\\Users\\B74Z3\\Desktop\\Test\\Test.rar"']' returned non-zero exit status 1"
What is the problem with the Code:
import subprocess
response=subprocess.check_output(['"C:\\Users\\B74Z3\\Desktop\\Test\\UnRAR.exe" e -p123 "C:\\Users\\B74Z3\\Desktop\\Test\\Test.rar"'], shell=True)
I'd comment on this, but I don't have enough reputation to do so.
Try running the command without the shell interface, that is,
response=subprocess.check_output(["""C:\Users\B74Z3\Desktop\Test\UnRAR.exe""", "e", "-p123', """C:\Users\B74Z3\Desktop\Test\Test.rar"""])
I've also remove the complexity of adding additional backslashes from your command by using triple quotes. This is almost more precise in that you know exactly what command and arguments is being run.
Also on windows the shell=True is not needed unless you're running a shell built in command, https://docs.python.org/3/library/subprocess.html#popen-constructor:
On Windows with shell=True, the COMSPEC environment variable specifies the default shell. The only time you need to specify shell=True on Windows is when the command you wish to execute is built into the shell (e.g. dir or copy). You do not need shell=True to run a batch file or console-based executable.
Related
This question already has answers here:
Python subprocess.run('ls',shell=True) not working on windows
(2 answers)
Closed 1 year ago.
I want to run shell scripts and git-bash commands using subprocess.run(), in python 3.7.4. When I run the simple example on the subprocess documentation page this happens:
import subprocess
subprocess.run(["ls", "-l"])
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "C:\pycharm\project\envs\lib\subprocess.py", line 472, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\pycharm\project\envs\lib\subprocess.py", line 775, in __init__
restore_signals, start_new_session)
File "C:\pycharm\project\envs\lib\subprocess.py", line 1178, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
# it also fails with shell=True
subprocess.call(["ls", "-l"], shell=True)
'ls' is not recognized as an internal or external command,
operable program or batch file.
1
The message from shell=True is a message from windows cmd, which suggests subprocess is not sending commands to git-bash.
I am using a conda environment located in the project/envs/ folder for python. I have also installed git-bash.
I also tried setting the env and got the same error.
import os
import subprocess
my_env = os.environ.copy()
my_env["PATH"] = 'C:\Program Files\Git\;' + my_env["PATH"]
subprocess.run(['git-bash.exe', 'ls', '-l'], env=my_env)
Traceback (most recent call last):
File "<input>", line 3, in <module>
File "C:\pycharm\project\envs\lib\subprocess.py", line 472, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\pycharm\project\envs\lib\subprocess.py", line 775, in __init__
restore_signals, start_new_session)
File "C:n\pycharm\project\envs\lib\subprocess.py", line 1178, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
I can get it to run by pointing at the git-bash.exe, but it returns an empty string instead of the files in my directory
import subprocess
subprocess.run(['C:\Program Files\Git\git-bash.exe', 'ls', '-l'], capture_output=True)
CompletedProcess(args=['C:\\Program Files\\Git\\git-bash.exe', 'ls', '-l'], returncode=0, stdout=b'', stderr=b'')
I would appreciate any advice on the best way to get this working as shown on the subprocess documentation page.
I found that I can run commands using ...Git\bin\bash.exe instead of the ...\Git\git-bash.exe, like this:
import subprocess
subprocess.run(['C:\Program Files\Git\\bin\\bash.exe', '-c','ls'], stdout=subprocess.PIPE)
CompletedProcess(args=['C:\\Program Files\\Git\\bin\\bash.exe', '-c', 'ls'], returncode=0, stdout=b'README.md\n__pycache__\nconda_create.sh\nenvs\nmain.py\ntest.sh\nzipped\n')
Try this
p = subprocess.Popen(("ls", "-l"), stdout=subprocess.PIPE)
nodes = subprocess.check_output(("grep"), stdin=p.stdout)
p.wait()
ls is Linux shell command for listing files and directories
dir is Windows command line command for listing files and directories
Try to run dir in Windows command line. If it works, try to run the same command using python subprocess:
import subprocess
subprocess.run(["dir"])
For a machine with Windows Operating System, Try the following
import subprocess
subprocess.run(["dir", "/p"], shell=True)
"ls" is replaced with "dir", "-l" is replaced with "/l" and the "shell" is set to true
For a machine with Linux/Mac Operating System, Try the following
import subprocess
subprocess.run(["ls", "-l"])
I was wondering how I could use the script from this page in Python:
http://www.fmwconcepts.com/imagemagick/textcleaner/index.php
When downloading it, there is no extension. I tried renaming it to "textcleaner.sh" and run the following:
import subprocess
subprocess.check_call(['textcleaner.sh', '-g', '-e', 'normalize', 'input_image.jpg', 'output_image.jpg'])
But I get this error when doing so:
Traceback (most recent call last):
File "C:/Users/ikdem/PycharmProjects/McAffeeeeeee/bash test.py", line 11, in <module>
subprocess.check_call(['textcleaner.sh', '-g', '-e', 'normalize', 'opl-small-1.jpg', 'output.jpg'])
File "C:\Users\ikdem\AppData\Local\Programs\Python\Python35\lib\subprocess.py", line 579, in check_call
retcode = call(*popenargs, **kwargs)
File "C:\Users\ikdem\AppData\Local\Programs\Python\Python35\lib\subprocess.py", line 560, in call
with Popen(*popenargs, **kwargs) as p:
File "C:\Users\ikdem\AppData\Local\Programs\Python\Python35\lib\subprocess.py", line 950, in __init__
restore_signals, start_new_session)
File "C:\Users\ikdem\AppData\Local\Programs\Python\Python35\lib\subprocess.py", line 1220, in _execute_child
startupinfo)
OSError: [WinError 193] %1 is not a valid Win32 application
On a windows system, the default shell isn't bash. Renaming to .sh won't help, and windows tries to execv your file, which isn't a Windows executable, which explains the (bad) error message from Windows.
The best thing you could do would be to:
download & install MSYS2 ex in C:\MSYS2
change your line as follows:
code:
subprocess.check_call([r'C:\MSYS2\bin\sh','-c','textcleaner.sh', '-g', '-e', 'normalize', 'input_image.jpg', 'output_image.jpg'])
Windows does not have support for parsing a Unix-style shebang within its binary loader, and will require you to invoke a copy of sh.exe or bash.exe instead, passing it the path to the script followed by arguments to the script. You can typically find these executables as part of a MinGW or MSYS installation.
You can run my script on Windows, only if you use a Unix terminal, such as installing Cygwin or using Windows 10 built in Unix and install ImageMagick.
If you run Python from a unix terminal on Windows, it can be run. See http://www.imagemagick.org/discourse-server/viewtopic.php?f=4&t=32920, where I run a simple Imagemagick command.
You can run it natively on Windows using a version of that script converted into Magick.Net. See https://github.com/dlemstra/FredsImageMagickScripts.NET
Usage of my script in either form for commercial purposes will require obtaining a license. Usage otherwise is free.
As the title says:
>>> from subprocess import check_output
>>> check_output(['ln', '~/other_folder/src/models/sc_models.py', './src/models/sc_models.py'])
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/usr/lib/python2.7/subprocess.py", line 573, in check_output
raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '['ln', '~/other_folder/src/models/sc_models.py', './src/models/sc_models.py']' returned non-zero exit status 1
>>> exit()
$ ln ~/other_folder/src/models/sc_models.py ./src/models/sc_models.py
$
How can this be? How can it succeed from the command line, but fail from the Python subprocess call?
All tips are welcome!
You need to use os.path.expanduser:
On Unix and Windows, return the argument with an initial component of ~ or ~user replaced by that user‘s home directory.
import os
os.path.expanduser('~/other_folder/src/models/sc_models.py')
In [2]: os.path.expanduser("~")
Out[2]: '/home/padraic'
Python is looking for a directory named ~ in your cwd which obviously fails. When you run the code from bash the ~ is expanded, unless you were to use shell=True where the command would be passed to the shell and the shell would expand the tilde then you would need to use os.path.expanduser or pass the whole path i.e /home/user/other_folder...... I would stick to using shell=False withos.path.expanduser("~").
Here's my copy.py:
from subprocess import call
call("copy p2.txt p3.txt")
If in command prompt I use
copy p2.txt p3.txt it copies fine.
but when I use python copy.py it gives me:
Traceback (most recent call last):
File "copy.py", line 2, in <module>
call("copy p2.txt p3.txt")
File "C:\Python27\lib\subprocess.py", line 493, in call
return Popen(*popenargs, **kwargs).wait()
File "C:\Python27\lib\subprocess.py", line 679, in __init__
errread, errwrite)
File "C:\Python27\lib\subprocess.py", line 896, in _execute_child
startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
If I replace the python call to copy with xcopy, it works fine.
Why would this be?
When subprocess.call()ing a command like in a shell, you'll need to specify shell=True as well.
from subprocess import call
call("copy p2.txt p3.txt", shell=True)
The reason you need to use shell=True in this case is that the copy command in Windows is not actually an executable but a built-in command of the shell (if memory serves right). xcopy on the other hand is a real executable (in %WINDIR%\System32, which is usually in the %PATH%), so it can be called outside of a cmd.exe shell.
In this particular instance, shutil.copy or shutil.copy2 might be viable alternatives.
Please note that using shell=True can lead to security hazards, or as the docs put it:
Warning: Using shell=True can be a security hazard. See the warning under Frequently Used Arguments for details.
I put one new alias python3.3 in my .bash_profile so as to launch easily the python3.3 version of pyzo.
I can use this alias in a terminal without any problem, but when I use something like subprocess.check_call(args = ["python3.3", onePyFile]), I have the following error.
Traceback (most recent call last):
...
File "/Library/Frameworks/pyzo2013b/lib/python3.3/subprocess.py", line 540, in check_call
retcode = call(*popenargs, **kwargs)
File "/Library/Frameworks/pyzo2013b/lib/python3.3/subprocess.py", line 521, in call
with Popen(*popenargs, **kwargs) as p:
File "/Library/Frameworks/pyzo2013b/lib/python3.3/subprocess.py", line 818, in __init__
restore_signals, start_new_session)
File "/Library/Frameworks/pyzo2013b/lib/python3.3/subprocess.py", line 1416, in _execute_child
raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'python3.3'
I guess that my alias is not seen everywhere. So how can I fix my problem ? What is the good way to build my own alias ?
If I try subprocess.check_call(args = ["python3.3", onePyFile], shell = True), I have the following error.
onePyFile.py: python3.3: command not found
Traceback (most recent call last):
File "mainFile.py", line 72, in <module>
shell = True
File "/Library/Frameworks/pyzo2013b/lib/python3.3/subprocess.py", line 545, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['python3.3', 'onePyFile.py']' returned non-zero exit status 127
If i just use subprocess.check_call(args = ["python3.3", onePyFile]) where the first line of onePyFile is #! /usr/bin/env python3.3, I have the following error.
env: python3.3: No such file or directory
Traceback (most recent call last):
...
I think that my problem is more about the symbolic link than the Python call. But I do not know what is wrong. Indeed this is the first time that I make a personal symbolic link with an alias.
try subprocess.check_call(args = ["python3.3", onePyFile] , shell=True, env={'ENV':path_of_bash_profile})
If shell is True, the specified command will be executed through the shell. This can be useful if you are using Python primarily for the enhanced control flow it offers over most system shells and still want convenient access to other shell features such as shell pipes, filename wildcards, environment variable expansion, and expansion of ~ to a user’s home directory. However, note that Python itself offers implementations of many shell-like features (in particular, glob, fnmatch, os.walk(), os.path.expandvars(), os.path.expanduser(), and shutil).
That's because subprocess doesn't load a shell by default (see the docs), so it doesn't get what's in your .bash_profile.
Use this:
subprocess.check_call(args = ["python3.3", onePyFile], shell=True)
Edit: Seems like glasslion was faster than me!
Edit 2: I did a bit more digging around, and found something strange. Because shell=True didn't seem to work as expected I went with a more direct approach, calling bash directly.
a.py:
from subprocess import check_call
check_call(['bash', '-c', '. ~/.bash_profile && ABC a bc'])
a.sh (what would be your python3 executable):
echo $*
I first tried with alias ABC='~/Documents/a.sh' in .bash_profile:
$ python ~/Documents/a.py
bash: ABC: command not found
Traceback (most recent call last):
File "Documents/a.py", line 6, in <module>
check_call(['bash', '-c', '. ' + expanduser('~/.bash_profile') + ' && ABC a bc'])
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 511, in check_call
subprocess.CalledProcessError: Command '['bash', '-c', '. ~/.bash_profile && ABC a bc']' returned non-zero exit status 127
Then I switched from an alias to a function: ABC() { ~/Documents/a.sh $*; }
And it worked!
$ python ~/Documents/a.py
a bc
Bottom line is I got it to work, but I don't know why! Shells aren't reliable, so the best would be to skip the shell.
We can do that by using the shebang's principle (as Fred Mitchell suggested) in a clever way:
from subprocess import check_call
check_call(['/usr/bin/env', 'python3', onePyFile])
This will work if Python 3 was installed correctly, independently of it's path (what I suppose you wanted to achieve).
Why not just put a shebang to python3.3 in the first line of onePyFile and omit running python explicitly?