This question already has answers here:
Python subprocess.run('ls',shell=True) not working on windows
(2 answers)
Closed 1 year ago.
I want to run shell scripts and git-bash commands using subprocess.run(), in python 3.7.4. When I run the simple example on the subprocess documentation page this happens:
import subprocess
subprocess.run(["ls", "-l"])
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "C:\pycharm\project\envs\lib\subprocess.py", line 472, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\pycharm\project\envs\lib\subprocess.py", line 775, in __init__
restore_signals, start_new_session)
File "C:\pycharm\project\envs\lib\subprocess.py", line 1178, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
# it also fails with shell=True
subprocess.call(["ls", "-l"], shell=True)
'ls' is not recognized as an internal or external command,
operable program or batch file.
1
The message from shell=True is a message from windows cmd, which suggests subprocess is not sending commands to git-bash.
I am using a conda environment located in the project/envs/ folder for python. I have also installed git-bash.
I also tried setting the env and got the same error.
import os
import subprocess
my_env = os.environ.copy()
my_env["PATH"] = 'C:\Program Files\Git\;' + my_env["PATH"]
subprocess.run(['git-bash.exe', 'ls', '-l'], env=my_env)
Traceback (most recent call last):
File "<input>", line 3, in <module>
File "C:\pycharm\project\envs\lib\subprocess.py", line 472, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\pycharm\project\envs\lib\subprocess.py", line 775, in __init__
restore_signals, start_new_session)
File "C:n\pycharm\project\envs\lib\subprocess.py", line 1178, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
I can get it to run by pointing at the git-bash.exe, but it returns an empty string instead of the files in my directory
import subprocess
subprocess.run(['C:\Program Files\Git\git-bash.exe', 'ls', '-l'], capture_output=True)
CompletedProcess(args=['C:\\Program Files\\Git\\git-bash.exe', 'ls', '-l'], returncode=0, stdout=b'', stderr=b'')
I would appreciate any advice on the best way to get this working as shown on the subprocess documentation page.
I found that I can run commands using ...Git\bin\bash.exe instead of the ...\Git\git-bash.exe, like this:
import subprocess
subprocess.run(['C:\Program Files\Git\\bin\\bash.exe', '-c','ls'], stdout=subprocess.PIPE)
CompletedProcess(args=['C:\\Program Files\\Git\\bin\\bash.exe', '-c', 'ls'], returncode=0, stdout=b'README.md\n__pycache__\nconda_create.sh\nenvs\nmain.py\ntest.sh\nzipped\n')
Try this
p = subprocess.Popen(("ls", "-l"), stdout=subprocess.PIPE)
nodes = subprocess.check_output(("grep"), stdin=p.stdout)
p.wait()
ls is Linux shell command for listing files and directories
dir is Windows command line command for listing files and directories
Try to run dir in Windows command line. If it works, try to run the same command using python subprocess:
import subprocess
subprocess.run(["dir"])
For a machine with Windows Operating System, Try the following
import subprocess
subprocess.run(["dir", "/p"], shell=True)
"ls" is replaced with "dir", "-l" is replaced with "/l" and the "shell" is set to true
For a machine with Linux/Mac Operating System, Try the following
import subprocess
subprocess.run(["ls", "-l"])
Related
I wanted to run external programs using python but I receive an error saying I don't have the file
the code I wrote:
import subprocess
subprocess.run(["ls", "-l"])
Output:
Traceback (most recent call last):
File "C:\Users\hahan\desktop\Pythonp\main.py", line 3, in <module>
subprocess.run(["ls", "-l"])
File "C:\Users\hahan\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 501, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\Users\hahan\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 969, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Users\hahan\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 1438, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified
I expected it to return the files in that directory
The stack trace suggests you're using Windows as the operating system. ls not something that you will typically find on a Windows machine unless using something like CygWin.
Instead, try one of these options:
# use python's standard library function instead of invoking a subprocess
import os
os.listdir()
# invoke cmd and call the `dir` command
import subprocess
subprocess.run(["cmd", "/c", "dir"])
# invoke PowerShell and call the `ls` command, which is actually an alias for `Get-ChildItem`
import subprocess
subprocess.run(["powershell", "-c", "ls"])
ls is not a Windows command. The windows analogue is dir, so you could do something like
import subprocess
subprocess.run(['cmd', '/c', 'dir'])
However, if you're really just trying to list a directory it would be much better (and portable) to use something like os.listdir()
import os
os.listdir()
or pathlib
from pathlib import Path
list(Path().iterdir())
This question already has answers here:
Calling the "source" command from subprocess.Popen
(9 answers)
Closed 8 months ago.
I am tasked with automating the process of running bash script using python. Unfortunately I am not responsible for the bash script itself, so I have no idea how it works. When I run the script directly in the terminal using source adastralrc.sh , it works perfectly and gives the desired output.
However when I try to get python to run the file by using subprocess and the exact same command as the argument:
import subprocess
commands = ['source' , 'adastralrc.sh']
p = subprocess.run(commands)
I get the following error:
Traceback (most recent call last):
File "test2.py", line 6, in <module>
p = subprocess.call(commands)
File "/usr/lib64/python3.6/subprocess.py", line 287, in call
with Popen(*popenargs, **kwargs) as p:
File "/usr/lib64/python3.6/subprocess.py", line 729, in __init__
restore_signals, start_new_session)
File "/usr/lib64/python3.6/subprocess.py", line 1364, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'source adastralrc.sh': 'source adastralrc.sh'
(venv-c3dns) [vivegab#adl20213d1bld01 vivegab] (cth01/dns_mano_dev)
I am fairly inexperienced with subprocess but I thought that it was just an improved version of os.system() and should just enter the commands as if they were being typed by a person. So why am I getting the error above and what can be done to fix this?
def shell_source(script):
"""Sometime you want to emulate the action of "source" in bash,
settings some environment variables. Here is a way to do it."""
import subprocess, os
pipe = subprocess.Popen(". %s; env" % script, stdout=subprocess.PIPE, shell=True)
output = pipe.communicate()[0]
env = dict((line.split("=", 1) for line in output.splitlines()))
os.environ.update(env)
shell_source(adastralrc.sh)
Looks like a duplicate
Python IDLE shows an error when I am trying to extract files using winrar(UnRAR.exe):
"Traceback (most recent call last):
File "<pyshell#32>", line 1, in <module>
response=subprocess.check_output(['"C:\\Users\\B74Z3\\Desktop\\Test\\UnRAR.exe" e -p123 "C:\\Users\\B74Z3\\Desktop\\Test\\Test.rar"'], shell=True)
File "C:\Program Files\Python 3.5\lib\subprocess.py", line 629, in check_output
**kwargs).stdout
File "C:\Program Files\Python 3.5\lib\subprocess.py", line 711, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['"C:\\Users\\B74Z3\\Desktop\\Test\\UnRAR.exe" e -p123 "C:\\Users\\B74Z3\\Desktop\\Test\\Test.rar"']' returned non-zero exit status 1"
What is the problem with the Code:
import subprocess
response=subprocess.check_output(['"C:\\Users\\B74Z3\\Desktop\\Test\\UnRAR.exe" e -p123 "C:\\Users\\B74Z3\\Desktop\\Test\\Test.rar"'], shell=True)
I'd comment on this, but I don't have enough reputation to do so.
Try running the command without the shell interface, that is,
response=subprocess.check_output(["""C:\Users\B74Z3\Desktop\Test\UnRAR.exe""", "e", "-p123', """C:\Users\B74Z3\Desktop\Test\Test.rar"""])
I've also remove the complexity of adding additional backslashes from your command by using triple quotes. This is almost more precise in that you know exactly what command and arguments is being run.
Also on windows the shell=True is not needed unless you're running a shell built in command, https://docs.python.org/3/library/subprocess.html#popen-constructor:
On Windows with shell=True, the COMSPEC environment variable specifies the default shell. The only time you need to specify shell=True on Windows is when the command you wish to execute is built into the shell (e.g. dir or copy). You do not need shell=True to run a batch file or console-based executable.
I am trying to create a script that will run my other python programs. I am new to subprocess module so this is a bit confusing to me.
project structure
/qe-functional
/qe
/tests
cron_functional.py
test_web_events.py
setup.sh
cron_functional.py
print(os.getcwd())
# print(subprocess.check_output('ls'))
runtag = "daily_run_" + datetime.today().strftime("%m_%d_%y")
testrun = "source ../../setup.sh; ./test_web_events.py -n 10 -t prf -E ctg-businessevent -p post {}".format(runtag)
cmd = testrun.split()
print(cmd)
subprocess.check_output(cmd)
output
$ python cron_functional.py
/Users/bli1/Development/QE/qe-functional/qe/tests
['source', '../../setup.sh;', './test_web_events.py', '-n', '10', '-t', 'prf', '-E', 'ctg-businessevent', '-p', 'post', 'daily_run_05_26_15']
Traceback (most recent call last):
File "cron_functional.py", line 11, in <module>
subprocess.check_output(cmd)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 566, in check_output
process = Popen(stdout=PIPE, *popenargs, **kwargs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 709, in __init__
errread, errwrite)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1326, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
source is an internal shell command, not an executable. What you want is not to run one source command with 11 arguments, but a one-liner shell script. You need to pass the whole script as one string to be interpreted by the shell.
subprocess.check_output(testrun, shell=True)
You haven't said what setup.sh does. If it's setting up environment variables and changing the working directory, consider doing that within Python instead. Then you can run
subprocess.check_output(['./test_web_events.py', '-n', '10', …, '-p', 'post', runtag])
… without involving the shell.
This question already has answers here:
When to use Shell=True for Python subprocess module [duplicate]
(2 answers)
Python Subprocess Call with variables [duplicate]
(1 answer)
Closed 8 months ago.
I am using Python 2.6 for reasons I cannot avoid. I have run the following tiny bit of code on the Idle command line and am getting an error I do not understand. How can I get around this?
>>> import subprocess
>>> x = subprocess.call(["dir"])
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
x = subprocess.call(["dir"])
File "C:\Python26\lib\subprocess.py", line 444, in call
return Popen(*popenargs, **kwargs).wait()
File "C:\Python26\lib\subprocess.py", line 595, in __init__
errread, errwrite)
File "C:\Python26\lib\subprocess.py", line 821, in _execute_child
startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
>>>
Try setting shell=True:
subprocess.call(["dir"], shell=True)
dir is a shell command meaning there is no executable that you could call. So dir can only be called from a shell, hence the shell=True.
Note that subprocess.call will only execute the command without giving you its output. It will only return the exit status of it (usually 0 when it was successful).
If you want to get the output, you can use subprocess.check_output:
>>> subprocess.check_output(['dir'], shell=True)
' Datentr\x84ger in Laufwerk C: ist … and more German output'
To explain why it works on Unix: There, dir is actually an executable, usually placed at /bin/dir, and as such accessible from the PATH. In Windows, dir is a feature of the command interpreter cmd.exe or the Get-ChildItem cmdlet in PowerShell (aliased to dir).