We Keep Coding

How to run bash commands using subprocess.run on windows [duplicate]

This question already has answers here:
Python subprocess.run('ls',shell=True) not working on windows
(2 answers)
Closed 1 year ago.
I want to run shell scripts and git-bash commands using subprocess.run(), in python 3.7.4. When I run the simple example on the subprocess documentation page this happens:
import subprocess
subprocess.run(["ls", "-l"])
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "C:\pycharm\project\envs\lib\subprocess.py", line 472, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\pycharm\project\envs\lib\subprocess.py", line 775, in __init__
restore_signals, start_new_session)
File "C:\pycharm\project\envs\lib\subprocess.py", line 1178, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
# it also fails with shell=True
subprocess.call(["ls", "-l"], shell=True)
'ls' is not recognized as an internal or external command,
operable program or batch file.
1
The message from shell=True is a message from windows cmd, which suggests subprocess is not sending commands to git-bash.
I am using a conda environment located in the project/envs/ folder for python. I have also installed git-bash.
I also tried setting the env and got the same error.
import os
import subprocess
my_env = os.environ.copy()
my_env["PATH"] = 'C:\Program Files\Git\;' + my_env["PATH"]
subprocess.run(['git-bash.exe', 'ls', '-l'], env=my_env)
Traceback (most recent call last):
File "<input>", line 3, in <module>
File "C:\pycharm\project\envs\lib\subprocess.py", line 472, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\pycharm\project\envs\lib\subprocess.py", line 775, in __init__
restore_signals, start_new_session)
File "C:n\pycharm\project\envs\lib\subprocess.py", line 1178, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
I can get it to run by pointing at the git-bash.exe, but it returns an empty string instead of the files in my directory
import subprocess
subprocess.run(['C:\Program Files\Git\git-bash.exe', 'ls', '-l'], capture_output=True)
CompletedProcess(args=['C:\\Program Files\\Git\\git-bash.exe', 'ls', '-l'], returncode=0, stdout=b'', stderr=b'')
I would appreciate any advice on the best way to get this working as shown on the subprocess documentation page.

I found that I can run commands using ...Git\bin\bash.exe instead of the ...\Git\git-bash.exe, like this:
import subprocess
subprocess.run(['C:\Program Files\Git\\bin\\bash.exe', '-c','ls'], stdout=subprocess.PIPE)
CompletedProcess(args=['C:\\Program Files\\Git\\bin\\bash.exe', '-c', 'ls'], returncode=0, stdout=b'README.md\n__pycache__\nconda_create.sh\nenvs\nmain.py\ntest.sh\nzipped\n')

Try this
p = subprocess.Popen(("ls", "-l"), stdout=subprocess.PIPE)
nodes = subprocess.check_output(("grep"), stdin=p.stdout)
p.wait()

ls is Linux shell command for listing files and directories
dir is Windows command line command for listing files and directories
Try to run dir in Windows command line. If it works, try to run the same command using python subprocess:
import subprocess
subprocess.run(["dir"])

For a machine with Windows Operating System, Try the following
import subprocess
subprocess.run(["dir", "/p"], shell=True)
"ls" is replaced with "dir", "-l" is replaced with "/l" and the "shell" is set to true
For a machine with Linux/Mac Operating System, Try the following
import subprocess
subprocess.run(["ls", "-l"])

Python: subprocess.Popen returns OSError: [Errno 2] No such file or directory

In my script I have the below line:
proc = subprocess.Popen(["qstat", "-xml", "-u", "*", "-r", "-s", "r"], stdout=subprocess.PIPE)
It works fine when I run it, but when it is set up as a cronjob I am getting back:
Traceback (most recent call last): File
File "/seq/software/current/bin/createOpsDashboard.py", line 171,
in get_pod_occupied_slots
proc = subprocess.Popen(["qstat", "-xml", "-u", "*", "-r", "-s", "r"], stdout=subprocess.PIPE)
File "/software/free/Linux/redhat_5_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/lib/python2.7/subprocess.py", line 672, in __init__ errread, errwrite)
File "/software/free/Linux/redhat_5_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/lib/python2.7/subprocess.py", line 1202, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
What is causing this? The cronjob is calling a shell script which load Python-2.7 and then calls the real script.

Use full path for "qstat", e.g. "/usr/local/bin/qstat". PATH environment variable for cronjob is usually restricted to default path (e.g. /bin:/usr/bin).

running separate python programs with subprocess

I am trying to create a script that will run my other python programs. I am new to subprocess module so this is a bit confusing to me.
project structure
/qe-functional
/qe
/tests
cron_functional.py
test_web_events.py
setup.sh
cron_functional.py
print(os.getcwd())
# print(subprocess.check_output('ls'))
runtag = "daily_run_" + datetime.today().strftime("%m_%d_%y")
testrun = "source ../../setup.sh; ./test_web_events.py -n 10 -t prf -E ctg-businessevent -p post {}".format(runtag)
cmd = testrun.split()
print(cmd)
subprocess.check_output(cmd)
output
$ python cron_functional.py
/Users/bli1/Development/QE/qe-functional/qe/tests
['source', '../../setup.sh;', './test_web_events.py', '-n', '10', '-t', 'prf', '-E', 'ctg-businessevent', '-p', 'post', 'daily_run_05_26_15']
Traceback (most recent call last):
File "cron_functional.py", line 11, in <module>
subprocess.check_output(cmd)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 566, in check_output
process = Popen(stdout=PIPE, *popenargs, **kwargs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 709, in __init__
errread, errwrite)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1326, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

source is an internal shell command, not an executable. What you want is not to run one source command with 11 arguments, but a one-liner shell script. You need to pass the whole script as one string to be interpreted by the shell.
subprocess.check_output(testrun, shell=True)
You haven't said what setup.sh does. If it's setting up environment variables and changing the working directory, consider doing that within Python instead. Then you can run
subprocess.check_output(['./test_web_events.py', '-n', '10', …, '-p', 'post', runtag])
… without involving the shell.

How to delete bash history in python script on Mac?

I want to delete bash history with a python script on my Macbook Pro.
I know two ways to delete bash history with bash shell
1.rm ~/.bash_history
2.history -c
But these command does not work in python script with subprocess:
1.rm ~/.bash_history
import subprocess
subprocess.call([‘rm’, ‘~/.bash_history'])
error:
rm: ~/.bash_history: No such file or directory
2.history -c
import subprocess
subprocess.call(['history', '-c'])
error:
File "test.py", line 8, in
subprocess.call(['history', '-c'])
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line >524, in call
return Popen(*popenargs, **kwargs).wait()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line >711, in init
errread, errwrite)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line >1308, in _execute_child
raise child_exception
Any ideas?

You have two questions here:
First, python doesn't understand ~, you need to expand it:
subprocess.call(['rm', os.path.expanduser('~/.bash_history')])
Second, history is a shell built-in. Use the shell to invoke it:
subprocess.call(['bash', '-c', 'history -c'])

How to invoke C executable file using Python

I have a compiled binary for Linked list written in c. I placed the executable in /usr/bin/ as /usr/bin/app where app is the name of the executable. This was compiled using gcc.
Can anyone help me to invoke this (app) using a python script.
I have written a script below to do this but seems to give errors. I am very new to python and have very basic knowledge on this. I am just exploring pythons features.
Below is the script code:
#!/usr/bin/env python
import subprocess
proc = subprocess.Popen(['\usr\bin\app'],
stdin = subprocess.PIPE,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE
)
(out, err) = proc.communicate()
print out
Here are the errors:
Traceback (most recent call last):
File "./LinkedList.py", line 7, in <module>
stderr = subprocess.PIPE
File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Thank you for your assistancce

Per Comments the answer was:
Use forward slashes '/usr/bin/app'
Personally though I would strongly consider using os.path.join or str.join and os.sep so you don't have to remember which way the slashes should go.
http://docs.python.org/2/library/os.html
http://docs.python.org/2/library/os.path.html