This question already has answers here:
Python subprocess Popen: Why does "ls *.txt" not work? [duplicate]
(2 answers)
Closed 6 years ago.
>>> import subprocess
>>> child = subprocess.Popen(["ls", "examples/*"], stdout=subprocess.PIPE)
>>> ls: examples/*: No such file or directory
But from terminal it works
Beagle:kumarshubham$ ls examples/*
examples/convert_greyscale.py examples/feat_det_harris_corner.py examples/read_display_image.py
examples/example_set_roi.py examples/manipulate_img_matplotlib.py examples/remove_matplotlib_cache.py
Can one guide me where i am going wrong?
import subprocess
child = subprocess.Popen(["cd /to-your-PATH/; ls", "examples/*"],shell=True, stdout=subprocess.PIPE)
child.stdout.read()
This is because of (*) wild card usage. You need to supply shell=True to execute the command through shell interpreter
>>> import subprocess
>>> child = subprocess.Popen(["ls", "examples/*"], stdout=subprocess.PIPE, shell=True)
You should add shell=True in your Popen call even if * is useless in your case, ls examples/ should return the same output:
child = subprocess.Popen(["ls", "examples/*"], stdout=subprocess.PIPE, shell=True)
Plus, more pythonic approach could be:
import os
os.listdir('examples')
you can use additional parameter shell=True, then it would look:
child = subprocess.Popen(["ls", "examples/*"], shell=True, stdout=subprocess.PIPE)
NB! even this will work, - according to official python documentation https://docs.python.org/2/library/subprocess.html using shell=True might be a security issue.
Related
INPUTS is the variable I gave for the absolute path of a directory of possible input files. I want to check their status before going through my pipeline. So I tried:
import subprocess
import argparse
INPUTS = '/home/username/WinterResearch/Inputs'
status = subprocess.Popen(['ls', '-lh', INPUTS], shell=True, stdout=subprocess.PIPE)
stdout = status.communicate()
status.stdout.close()
I have also tried the often used
from shlx import split
import subprocess
import argparse
cmd = 'ls -lh INPUTS'
status = subprocess.Popen(cmd.split(), shell=True, stdout=subprocess.PIPE)
and
cmd = "ls -lh 'INPUTS'"
I do not receive an error code. The process simply does not output anything to the terminal window. I am not sure why the python script simply skips over this instead of stating there is an error. I do receive an error when I include close_fds=True that states int cannot use communicate(). So how can I receive an output from some ls -lh INPUTS equivalent using subprocess.Popen()?
You don't see any output because you're not printing to console stdout — it's saved into a variable (named "stdout"). Popen is overkill for this task anyway since you aren't piping the command to another. check_output should work fine with subprocess for this purpose:
import subprocess
subprocess.check_output("ls -lh {0}".format(INPUTS), shell=True)
subprocess.check_output(args, *, stdin=None, stderr=None, shell=False,
universal_newlines=False)
Run command with arguments and return its
output as a byte string.
METHOD WITH LESSER SECURITY RISK: (see warnings plastered throughout this page)
EDIT: Using communicate() can avoid the potential
shell=True security risk:
output = subprocess.Popen(["ls", "-lh", INPUTS]).communicate()[0]
print(output)
From your first snippet:
stdout = status.communicate()
status.stdout.close()
Nothing is being printed here. You may need to change it to the following (or your preferred form/format)
stdout = status.communicate()
print stdout
status.stdout.close()
This question already has answers here:
Using greater than operator with subprocess.call
(2 answers)
Closed 5 years ago.
Forewarning, question is probably more about a lack of understanding of the bsub command and login shells than python's Popen().
I am trying to submit a LSF script within a python script using subprocess.Popen()
pipe = subprocess.Popen(shlex.split("bsub < script.lsf"),
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
It seems a subprocess is properly launched and the command executed, but I get an error in stderr from the bsub command that reads "Empty job. Job not submitted."
I was worried it had something to do with subprocess launching a login-shell, so I tried to use shell=True within the Popen command to alleviate that.
pipe = subprocess.Popen("bsub < script.lsf", shell=True,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
This successfully submits the job as expected. So my question is how can I submit this job without using shell=True?
I've tried using some bsub options such as
pipe = subprocess.Popen(shlex.split("bsub -L /bin/sh < script.lsf"),
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
with the same "Empty job. Job not submitted." error returned. I feel this is close to what I am looking for, but I don't fully understanding what "The name of the login shell (must specify an absolute path" is exactly. Perhaps it is not /bin/sh on the system I am using. Is there a way to print this path?
< and > are not arguments to your command (in this case bsub), but are instructions to the shell about where stdin or stdout (respectively) should be directed before that command is started.
Thus, the appropriate replacement is to specify this redirection with a separate argument to Popen:
pipe = subprocess.Popen(['bsub', '-L', '/bin/sh'],
stdout=subprocess.PIPE, stderr=subprocess.PIPE,
stdin=open('script.lsf', 'r'))
This question already has answers here:
How to use `subprocess` command with pipes
(7 answers)
Closed 5 years ago.
I write this command on linux
netstat -ant | wc -l
but when I try to call from python with
subprocess.Popen(['netstat','-ant','|','wc','-l'])
I cant get all output, I see just result of first command (netstat -ant).
How can I process this command on python ? (note: this command gives a int as a result)
I don't know if there's any easier method but you can go like:
from subprocess import run, Popen, PIPE
sess1 = run(['netstat', 'ant'], stdout=PIPE)
sess2 = Popen(['grep', '"SYN"'], stdin=PIPE)
sess2.stdin.write(sess1.stdout)
sess2.communicate() # required?
As an example, here's what I've tried:
#!/usr/bin/env python3
from subprocess import Popen
message = "Lo! I am up on an ox."
Popen('less', shell=True).communicate(input=message)
As the last line, I also tried:
Popen('less', stdin=message, shell=True)
I can do what I want with:
Popen('echo "%s" | less' % message, shell=True)
Is there a more pythonic way of doing it?
Thanks!
#hyades answer above is certainly correct, and depending on what exactly you want might be best, but the reason your second example didn't work is because the stdin value must be file-like (just like unix). The following also works for me.
with tempfile.TemporaryFile(mode="w") as f:
f.write(message)
f.seek(0)
Popen("less", stdin=f)
import subprocess
p = subprocess.Popen('less', shell=True, stdout = subprocess.PIPE, stdin = subprocess.PIPE)
p.stdin.write('hey!!!'.encode('utf-8'))
print(p.communicate())
You can set up a PIPE for communication with the process
It is enough to add stdin=subprocess.PIPE (to redirect child's stdin) as #hyades suggested and universal_newlines=True (to enable text mode) to your code in order to pass a string to the child process:
#!/usr/bin/env python
from subprocess import Popen, PIPE
message = "Lo! I am up on an ox."
Popen(['cat'], stdin=PIPE,
universal_newlines=True).communicate(input=message)
Don't use shell=True unless you have a reason.
This question already has answers here:
How to get the stdout and stderr for subprocess.call?
(2 answers)
Closed 8 years ago.
How do I capture the stdout and stderr of a call statement,I have the following command but I am not sure its is successful or not?can anyone provide inputs?
call("export COMMANDER_SERVER=commander.company.com" ,shell=True)
You can use pretty much any method in subprocess but need to attach pipes. A simple way is in the docs:
>>> subprocess.check_output(
... "ls non_existent_file; exit 0",
... stderr=subprocess.STDOUT,
... shell=True)
'ls: non_existent_file: No such file or directory\n'
The docs also say the following about subprocess.call:
Note
Do not use stdout=PIPE or stderr=PIPE with this function as that can deadlock based on the child process output volume. Use Popen with the communicate() method when you need pipes.
I think you are looking for stdout=subprocess.PIPE
The proper way to do this is to create a Popen object. For example:
from subprocess import Popen, PIPE
proc = Popen('command arg --option', stdout=PIPE, stderr=PIPE, shell=True)
return_code = proc.wait()
stdout, stderr = proc.communicate()
print(stdout if return_code==0 else stderr)
You can find out more about Popen objects in the Python docs.