I'm trying to do the following:
output = run("ls -l backups")
for line in output.split("/n"):
do_stuff(line)
Any way of having the stdout of ls sent to output?
To be more specific I'm using a CLI app called s3cmd which does something similar to ls, but with remote Amazon S3 buckets.
So a replacement for ls won't help unfortunately.
Exactly what you are asking for should be happening. From the docs:
run will return the result of the remote program’s stdout as a single (likely multiline) string.
run(), and related commands like local() and sudo(), return an _AttributeString object that is just a wrapper around stdout with attribute access to additional information like failure/success booleans, stderr, the command run, etc. The result object also has a stdout attribute, which is just more explicit.
To troubleshoot, print type(output), output to be sure the response is what you expect. Examine output.failed and output.stderr. It could be the command isn't doing what you expect, there is no "backups" directory, etc.
Try as below using String IO
from fabric.api import *
from StringIO import StringIO
fh = StringIO()
run("ls -l backups", stdout=fh)
fh.seek(0)
for line in fh.readlines():
do_stuff(line)
In case you need to use run(), you can do it like this:
with settings(
hide('warnings', 'running', 'stdout', 'stderr'),
warn_only=True
):
command = 'ls -l backups'
output = run(command)
for line in output.splitlines():
do_stuff(line)
For local() there is a bit more simple solution:
command = 'ls -l backups'
output = local(command, capture=True)
for line in output.splitlines():
do_stuff(line)
I hope it helps.
Try split using "\r\n":
output = run("ls -l backups")
output_stdout = output.stdout.split("\r\n")
You can also use this if you are using the local() api, by setting the capture=True
#task
def login_ecr_docker():
ecr_login = local("aws ecr get-login --region us-west-2", capture=True)
docker_login = ecr_login.stdout
status = local(docker_login, capture=True)
print (status.stdout)
Just simply return it:
def output():
return run("ls -l backups")
a = execute(output, host=hostname)
print a
a will be dictionary of results.
Related
I have jar file when executed will print out the output on the terminal, Im not sure how can i pass the jar file output as a json file.
The code below just print the jar file output on the terminal
subprocess.Popen(['java', '-jar', '/home/myfolder/collect.jar'])
I'm thinking of below but no idea to start with...
with open('collect.json', 'w') as fp:
xxxxxxxxxxx
Hope someone could advise further. Thank you.
This should do the trick. If you look at the subprocess documentation you can see that check_output runs a command with arguments and return its output as a byte string.
import multiprocessing as mp
import subprocess
command = "java -jar /home/myfolder/collect.jar"
def runCommand(q):
commandOutput = subprocess.check_output(command.split()).decode("utf-8")
q.put(commandOutput)
q = mp.Queue()
commandProcess = mp.Process(target=runCommand, args=(q, ))
commandProcess.start()
output = q.get()
print(output)
with open('collect.json', w) as fp:
fp.write(output)
Didn't run the code, but should work.
You can try something like this
with open('collect.json','w') as fp :
subprocess.Popen('java -jar .//home/myfolder/collect.jar',stdout=fp).wait()
for further information, see this question How to get the output from .jar execution in python codes?
I hope this helped.
I have been looking for an answer for how to execute a java jar file through python and after looking at:
Execute .jar from Python
How can I get my python (version 2.5) script to run a jar file inside a folder instead of from command line?
How to run Python egg files directly without installing them?
I tried to do the following (both my jar and python file are in the same directory):
import os
if __name__ == "__main__":
os.system("java -jar Blender.jar")
and
import subprocess
subprocess.call(['(path)Blender.jar'])
Neither have worked. So, I was thinking that I should use Jython instead, but I think there must a be an easier way to execute jar files through python.
Do you have any idea what I may do wrong? Or, is there any other site that I study more about my problem?
I would use subprocess this way:
import subprocess
subprocess.call(['java', '-jar', 'Blender.jar'])
But, if you have a properly configured /proc/sys/fs/binfmt_misc/jar you should be able to run the jar directly, as you wrote.
So, which is exactly the error you are getting?
Please post somewhere all the output you are getting from the failed execution.
This always works for me:
from subprocess import *
def jarWrapper(*args):
process = Popen(['java', '-jar']+list(args), stdout=PIPE, stderr=PIPE)
ret = []
while process.poll() is None:
line = process.stdout.readline()
if line != '' and line.endswith('\n'):
ret.append(line[:-1])
stdout, stderr = process.communicate()
ret += stdout.split('\n')
if stderr != '':
ret += stderr.split('\n')
ret.remove('')
return ret
args = ['myJarFile.jar', 'arg1', 'arg2', 'argN'] # Any number of args to be passed to the jar file
result = jarWrapper(*args)
print result
I used the following way to execute tika jar to extract the content of a word document. It worked and I got the output also. The command I'm trying to run is "java -jar tika-app-1.24.1.jar -t 42250_EN_Upload.docx"
from subprocess import PIPE, Popen
process = Popen(['java', '-jar', 'tika-app-1.24.1.jar', '-t', '42250_EN_Upload.docx'], stdout=PIPE, stderr=PIPE)
result = process.communicate()
print(result[0].decode('utf-8'))
Here I got result as tuple, hence "result[0]". Also the string was in binary format (b-string). To convert it into normal string we need to decode with 'utf-8'.
With args: concrete example using Closure Compiler (https://developers.google.com/closure/) from python
import os
import re
src = test.js
os.execlp("java", 'blablabla', "-jar", './closure_compiler.jar', '--js', src, '--js_output_file', '{}'.format(re.sub('.js$', '.comp.js', src)))
(also see here When using os.execlp, why `python` needs `python` as argv[0])
How about using os.system() like:
os.system('java -jar blabla...')
os.system(command)
Execute the command (a string) in a subshell. This is implemented by calling the Standard C function system(), and has the same limitations. Changes to sys.stdin, etc. are not reflected in the environment of the executed command.
I'm new to Python. I have list of Unix commmands ("uname -a","uptime","df -h","ifconfig -a","chkconfig --list","netstat -rn","cat /proc/meminfo","ls -l /dev") and I want to run them and redirect the entire output to a .txt file. I searched a lot but didn't get a proper solution, or I understood things wrongly.
I'm able to get the output on stdout with this for loop but I can't redirect to a file.
def commandsoutput():
command = ("uname -a","uptime","df -h","ifconfig -a","chkconfig --list","netstat -rn","cat /proc/meminfo","ls -l /dev")
for i in command:
print (os.system(i))
commandsoutput()
os.system returns the exit code of the command, not its output. It is also deprecated.
Use subprocess instead:
import subprocess
def commandsoutput():
command = ("uname -a","uptime","df -h","ifconfig -a","chkconfig --list","netstat -rn","cat /proc/meminfo","ls -l /dev")
with open('log.txt', 'a') as outfile:
for i in command:
subprocess.call(i, stdout=outfile)
commandsoutput()
This answer uses os.popen, which allows you to write the output of the command in your file:
import os
def commandsoutput():
commands = ("uname -a","uptime","df -h","ifconfig -a","chkconfig --list","netstat -rn","cat /proc/meminfo","ls -l /dev")
with open('output.txt','a') as outfile:
for command in commands:
outfile.write(os.popen(command).read()+"\n")
commandsoutput()
I am writing a script to extract something from a specified path. I am returning those values into a variable. How can i check whether the shell command has returned something or nothing.
My Code:
def any_HE():
global config, logger, status, file_size
config = ConfigParser.RawConfigParser()
config.read('config2.cfg')
for section in sorted(config.sections(), key=str.lower):
components = dict() #start with empty dictionary for each section
#Retrieving the username and password from config for each section
if not config.has_option(section, 'server.user_name'):
continue
env.user = config.get(section, 'server.user_name')
env.password = config.get(section, 'server.password')
host = config.get(section, 'server.ip')
print "Trying to connect to {} server.....".format(section)
with settings(hide('warnings', 'running', 'stdout', 'stderr'),warn_only=True, host_string=host):
try:
files = run('ls -ltr /opt/nds')
if files!=0:
print '{}--Something'.format(section)
else:
print '{} --Nothing'.format(section)
except Exception as e:
print e
I tried checking 1 or 0 and True or false but nothing seems to be working. In some servers, the path '/opt/nds/' does not exist. So in that case, nothing will be there on files. I wanted to differentiate between something returned to files and nothing returned to files.
First, you're hiding stdout.
If you get rid of that you'll get a string with the outcome of the command on the remote host. You can then split it by os.linesep (assuming same platform), but you should also take care of other things like SSH banners and colours from the retrieved outcome.
As perror commented already, the python subprocess module offers the right tools.
https://docs.python.org/2/library/subprocess.html
For your specific problem you can use the check_output function.
The documentation gives the following example:
import subprocess
subprocess.check_output(["echo", "Hello World!"])
gives "Hello World"
plumbum is a great library for running shell commands from a python script. E.g.:
from plumbum.local import ls
from plumbum import ProcessExecutionError
cmd = ls['-ltr']['/opt/nds'] # construct the command
try:
files = cmd().splitlines() # run the command
if ...:
print ...:
except ProcessExecutionError:
# command exited with a non-zero status code
...
On top of this basic usage (and unlike the subprocess module), it also supports things like output redirection and command pipelining, and more, with easy, intuitive syntax (by overloading python operators, such as '|' for piping).
In order to get more control of the process you run, you need to use the subprocess module.
Here is an example of code:
import subprocess
task = subprocess.Popen(['ls', '-ltr', '/opt/nds'], stdout=subprocess.PIPE)
print task.communicate()
I'm asking a very similar question to this one. I'm creating a pdf using wkhtmltopdf on an Ubuntu server in Django.
from tempfile import *
from subprocess import Popen, PIPE
tempfile = gettempdir()+"/results.pdf"
papersize = 'Tabloid'
orientation = 'Landscape'
command_args = "wkhtmltopdf -O %s -s %s -T 0 -R 0 -B 0 -L 0 http://pdfurl %s" %(orientation, papersize, tempfile)
popen = Popen(command_args, stdout=PIPE, stderr=PIPE)
pdf_contents = popen.stdout().read()
popen.terminate()
popen.wait()
response = HttpResponse(pdf_contents, mimetype='application/pdf')
return response
This gives me a "no such file or directory" error on the popen = Popen... line. So I changed that line to
popen = Popen(["sh", "-c", command_args], stdout=PIPE, stderr=PIPE)
and now I get a "'file' object is not callable" error on the pdf_contents =... line.
I've also tried adding .communicate() to the popen =... line but I can't seem to locate the pdf output that way. I should add that typing the command_args line into the command line creates a pdf just fine. Can anyone point me in the right direction?
wkhtmltopdf is not outputting the contents of the PDF for Popen to read it. pdf_contents correctly contains the output of the command (nothing). You will need to read the contents of the output file if you want to return it to the client (see below), or skip the output file and make wkhtmltopdf output the contents of the pdf directly,
from tempfile import *
from subprocess import Popen, PIPE
tempfile = gettempdir()+"/results.pdf"
command_args = "/path/to/wkhtmltopdf -O %s -s %s -T 0 -R 0 -B 0 -L 0 http://pdfurl %s" % ('Landscape', 'Tabloid', tempfile)
popen = Popen(["sh", "-c", command_args])
popen.wait()
f = open(tempfile, 'r')
pdf_contents = f.read()
f.close()
return HttpResponse(pdf_contents, mimetype='application/pdf')
Your first version fails because python does not know where wkhtmltopdf is located. Python will not check your path for that. Your second version passes the command to a shell which takes care of that. You achieve the same effect by passing a shell=True argument.
The second problem (as others have noted) is that you call stdout() when you shouldn't.
The third problem is that your wkhtmltopdf command is wrong. You are doing:
wkhtmltopdf -O %s -s %s -T 0 -R 0 -B 0 -L 0 http://pdfurl tempfile/results.pdf
Instead you should pass
wkhtmltopdf -O %s -s %s -T 0 -R 0 -B 0 -L 0 http://pdfurl -
That way wkhtmltopdf will write the output to standard output and you can read it. If you pass another - as the source, you can send the html over the standard input.
The reason you're getting 'file' object is not callable is because once you have your popen object, stdout is a filehandle, not a method. Don't call it, just use it:
popen = Popen(command_args, stdout=PIPE, stderr=PIPE)
pdf_contents = popen.stdout.read()
You might want to consider changing
popen = Popen(command_args, stdout=PIPE, stderr=PIPE)
pdf_contents = popen.stdout().read()
# ...
response = ...
to
pdf_contents = subprocess.check_output(command_args.split())
response = ...
or in older versions:
process = Popen(command_args.split(), stdout=PIPE, stderr=PIPE)
pdf_contents = process.stdout.read()
response = ...
I suggest you take a look at the check_output function.
EDIT: Also, don't call terminate(), as it will kill the process without waiting it to complete, possibly resulting in a corrupted PDF. You will pretty much only need to use wait(), as it will wait for the process to complete (and thus output all it has to output). When using the check_output() function, you need not to worry about it, as it waits for the process to complete by "default".
Other than that, naming a variable with the same name as a module (I'm talking about tempfile) is a bad idea. I suggest you to change it to tmpfile and to check out NamedTemporaryFiles as it is safer to use than what you are doing right now.