I am running a command on the remote machine:
remote_output = run('mysqldump --no-data --user=username --password={0} database'.format(password))
I would like to capture the output, but not have it all printed to the screen. What's the easiest way to do this?
It sounds like Managing output section is what you're looking for.
To hide the output from the console, try something like this:
from __future__ import with_statement
from fabric.api import hide, run, get
with hide('output'):
run('mysqldump --no-data test | tee test.create_table')
get('~/test.create_table', '~/test.create_table')
Belows is the sample results:
No hosts found. Please specify (single) host string for connection: 192.168.6.142
[192.168.6.142] run: mysqldump --no-data test | tee test.create_table
[192.168.6.142] download: /home/quanta/test.create_table <- /home/quanta/test.create_table
Try this if you want to hide everything from log and avoid fabric throwing exceptions when command fails:
from __future__ import with_statement
from fabric.api import env,run,hide,settings
env.host_string = 'username#servernameorip'
env.key_filename = '/path/to/key.pem'
def exec_remote_cmd(cmd):
with hide('output','running','warnings'), settings(warn_only=True):
return run(cmd)
After that, you can check commands result as shown in this example:
cmd_list = ['ls', 'lss']
for cmd in cmd_list:
result = exec_remote_cmd(cmd)
if result.succeeded:
sys.stdout.write('\n* Command succeeded: '+cmd+'\n')
sys.stdout.write(result+"\n")
else:
sys.stdout.write('\n* Command failed: '+cmd+'\n')
sys.stdout.write(result+"\n")
This will be the console output of the program (observe that there aren't log messages from fabric):
* Command succeeded: ls
Desktop espaiorgcats.sql Pictures Public Videos
Documents examples.desktop projectes scripts
Downloads Music prueba Templates
* Command failed: lss
/bin/bash: lss: command not found
For fabric==2.4.0 you can hide output using the following logic
conn = Connection(host="your-host", user="your-user")
result = conn.run('your_command', hide=True)
result.stdout.strip() # here you can get the output
As other answers allude, fabric.api doesn't exist anymore (as of writing, fabric==2.5.0) 8 years after the question. However the next most recent answer here implies providing hide=True to every .run() call is the only/accepted way to do it.
Not being satisfied I went digging for a reasonable equivalent to a context where I can specify it only once. It feels like there should still be a way using an invoke.context.Context but I didn't want to spend any longer on this, and the easiest way I could find was using invoke.config.Config, which we can access via fabric.config.Config without needing any additional imports.
>>> import fabric
>>> c = fabric.Connection(
... "foo.example.com",
... config=fabric.config.Config(overrides={"run": {"hide": True}}),
... )
>>> result = c.run("hostname")
>>> result.stdout.strip()
'foo.example.com'
As of Fabric 2.6.0 hide argument to run is not available.
Expanding on suggestions by #cfillol and #samuel-harmer, using a fabric.Config may be a simpler approach:
>>> import fabric
>>> conf = fabric.Config()
>>> conf.run.hide = True
>>> conf.run.warn = True
>>> c = fabric.Connection(
... "foo.example.com",
... config=conf
... )
>>> result = c.run("hostname")
This way no command output is printed and no exception is thrown on command failure.
As Samuel Harmer also pointed out in his answer, it is possible to manage output of the run command at the connection level.
As of version 2.7.1:
from fabric import Config, Connection
connection = Connection(
host,
config = Config(overrides = {
"run": { "hide": "stdout" }
}),
...
)
Related
I am using Fabric2 version and I don't see It has exist method in it to check if folder path has existed in the remote server. Please let me know how can I achieve this in Fabric 2 http://docs.fabfile.org/en/stable/.
I have seen a similar question Check If Path Exists Using Fabric, But this is for fabric 1.x version
You can execute the test command remotely with the -d option to test if the file exist and is a directory while passing the warn parameter to the run method so the execution doesn't stop in case of a non-zero exit status code. Then the value failed on the result will be True in case that the folder doesn't exist and False otherwise.
folder = '/path/to/folder'
if c.run('test -d {}'.format(folder), warn=True).failed:
# Folder doesn't exist
c.run('mkdir {}'.format(folder))
exists method from fabric.contrib.files was moved to patchwork.files with a small signature change, so you can use it like this:
from fabric2 import Connection
from patchwork.files import exists
conn = Connection('host')
if exists(conn, SOME_REMOTE_DIR):
do_something()
The below code is to check the existence of the file (-f), just change to '-d' to check the existence of a directory.
from fabric import Connection
c = Connection(host="host")
if c.run('test -f /opt/mydata/myfile', warn=True).failed:
do.thing()
You can find it in the Fabric 2 documentation below:
https://docs.fabfile.org/en/2.5/getting-started.html?highlight=failed#bringing-it-all-together
Hi That's not so difficult, you have to use traditional python code to check if a path already exists.
from pathlib import Path
from fabric import Connection as connection, task
import os
#task
def deploy(ctx):
parent_deploy_dir = '/var/www'
deploy_dir ='/var/www/my_folder'
host = 'REMOTE_HOST'
user = 'USER'
with connection(host=host, user=user) as c:
with c.cd(parent_deploy_dir):
if not os.path.isdir(Path(deploy_dir)):
c.run('mkdir -p ' + deploy_dir)
Running Fabric 1.10, Python 2.7 on RHEL6.5. We have a cloud environment where we provide OS layer support and there is no central LDAP so everything is done via scripting, OS level accounts and RSA keys. One thing you have to do is keep an eye on your account to ensure that your key is working but also that your password on each system is correct so you can sudo. I devised a small test specifically for this....
def sudotest(cmd="echo TEST"):
with hide('status','warnings','stderr', 'stdout','running','everything','status'), settings(warn_only=True):
env.abort_on_prompts = True
try:
result=sudo(cmd)
if (result.succeeded):
print(env.host + ":SUCCESS")
else:
print(env.host + ":FAIL")
except SystemExit:
print(env.host + ":FAIL")
except:
print(env.host + ":FAIL - catchall")
You'll notice I've been trying all kinds of settings and even doubling up as I've seen odd behavior with hiding output. For example without the double 'status' I still see 'disconnecting' messages. So I've tried throwing 'everything' and 'stderr' all over with no change in output. I added the inclusive except: just to see if it would help, no change. Also no change when trying an else:
In a good run I see this output:
[host1] Executing task 'sudotest'
host1 :SUCCESS
host2 :SUCCESS
So first observation is I'm still getting that first running message. Otherwise it's exactly what I want.
On an intentionally failed run of the same host list I get:
[host1] Executing task 'sudotest'
[host1] out: Sorry, try again.
[host1] out: sudo password:
Fatal error: Needed to prompt for a connection or sudo password (host: host1), but abort-on-prompts was set to True
Aborting.
host1 :FAIL
[host2] out: Sorry, try again.
[host2] out: sudo password:
Fatal error: Needed to prompt for a connection or sudo password (host: host2), but abort-on-prompts was set to True
Aborting.
host2 :FAIL
I've got the following env settings:
env.disable_known_hosts = True
env.skip_bad_hosts = True
env.remote_interupt = True
env.warn_only = True
env.eagerly_disconnect = True
I'm running it through fab, not python, i.e.: fab sudotest
Usually with me it's because I'm missing something simple. I can live with the first running message, it's the fatal error and the extra stderr output that turns a nice clean output into a wreck.
So what's missing? Thanks.
You can try with quiet=True:
output = sudo(cmd, quiet=True):
try with:
import fabric.api as fab
fab.output['aborts'] = False
I looked at fabric abort function and there is a check on "output.aborts"
more info here output_controls
When I run the Python code below:
import workflow
import console
import paramiko
import time
strComputer = 'server.com'
strUser = 'user'
strPwd = 'passwd'
client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect(hostname=strComputer, username=strUser, password=strPwd)
channel = client.invoke_shell()
channel.send('ls\n')
time.sleep(3)
output=channel.recv(2024)
print(output)
#Close the connection
client.close()
print('Connection closed.')
I get the desired output mixed with ugly characters:
Last login: Thu Jun 19 23:37:55 2014 from 192.168.0.10
ls
user#server:~$ ls
[0m[01;34mbin[0m Rplots1.pdf
[01;32mbtsync[0m Rplots.pdf
btsync.conf~ [01;31mrstudio-server-0.95.265-amd64.deb[0m
[01;31mbtsync_glibc23_x64.tar[0m screen.vba
[01;34mbudget[0m [01;34mshiny[0m
[01;3
Connection closed.
Can anyone explain me what is going on, and how to get a pretty output instead?
Thanks
those are terminal color codes used by ls to highlight directories, executable files etc. you can call /bin/ls (or, on some distributions, ls --color=never) explicitly to avoid calling aliases etc. and get un-colored output.
the colors are defined using those cryptic codes like [0m[01;34m.
here's how the terminal looks like when ls coloring is enabled:
I m trying to use fabric module through simple python module
remoteExc.py
from fabric.api import *
def clone_repo(IPADDRESS,USER,fPath,git_url):
env.hosts_string = IPADDRESS
env.user = USER
env.key_filename = fPath
env.disable_known_hosts = 'True'
run('git clone %s' % (git_url))
mainFile.py
from remoteExc import clone_repo
clone_repo(ipAddress,user,fPath,git_url)
When i execute it says
python mainfile.py
No hosts found. Please specify (single) host string for connection:
Please enlight me where i make a mistake
Typo. env.host_string = IPADDRESS - you've got an env.hosts_string instead.
Also, generally you run fabric via fab - unless you're trying to do something fairly non-standard, be aware that running it via python probably isn't what you want to do. See the Fabric docs for a pretty good intro.
http://docs.fabfile.org/en/1.7/tutorial.html
I'm using a crontab to run a maintenance script for my minecraft server. Most of the time it works fine, unless the crontab tries to use the restart script. If I run the restart script manually, there aren't any issues. Because I believe it's got to do with path names, I'm trying to make sure it's always doing any minecraft command FROM the minecraft directory. So I'm encasing the command in pushd/popd:
os.system("pushd /directory/path/here")
os.system("command to sent to minecraft")
os.system("popd")
Below is an interactive session taking minecraft out of the equation. A simple "ls" test. As you can see, it does not at all run the os.system command from the pushd directory, but instead from /etc/ which is the directory in which I was running python to illustrate my point.Clearly pushd isn't working via python, so I'm wondering how else I can achieve this. Thanks!
>>> def test():
... import os
... os.system("pushd /home/[path_goes_here]/minecraft")
... os.system("ls")
... os.system("popd")
...
>>> test()
~/minecraft /etc
DIR_COLORS cron.weekly gcrypt inputrc localtime mime.types ntp ppp rc3.d sasldb2 smrsh vsftpd.ftpusers
DIR_COLORS.xterm crontab gpm-root.conf iproute2 login.defs mke2fs.conf ntp.conf printcap rc4.d screenrc snmp vsftpd.tpsave
X11 csh.cshrc group issue logrotate.conf modprobe.d odbc.ini profile rc5.d scsi_id.config squirrelmail vz
adjtime csh.login group- issue.net logrotate.d motd odbcinst.ini profile.d rc6.d securetty ssh warnquota.conf
aliases cyrus.conf host.conf java lvm mtab openldap protocols redhat-release security stunnel webalizer.conf
alsa dbus-1 hosts jvm lynx-site.cfg multipath.conf opt quotagrpadmins resolv.conf selinux sudoers wgetrc
alternatives default hosts.allow jvm-commmon lynx.cfg my.cnf pam.d quotatab rndc.key sensors.conf sysconfig xinetd.conf
bashrc depmod.d hosts.deny jwhois.conf mail named.caching-nameserver.conf passwd rc rpc services sysctl.conf xinetd.d
blkid dev.d httpd krb5.conf mail.rc named.conf passwd- rc.d rpm sestatus.conf termcap yum
cron.d environment imapd.conf ld.so.cache mailcap named.rfc1912.zones pear.conf rc.local rsyslog.conf setuptool.d udev yum.conf
cron.daily exports imapd.conf.tpsave ld.so.conf mailman netplug php.d rc.sysinit rwtab shadow updatedb.conf yum.repos.d
cron.deny filesystems init.d ld.so.conf.d makedev.d netplug.d php.ini rc0.d rwtab.d shadow- vimrc
cron.hourly fonts initlog.conf libaudit.conf man.config nscd.conf pki rc1.d samba shells virc
cron.monthly fstab inittab libuser.conf maven nsswitch.conf postfix rc2.d sasl2 skel vsftpd
sh: line 0: popd: directory stack empty
===
(CentOS server with python 2.4)
In Python 2.5 and later, I think a better method would be using a context manager, like so:
import contextlib
import os
#contextlib.contextmanager
def pushd(new_dir):
previous_dir = os.getcwd()
os.chdir(new_dir)
try:
yield
finally:
os.chdir(previous_dir)
You can then use it like the following:
with pushd('somewhere'):
print os.getcwd() # "somewhere"
print os.getcwd() # "wherever you started"
By using a context manager you will be exception and return value safe: your code will always cd back to where it started from, even if you throw an exception or return from inside the context block.
You can also nest pushd calls in nested blocks, without having to rely on a global directory stack:
with pushd('somewhere'):
# do something
with pushd('another/place'):
# do something else
# do something back in "somewhere"
Each shell command runs in a separate process. It spawns a shell, executes the pushd command, and then the shell exits.
Just write the commands in the same shell script:
os.system("cd /directory/path/here; run the commands")
A nicer (perhaps) way is with the subprocess module:
from subprocess import Popen
Popen("run the commands", shell=True, cwd="/directory/path/here")
pushd and popd have some added functionality: they store previous working directories in a stack - in other words, you can pushd five times, do some stuff, and popd five times to end up where you started. You're not using that here, but it might be useful for others searching for the questions like this. This is how you can emulate it:
# initialise a directory stack
pushstack = list()
def pushdir(dirname):
global pushstack
pushstack.append(os.getcwd())
os.chdir(dirname)
def popdir():
global pushstack
os.chdir(pushstack.pop())
I don't think you can call pushd from within an os.system() call:
>>> import os
>>> ret = os.system("pushd /tmp")
sh: pushd: not found
Maybe just maybe your system actually provides a pushd binary that triggers a shell internal function (I think I've seen this on FreeBSD beforeFreeBSD has some tricks like this, but not for pushd), but the current working directory of a process cannot be influenced by other processes -- so your first system() starts a shell, runs a hypothetical pushd, starts a shell, runs ls, starts a shell, runs a hypothetical popd... none of which influence each other.
You can use os.chdir("/home/path/") to change path: http://docs.python.org/library/os.html#os-file-dir
No need to use pushd -- just use os.chdir:
>>> import os
>>> os.getcwd()
'/Users/me'
>>> os.chdir('..')
>>> os.getcwd()
'/Users'
>>> os.chdir('me')
>>> os.getcwd()
'/Users/me'
Or make a class to use with 'with'
import os
class pushd: # pylint: disable=invalid-name
__slots__ = ('_pushstack',)
def __init__(self, dirname):
self._pushstack = list()
self.pushd(dirname)
def __enter__(self):
return self
def __exit__(self, exec_type, exec_val, exc_tb) -> bool:
# skip all the intermediate directories, just go back to the original one.
if self._pushstack:
os.chdir(self._pushstack.pop(0)))
if exec_type:
return False
return True
def popd(self) -> None:
if len(self._pushstack):
os.chdir(self._pushstack.pop())
def pushd(self, dirname) -> None:
self._pushstack.append(os.getcwd())
os.chdir(dirname)
with pushd(dirname) as d:
... do stuff in that dirname
d.pushd("../..")
d.popd()
If you really need a stack, i.e. if you want to do several pushd and popd,
see naught101 above.
If not, simply do:
olddir = os.getcwd()
os.chdir('/directory/path/here')
os.system("command to sent to minecraft")
os.chdir(olddir)