I'm running some deployment tasks with Fabric that needs to checkout/update a Mercurial repository to the machine and then execute the appropriate copying/configuration.
Every time that I instatiate a new machine (we're currently using EC2 for our infrastructure) or when I run hg pull in the machine it'll ask for my ssh key passphrase, that's a bit annoying when we need to initialize a dozen machines at a time.
I've tried to run ssh-add in Fabric when the new EC2 instance is initialized but it seems like that ssh-agent isn't running for that shell and I get a Could not open a connection to your authentication agent. message from the output of Fabric.
How would I make ssh-add work when connected to the instance by the Fabric script?
A comment on fabric's issue tracker solved this for me. It's a modified version of the lincolnloop solution. Using this "run" instead of fabric's will pipe your commands through ssh locally, allowing your local ssh-agent to provide the keys.
from fabric.api import env, roles, local, output
from fabric.operations import _shell_escape
def run(command, shell=True, pty=True):
"""
Helper function.
Runs a command with SSH agent forwarding enabled.
Note:: Fabric (and paramiko) can't forward your SSH agent.
This helper uses your system's ssh to do so.
"""
real_command = command
if shell:
cwd = env.get('cwd', '')
if cwd:
cwd = 'cd %s && ' % _shell_escape(cwd)
real_command = '%s "%s"' % (env.shell,
_shell_escape(cwd + real_command))
if output.debug:
print("[%s] run: %s" % (env.host_string, real_command))
elif output.running:
print("[%s] run: %s" % (env.host_string, command))
local("ssh -A %s '%s'" % (env.host_string, real_command))
Please note that I'm running Fabric 1.3.2, and this fix won't be needed much longer.
Related
Sorry if that Title is confusing. I am running a remote python script over the local network using Fabric:
#Path to file being run
env.source = 'c/yadda/yadda'
env.file = 'run_tests.py'
env.set = 'source ~USERNAME/ENVIRONMENT'
#create the task of changing the directory and running the test file from there
def link():
print('Connecting to remote computer and setting envirnoment...')
run ('%s' % env.set)
run ('cd %s && ./%s' % (env.source, env.file))
--- The env.set is used to bring the Python version up to a more recent version so that the file commands work (open with was causing problems).
THE PROBLEM is that the env.set has tsch commands such as setenv which, when run in the Bash shell, give errors. Is there any way that I can write the above to incorporate tsch commands?
Nevermind I solved it myself :)
Setting in the top:
env.shell = '/bin/rbash -l -c'
I got around this environment issue.
The entire script runs fine. I will also note that if I copy and paste the cron job into the shell and run it manually it works with no issues.
Base = '/home/user/git/'
GIT_out = Base + ("git_file.txt")
FILE_NAME = Base + 'rules/file.xml'
CD_file = open(Base + "rules/reports/CD.txt", 'r')
os.chdir(Base + 'rules')
gitFetchPull = "git fetch --all ;sleep 3 ; git pull --all"
git1 = subprocess.Popen(gitFetchPull, shell=True, stdout=subprocess.PIPE)
gitOut = git1.stdout.read()
print(gitOut)
When I read the output from cron it appears to not be able to authenticate
Received disconnect from 172.17.3.18: 2: Too many authentication failures for tyoffe4
fatal: The remote end hung up unexpectedly
error: Could not fetch origin
cron job
* * * /usr/bin/python /home/tyoffe4/git/rules/reports/cd_release.py >/home/tyoffe4/git/rules/reports/cd_release.out 2>&1
This is likely an issue of the cron environment not having the environment variables set up by your ssh agent. Therefore when git makes an ssh connection, it can't authenticate, because it can't contact your ssh agent and get keys.
This answer probably has what you're looking for:
ssh-agent and crontab -- is there a good way to get these to meet?
If for some reason it's not ssh-agent related, try print os.environ at the top of your script to dump the value of all environment variables.
Compare the output from cron and running env in your bash shell. There are likely some differences, and one of them is the source of your error.
If you set up the same environment variables in your shell as you have in cron, the behavior should reproduce.
I'm using python fabric to deploy binaries to an ec2 server and am attempting to run them in background (a subshell).
All the fabric commands for performing local actions, putting files, and executing remote commands w/o elevated privileges work fine. The issue I run into is when I attempt to run the binary.
with cd("deploy"):
run('mkdir log')
sudo('iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080', user="root")
result = sudo('./dbserver &', user="root") # <---- This line
print result
if result.failed:
print "Running dbserver failed"
else:
print "DBServer now running server" # this gets printed despite the binary not running
After I login to the server and ps aux | grep dbserver nothing shows up. How can I get fabric to execute the binary? The same command ./dbserver & executed from the shell does exactly what I want it to. Thanks.
This is likey reated to TTY issues, and/or that you're attempting to background a process.
Both of these are discussed in the FAQ under these two headings:
http://www.fabfile.org/faq.html#init-scripts-don-t-work
http://www.fabfile.org/faq.html#why-can-t-i-run-programs-in-the-background-with-it-makes-fabric-hang
Try making the sudo like this:
sudo('nohup ./dbserver &', user=root, pty=False)
I'm writing a fab script to do a git pull on a remote server, but I get Permission denied (publickey,keyboard-interactive). when fabric runs the command.
If I ssh to the server and then do the pull, it works. (I've already setup the keys on the server, so it doesn't ask for passphrases, etc.)
Here's my fabric task:
import fabric.api as fab
def update():
'''
update workers code
'''
with fab.cd('~/myrepo'):
# pull changes
print colors.cyan('Pulling changes...')
fab.run('git pull origin master')
How do I get it to work with Fabric?
Edit: My server is a Google Compute instance, and it provides a gcutil tool to ssh to the instance. This is the command it runs to connect to the server:
ssh -o UserKnownHostsFile=/dev/null -o CheckHostIP=no -o StrictHostKeyChecking=no -i /Users/John/.ssh/google_compute_engine -A -p 22 John#123.456.789.101
The script is able to connect to the server AFAICT (it's able to run commands on the server like cd and supervisor and git status), it's just git pull that fails.
you need to edit fabfile like this in order to enable ssh agent fowarding option.
from fabric.api import *
env.hosts = ['123.456.789.101']
env.user = 'John'
env.key_filename = '/Users/John/.ssh/google_compute_engine'
env.forward_agent = True
def update():
'''
update workers code
'''
with cd('~/myrepo'):
# pull changes
print colors.cyan('Pulling changes...')
run('git pull origin master')
I'm trying to run netsh command on remote windows hosts (windows domain environment with admin rights). The following code works fine on local host but I would like to run it on remote hosts as well using python.
import subprocess
netshcmd=subprocess.Popen('netsh advfirewall show rule name=\”all\”', shell=True, stderr=subprocess.PIPE, stdout=subprocess.PIPE )
output, errors = netshcmd.communicate()
The problem is that I'm no sure how/what method to use to initiate the connection to remote hosts and then run the subprocess commands. I cannot use ssh or pstools and would like try to implement it using existing pywin32 modules if possible.
I have used WMI module in a past which makes it very easy to query remote host but I couldn't find any way to query firewall policies over WMI and that's why using subprocess.
First you login the remote host machine using of pxssh modules Python: How can remote from my local pc to remoteA to remoteb to remote c using Paramiko
remote login of windows:
child = pexpect.spawn('ssh tiger#172.16.0.190 -p 8888')
child.logfile = open("/tmp/mylog", "w")
print child.before
child.expect('.*Are you sure you want to continue connecting (yes/no)?')
child.sendline("yes")
child.expect(".*assword:")
child.sendline("tiger\r")
child.expect('Press any key to continue...')
child.send('\r')
child.expect('C:\Users\.*>')
child.sendline('dir')
child.prompt('C:\Users\.*>')
Python - Pxssh - Getting an password refused error when trying to login to a remote server
and send your netsh command
I will recommend using Fabric, it's a powerful python tool with a suite of operations for executing local or remote shell commands, as well as auxiliary functionality such as prompting the running user for input, or aborting execution:
install fabric : pip install fabric
write the following script named remote_cmd.py:
"""
Usage:
python remote_cmd.py ip_address username password your_command
"""
from sys import argv
from fabric.api import run, env
def set_host_config(ip, user, password):
env.host_string = ip
env.user = user
env.password = password
def cmd(your_command):
"""
executes command remotely
"""
output = run(your_command)
return output
def main():
set_host_config(argv[1], argv[2], argv[3])
cmd(argv[4]))
if __name__ == '__main__':
main()
Usage:
python remote_cmd.py ip_address username password command