We Keep Coding

fabric context manager with sudo

Is it possible to have a context manager that just keeps the state of the previous run execution. In code:
EDIT: Not a working solution, something I expected
with sudo('. myapp'): #this runs a few things and sets many env variables
run('echo $ENV1') # $ENV1 isn't set because the sudo command ran independently
I am trying to run several commands but want to keep state between each command ?
I tried using the prefix context manager but it doesn't work with the shell_env context manager: When running this code
with shell_env(ENV1="TEST"):
with prefix(". myapp"):
run("echo $ENV2")
I expected my ENV to be set then run my application which should have set env2 but the prefix runs before the shell_env ?

Don't really understand the question asked here. Could you give a little more detail in what you are trying to accomplish. However I tried the same thing (with sudo('. myapp)) you did which threw an AttributeError __exit__ exception.
Finally I've tried the to use prefix to source the bash file and executing a sudo command line within this context, which works just fine.
#fab.task
def trythis():
with fab.prefix('. testenv'):
fab.sudo('echo $ENV1')
When executing the task I get the following output.
[host] Executing task 'trythis'
[host] sudo: echo $ENV1
[host] out: sudo password:
[host] out: testing
[host] out:
Done.
Disconnecting from host... done.

with shell_env(ENV1="TEST"):
with prefix(". myapp"):
run("echo $ENV2")
I expected my ENV to be set then run my application which should have set env2 but the prefix runs before the shell_env ?
Given fabric's documentation the code you've written will generate:
export ENV1="TEST" && . myapp && echo $ENV2
Given that myapp creates ENV2, your code should work the way you want it to work, though not all shell interpret the dot operator the same way, using source is always a better idea.
with shell_env(ENV1="TEST"):
with prefix("source myapp"):
run("echo $ENV2")
You may consider a bug in myapp though, and/or double check that all path and working directory are correctly set.

Python fabric unable to start process

I'm using python fabric to deploy binaries to an ec2 server and am attempting to run them in background (a subshell).
All the fabric commands for performing local actions, putting files, and executing remote commands w/o elevated privileges work fine. The issue I run into is when I attempt to run the binary.
with cd("deploy"):
run('mkdir log')
sudo('iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080', user="root")
result = sudo('./dbserver &', user="root") # <---- This line
print result
if result.failed:
print "Running dbserver failed"
else:
print "DBServer now running server" # this gets printed despite the binary not running
After I login to the server and ps aux | grep dbserver nothing shows up. How can I get fabric to execute the binary? The same command ./dbserver & executed from the shell does exactly what I want it to. Thanks.

This is likey reated to TTY issues, and/or that you're attempting to background a process.
Both of these are discussed in the FAQ under these two headings:
http://www.fabfile.org/faq.html#init-scripts-don-t-work
http://www.fabfile.org/faq.html#why-can-t-i-run-programs-in-the-background-with-it-makes-fabric-hang
Try making the sudo like this:
sudo('nohup ./dbserver &', user=root, pty=False)

How can I run a django management command by cron job

I am working with a django app called django-mailbox. The purpose of this is to import email messages via pop3 and other protocols and store them in a db. I want to do this at regular intervals via a chron. In the documentation http://django-mailbox.readthedocs.org/en/latest/topics/polling.html it states:
Using a cron job
You can easily consume incoming mail by running the management command named getmail (optionally with an argument of the name of the mailbox you’d like to get the mail for).:
python manage.py getmail
Now I can run this at the command line locally and it works but if this was deployed to an outside server which was only accessible by a URL how would this command be given?

If you are using a virtual env use the python binary from the virtualenv
* * * * * /path/to/virtualenv/bin/python /path/to/project/manage.py management_command

on the server machine:
$ sudo crontab -l
no crontab for root
$ sudo crontab -e
no crontab for root - using an empty one
Select an editor. To change later, run 'select-editor'.
1. /bin/ed
2. /bin/nano <---- easiest
3. /usr/bin/vim.basic
4. /usr/bin/vim.tiny
Choose 1-4 [2]:
choose your preferred editor
then see http://en.wikipedia.org/wiki/Cron for how to schedule when will the command run, direct it to some .sh file on your machine, make sure you give full path as this is going to run in root user context.
the script the cron will run may look something like:
#!/bin/bash
cd /absolute/path/to/django/project
/usr/bin/python ./manage.py getmail

Running fabric scripts as root

I am trying to use fabric to automate some administrative work that I am doing on a couple of servers. The general flow is the following:
SSH with local user
run: sudo su - to become root (providing local user password again)
Do the work as root :)
Unfortunately using run('sudo su -') blocks execution of the scripts and allows user input. When I type exit or Ctrl+D the scipt resumes, but without root privileges.
I have seen a similar problem in Switching user in Fabric but I am restricted to sudo su - because I am not allowed to change the /etc/sudoers file which contains the following line:
localuser ALL = /usr/bin/su -
I browsed the source of fabric trying to find a workaround but with no success.

Having faced the same problem as yours, (only sudo su - user allowed by admin, sudo -u user -c cmd not allowed), here's my working solution with fabric:
from ilogue.fexpect import expect, expecting, run
def sudosu(user, cmd):
cmd += ' ;exit'
prompts = []
prompts += expect('bash', cmd)
prompts += expect('assword:', env.password)
with expecting(prompts):
run('sudo su - ' + user)
def host_type():
sudosu('root', 'uname -s')

There are several solutions for your issue. First, you want to run commands using sudo. You can use the fabric method sudo instead of run that runs a shell command on a remote host, with superuser privileges(sudo ref).
For example, these commands are executed using sudo :
sudo("~/install_script.py")
sudo("mkdir /var/www/new_docroot", user="www-data")
sudo("ls /home/jdoe", user=1001)
result = sudo("ls /tmp/")
Another idea is that you want to wrap a set of commands (that need to be sudoed).
You can use Fabric context managers (ref) to do that. Particularly, you can use prefix or settings.
For example:
with settings(user='root'):
run('do something')
run('do another thing')
will ask you once the root password then execute commands as root.
You can tweek settings to store the password.

There is one solution for the following problem Sorry, user localuser is not allowed to execute '/usr/bin/su - -c /bin/bash -l -c pwd' as root on hostname.
You can try sudo('mkdir ~/test_fabric',shell=False). Using param shell to avoid the bash param -l.

Run ssh-add with Fabric in a machine

I'm running some deployment tasks with Fabric that needs to checkout/update a Mercurial repository to the machine and then execute the appropriate copying/configuration.
Every time that I instatiate a new machine (we're currently using EC2 for our infrastructure) or when I run hg pull in the machine it'll ask for my ssh key passphrase, that's a bit annoying when we need to initialize a dozen machines at a time.
I've tried to run ssh-add in Fabric when the new EC2 instance is initialized but it seems like that ssh-agent isn't running for that shell and I get a Could not open a connection to your authentication agent. message from the output of Fabric.
How would I make ssh-add work when connected to the instance by the Fabric script?

A comment on fabric's issue tracker solved this for me. It's a modified version of the lincolnloop solution. Using this "run" instead of fabric's will pipe your commands through ssh locally, allowing your local ssh-agent to provide the keys.
from fabric.api import env, roles, local, output
from fabric.operations import _shell_escape
def run(command, shell=True, pty=True):
"""
Helper function.
Runs a command with SSH agent forwarding enabled.
Note:: Fabric (and paramiko) can't forward your SSH agent.
This helper uses your system's ssh to do so.
"""
real_command = command
if shell:
cwd = env.get('cwd', '')
if cwd:
cwd = 'cd %s && ' % _shell_escape(cwd)
real_command = '%s "%s"' % (env.shell,
_shell_escape(cwd + real_command))
if output.debug:
print("[%s] run: %s" % (env.host_string, real_command))
elif output.running:
print("[%s] run: %s" % (env.host_string, command))
local("ssh -A %s '%s'" % (env.host_string, real_command))
Please note that I'm running Fabric 1.3.2, and this fix won't be needed much longer.