I have just one command in fabfile.py:
#role('dev')
def test():
local('...')
Now, I can use --role=dev in every command, but this is extremely stupid.
What I want is to install my project in a host once, with a certain role, then use it without repeating this parameter.
I typically include the following in my fabfile.py:
if not len(env.roles):
env.roles = ["test"]
This says if env.roles is not defined (via the command line for instance) that it should be defined as "test" in my case. So in your case I would alter the above to substitute dev for test and thus you would have:
if not len(env.roles):
env.roles = ["dev"]
By doing this you should find that you get the behavior you are looking for while providing you the ability to override if you so desire at any point in the future.
EDIT: I'm editing this to include a small example fabfile.py and explanation of usage.
env.roledefs = {
'test': ['test.fabexample.com'],
'stage': ['stage.fabexample.com'],
'prod': ['web01.fabexample.com', 'web02.fabexample.com', 'web03.fabexample.com'],
}
# default role will be test
env.roles = ['test']
def git_pull():
run("git pull")
def deploy():
target = "/opt/apps/FOO"
with cd(target):
git_pull()
sudo("service apache2 restart")
Now this fabfile will allow me to deploy code to any of three different environments: "test", "stage", or "prod". I select which environment I want to deploy to via the command line:
fab -R stage deploy
or,
fab --role=stage deploy
If I do not specify a role fabric will default to 'test' due to env.roles being set. Not that fabric isn't used to do anything to the local box, instead it acts on the local box (or boxes) as defined in env.roledefs although with some modifications it could be made to work locally as well.
Typically the fabric command is used from a development box to perform these operations remotely on the testing, staging, or production boxes, therefore specifying the role via the command line is not "extremely stupid" but is by design in this case.
You can use env.roledefs to associates roles with groups of hosts.
Related
I’m using Python 3.6 and Fabric 2.4. I’m using Fabric to SSH into a server and run some commands. I need to set an environment variable for the commands being run on the remote server. The documentation indicates that something like this should work:
from fabric import task
#task(hosts=["servername"])
def do_things(c):
c.run("command_to_execute", env={"KEY": "VALUE"})
But that doesn’t work. Something like this should also be possible:
from fabric import task
#task(hosts=["servername"])
def do_things(c):
c.config.run.env = {"KEY": "VALUE"}
c.run("command_to_execute")
But that doesn’t work either. I feel like I’m missing something. Can anyone help?
I was able to do it by setting inline_ssh_env=True, and then explicitly setting the env variable, ex:
with Connection(host=hostname, user=username, inline_ssh_env=True) as c:
c.config.run.env = {"MY_VAR": "this worked"}
c.run('echo $MY_VAR')
As stated on the site of Fabric:
The root cause of this is typically because the SSH server runs non-interactive commands via a very limited shell call: /path/to/shell -c "command" (for example, OpenSSH). Most shells, when run this way, are not considered to be either interactive or login shells; and this then impacts which startup files get loaded.
You read more on this page link
So what you try to do won't work, and the solution is to pass the environment variable you want to set explicitly:
from fabric import task
#task(hosts=["servername"])
def do_things(c):
c.config.run.env = {"KEY": "VALUE"}
c.run('echo export %s >> ~/.bashrc ' % 'ENV_VAR=VALUE' )
c.run('source ~/.bashrc' )
c.run('echo $ENV_VAR') # to verify if it's set or not!
c.run("command_to_execute")
You can try that:
#task
def qa(ctx):
ctx.config.run.env['counter'] = 22
ctx.config.run.env['conn'] = Connection('qa_host')
#task
def sign(ctx):
print(ctx.config.run.env['counter'])
conn = ctx.config.run.env['conn']
conn.run('touch mike_was_here.txt')
And run:
fab2 qa sign
When creating the Connection object, try adding inline_ssh_env=True.
Quoting the documentation:
Whether to send environment variables “inline” as prefixes in front of command strings (export VARNAME=value && mycommand here), instead of trying to submit them through the SSH protocol itself (which is the default behavior). This is necessary if the remote server has a restricted AcceptEnv setting (which is the common default).
According to that part of the official doc, the connect_kwargs attribute of the Connection object is intended to replace the env dict. I use it, and it works as expected.
I have a task decorated with #roles that I would occasionally like to run on a single host (for canary-testing deploys).
from fabric.api import *
env.roledefs = {
'web-workers': ['django#worker1', 'django#worker2'],
'some-other-role': ['django#worker2'],
}
#task
#roles('web-workers')
def bogomips():
run('uptime')
The docs for #roles states that:
...barring an override on the command line, my_func will be executed against the hosts listed [in the role]...
But I can't get the "override" functionality mentioned here to work... I've tried:
$ fab bogomips -H django#worker2
$ fab bogomips -R some-other-role
but it always executes on the entire role mentioned in the decorator...
What am I missing here? How can I override where a #roles-decorated task is run?
This is actually the intended behavior, according to the Execution model's Order of Precedence, and there's a slightly different syntax that you must use in this scenario.
So here's the command that doesn't work:
$ fab bogomips -R some-other-role # fabric ignores the -R values!
And here's the version that DOES work:
$ fab bogomips:roles=some-other-role
Here's the issue: #308: #roles and #hosts decorators ignore command line options
And the docs: http://docs.fabfile.org/en/1.0.0/usage/execution.html#order-of-precedence
Per-task, command-line host lists (fab mytask:host=host1) override absolutely everything else.
Per-task, decorator-specified host lists (#hosts('host1')) override the env variables.
Globally specified host lists set in the fabfile (env.hosts = ['host1']) can override such lists set on the command-line, but only if you’re not careful (or want them to.)
Globally specified host lists set on the command-line (--hosts=host1) will initialize the env variables, but that’s it.
Setup/Contraints
I want to run an ansible play located at /tmp/run_tests.yml and I want to perform the run in a python script, not command line (these are general constraints on the problem I am working on). I;ve tried several different approaches which all feel like guess work on reverse engineering the Runner class but have been unsuccessful. I was hoping to find out if this is possible and what the code would look like.
If I want to run a single command I can simply use the Ansible API's runner:
works.py (a simple example of using the Runner with a module)
ansible.runner.Runner(**{
"pattern": '*',
"module_name": 'ping',
"inventory": webInventory,
"remote_user": self.username,
"private_key_file": self.private_key
}).run()
doesnotwork.py (trying to use runner with a play)
hosts = ["127.0.0.0.1"] #dummy ip address
webInventory = ansible.inventory.Inventory(hosts)
runner = ansible.runner.Runner(pattern="*", )
response = runner.run(**{
"pattern": '*',
"module_name": "/tmp/run_tests.yml",
"inventory": webInventory,
"remote_user": "ubuntu",
"private_key_file": "~/.ssh/id_rsa"
})
Error Produced
{'contacted': {}, 'dark': {'127.0.0.1': {'failed': True, 'msg': 'module is missing interpreter line'}}}
From the source, the error suggests a shebang is missing and since I am new to ansible I speculate passing a yml file is not an appropriate file for a module_name. What would the runner command have to look like in order to run my python play?
I'm sure you've figured this out after 3+ months, but the module_name within an Ansible Runner object should be a module available from Ansible's module index, such as "apt" or "service".
I think you're looking for Ansible's ansible-playbook equivalent, which has its own run class method.
It looks like a working example of running a Playbook programmatically might be here.
You can find examples of the CLI ansible-playbook and how it's used in Ansible's github repository.
Is it possible to set the -s command-line parameter of pytest via a fixture or any other method?
What I want to achieve is to enable the -s (capture=no) option for exactly one of my tests?
I am thinking of a way to manipulate this option from within the test, or via a fixture.
To give an example of a test that needs disabled capturing here is a really simple test:
from fabric.state import env
from fabric.operations import run
class TestCapture:
def test_fabric(self):
env.user = '<ssh_username>'
env.password = '<ssh_password>'
env.host_string = '<hostname>'
who = run('whoami').stdout # use fabric command run to invoke whoami on <hostname>
assert who == '<username>'
In this test fabric is used to login on another machine via ssh, and run the whoami command. The result should be the name of the login user.
This test fails, if pytest's capturing is enabled.
It is possible to manipulate this using request.config.pluginmanager.getplugin('capturemanager').suspendcapture() and .resumecapture().
But this is a rather rare usecase which should not be needed normally. The only two cases I can think of right now for which this is reasonable are in the pdb and pytest-timeout plugins. So explaining your usecase a bit more might be helpful.
I have a single file script for operations automation (log file downloads, stop/start several containers. User is choosing what to do via command arguments) and want to have fabric functions in the same script as well as argument parsing class and possibly some other. How do I call fabric functions from within the same python script? I do not want to use "fab" as it is.
And as a side note, I'd like to have these calls parallel as well.
This is a model class that would ideally contain all necessary fabric functions:
class fabricFuncs:
def appstate(self):
env.hosts = hosts
run('sudo /home/user/XXX.sh state')
This is launcher section:
if __name__ == "__main__":
argParser().argParse()
fabricFuncs().ihsstate()
argParser sets variables globaly using command line arguments specified (just to clarify what that part does).
Which sadly results in a failure where no hosts are defined (env.hosts should contain that inside the function...or is it too late to declare them there?)
EDIT1:
I have tried launching the fabric function using this:
for h in env.hosts:
with settings(hosts_string=user + "#" + h):
fabricFuncs().ihsstate()
It kind of works. I kind of hoped though, that I will be able to paralelize the whole process using fabric module as it is (via decorators) without wraping the whole thing in threading code.
EDIT2:
I have tried this as well:
execute(fabricFuncs().ihsstate())
Which fails with:
Fatal error: Needed to prompt for the target host connection string (host: None)
Can i put a whole env.hosts variable into "settings" above without iterating over that list with a "for" statement?
EDIT3:
I have tried editing the fab function like this to see if env.hosts are set properly:
class fabricFuncs:
def appstate(self):
env.hosts = hosts
print env.hosts
run('sudo /home/user/XXX.sh state')
And it prints out correctly, but still the "run" command fails with:
Fatal error: Needed to prompt for the target host connection string (host: None)
Use the execute command:
from fabric.api import execute
execute(argParser().argParse())
execute(fabricFuncs().ihsstate())
if you run the script without fab command env.host will set to None.
so if you want to use 'execute' you have to pass also 'hosts' parameter.
try this:
from fabric.api import execute, run
if __name__ == "__main__":
hosts = ["host1", "host2"]
execute(run('sudo /home/user/XXX.sh state'), hosts=hosts)