We Keep Coding

ansible filter to map array and put in json template

I have an array variable as following
registries:
- type: primary
host: r1.example.com
- type: secondary
host: r2.example.com
I want to render the host attribute only from each array item inside a json.j2 template.
I tried the following in the template :
{
"insecure-registries": {{ registries | map(attribute='host') | to_json }}
}
Unfortunately , it does not work but it throw this error while running the playbook :
AnsibleError: Unexpected templating type error occurred on ({ \n
\"graph\": \"{{ docker_home }}\",\n \"insecure-registries\" : {{
registries | map(attribute='host') | to_json }}\n}): Object of type
'generator' is not JSON serializable"}

map returns a particular object type which is not a list. You need to transform that to a list before you feed it to to_json with the list filter
{
"insecure-registries": {{ registries | map(attribute='host') | list | to_json }}
}

Ansible Setting fact with dynamic key/value

I am trying to set ansible facts from the stdout of a command task I call from another role.
Role A:
- name: example command
command: client get -s {{ service }}
register: vars_string
- name: set vars
set_fact: vars={{ vars_string.stdout.split('\n')}}
when:
- vars_string.stdout | length > 0
- name: set vars as facts
set_fact: "{{ item }}"
with_items: "{{ vars }}"
vars output:
"vars": [
"tst=ansible",
"example=values"
]
Role B:
- debug:
var: tst
Results from Role B:
Expectation: { "tst": "ansible" }
Reality: { "tst": "VARIABLE IS NOT DEFINED!" }
I have tried to spit vars into a dict and use set_fact: "{{ item.key }}" : "{{ item.value }}" as well. This returned the same results.
I want to be able to call by the variable name returned from the command in future roles. Any ideas?

Two points about your code snippet that may interest you:
There is already a split-by-newline version of the output from your command, it's vars_string.stdout_lines
I can't tell if you just chose that variable by accident, or you were trying to actually assign to the vars built-in variable, but either way, don't do that
As best I can tell, there is no supported syntax for assigning arbitrary top-level host facts from within just a task.
You have two choices: write out those variables to a file, then use include_vars: to read them in -- which will assign them as host facts, or concede to the way set_fact: wants things and be content with those dynamic variables living underneath a known key in hostfacts
We'll show the latter first, because it's shorter:
- set_fact:
my_facts: >-
{{ "{" + (vars_string.stdout_lines
| map('regex_replace', '^([^=]+)=(.+)', '"\1": "\2"')
| join(",")) + "}"
}}
when:
- vars_string.stdout | length > 0
Of course, be aware that trickery won't work if your keys or values have non-JSON friendly characters in them, but if that simple version doesn't work, ask a follow-up question, because there are a lot more tricks in that same vein
The include_vars: way is:
- tempfile:
state: file
suffix: .json
register: vars_filename
- copy:
dest: '{{ vars_filename.path }}'
content: >-
{{ "{" + (vars_string.stdout_lines
| map('regex_replace', '^([^=]+)=(.+)', '"\1": "\2"')
| join(",")) + "}"
}}
- include_vars:
file: '{{ vars_filename.path }}'
- file:
path: '{{ vars_filename.path }}'
state: absent

Random filter in jinja2 reuse the same value everytime

I'm trying to pick up a value from a list randomly in jinja2, this way:
{{ ['#EDD7C9', '#EDD7C9', '#EEE'] | random }}
It selects an element randomly, but it reuses it every next time. If I change something in my template, then it picks up a different number again.
But weirdly, if I use range, it works just fine:
{{ range(1,100) |random }}
Any ideas of how I can pick a random item from a fixed list without having to create my own filters?
Just to add a more complete example:
{% for i in range(5) %}
{{ ['#EDD7C9', '#EDD7C9', '#EEE'] | random }}
{{ range(1,100) | random }} <br/>
{% endfor %}
outputs:
#EEE 8
#EEE 61
#EEE 58
#EEE 64
#EEE 76
Any help is appreciated!
thanks :)

This appears to be a bug introduced in Jinja 2.9. I can reproduce it with any of the 2.9 releases (2.9 - 2.9.6). Jinja 2.8 and Jinja 2.8.1 work fine, and random values are mixed in the rendering:
$ cat test.py
from jinja2 import Template
template = Template('''
{%- for i in range(10) -%}
{{ ['foo', 'bar', 'baz', 'eggs', 'ham', 'spam'] | random }}{{ ' ' }}
{%- endfor %}
''')
print(template.render())
$ bin/python test.py
eggs eggs eggs eggs eggs eggs eggs eggs eggs eggs
$ bin/pip -U Jinja2==2.8.1
Collecting jinja2==2.8.1
Using cached Jinja2-2.8.1-py2.py3-none-any.whl
Requirement already up-to-date: MarkupSafe in ./lib/python3.6/site-packages (from jinja2==2.8.1)
Installing collected packages: jinja2
Found existing installation: Jinja2 2.9
Uninstalling Jinja2-2.9:
Successfully uninstalled Jinja2-2.9
Successfully installed jinja2-2.8.1
$ bin/python test.py
spam ham eggs spam spam eggs eggs baz bar foo
Note that this only applies to a list literal embedded in the template with random directly after. If you pass in the list as a variable, then you do get a random() result each time; that variable can also be set in the template with {% set ... %}. It appears the template compiler is optimising one step too far somewhere.
So for now the work-arounds are:
Use {% set seq = ['#EDD7C9', '#EDD7C9', '#EEE'] %} and {{ seq | random }}
Pass in a list as a variable into the template and apply random to that
or to downgrade to 2.8.1.
I've filed issue #739 with the project to track this. The maintainers have excellent temporal skills and have fixed this a day ahead of time. A future version will include this fix (be it 2.9.7 or 2.10).

SaltStack: Use regexps in pillar file

I would like to do some basic pillar value settings for all boxes so that I can use them later in a unified way. Our minions are usually named in this format:
<project>-<env>.<role>-<sequence>.<domain>
Example pillar/base/top.sls:
base:
'*':
- basics
'I#project:mycoolproject and I#role:nginx':
- etc.
Example pillar/base/basics/init.sls:
{% if '-live.' in grains['id'] %}
env: production
{% elif '-qa.' in grains['id'] %}
env: qa
{% elif '-staging.' in grains['id'] %}
env: staging
{% else %}
env:
{% endif %}
{% set role = re.match("(?:live|qa|staging)\.([a-z_\-]+)\-', grains['id']).group(1) -%}
role: {{ role }}
The env part obviously works but I can't get the regex working. As far as I understood there is no way to import python module (i.e. import re) in jinja template. Any suggestions how to get regex functionality available in the pillar file if possible at all?

The simple answer is, "no". There is not a way to inject regex functionality directly into the jinja environment (I'm sure there's a way to extend jinja, but anyway..)
The way I addressed this was with an external module function, id_info.explode() and an external pillar.
Enable external modules on the master:
external_modules: /srv/extmod
External modules do not require any sort of special infrastructure--they are just regular python modules (not packages, mind you--the loader doesn't currently know how to properly side-load a package yet)
Put your python+regex logic there. Return a dictionary, assembled to your your liking.
Your external module would go in /srv/extmod/modules. You can call call this function from your pillar.sls
{% id_info = __salt__[id_info.explode()] -%}
{% subcomponent = id_info['subcomponent'] -%}
{% project = id_info['project'] -%}
etc...
A couple things to know:
The salt-master has to be restarted when an external module is added or modified. There isn't a way that I know of to incite the equivalent of a saltutil.refresh_modules() call on the salt-master, so there ya go.
The external_modules directive is not just for execution modules. In this scenario, you would also create /srv/extmod/{pillar,runners,outputers,etc}.
These modules are only available on the master

How do I pass parameters to a salt state file?

I want to create a group and user using salt state files, but I do not know the group, gid, user, uid, sshkey until I need to execute the salt state file which I would like to pass in as parameters.
I have read about Pillar to create the variable. How do I create pillars before execution?
/srv/salt/group.sls:
{{ name }}:
group.present:
- gid: {{ gid }}
- system: True
Command line:
salt 'SaltStack-01' state.sls group name=awesome gid=123456

If you really want to pass in the data on the command like you can also do it like this:
{{ pillar['name'] }}:
group.present:
- gid: {{ pillar['gid'] }}
- system: True
Then on the command line you can pass in the data like this:
salt 'SaltStack-01' state.sls group pillar='{"name": "awesome", "gid": "123456"}'

You use Pillars to create "dictionaries" that you can reference into State files. I'm not sure if I'm understanding you correctly, but here's an example of what you can do:
mkdir /srv/pillar/
Create /srv/pillar/groups.sls and paste something like this into it:
groups:
first: 1234
second: 5678
These are names and GIDs of the groups you want to create.
Create /srv/pillar/top.sls so you can apply this pillar to your minions. This is very similar to a salt top file, so you can either apply it to all minions ('*') or just the one ('SaltStack-01'):
base:
'hc01*':
- groups
To test that that has worked, you can run salt '*' pillar.items and you should find the groups pillar somewhere in the output.
Now, your /srv/salt/group.sls file should look like this:
{% for group,gid in pillar.get('groups',{}).items() %}
{{ group }}:
group.present:
- gid: {{ gid }}
{% endfor %}
This is a for loop: for every group and gid in the pillar groups, do the rest. So basically, you can look at it as if the state file is running twice:
first:
group.present:
- gid: 1234
And then:
second:
group.present:
- gid: 5678
This was incorporated from this guide.

if you do not want use Pillar
you can do as:
# /srv/salt/params.yaml
name: awesome
gid: 123456
and then:
# /srv/salt/groups.sls
{% import_yaml "params.yaml" as params %}
{{ params['name'] }}:
group.present:
- gid: {{ parmas['gid'] }}
- system: True
more details:doc

Another nice way to pass (incase you don't want to use pillars Nor create a file as other answers shows) - you can pass a local environment variable to salt and read it from within the sls file, like this:
Command:
MYVAR=world salt 'SaltStack-01' state.sls somesalt # Note the env variable passed at the beginning
sls file:
# /srv/salt/somesalt.sls
foo:
cmd.run:
- name: |
echo "hello {{ salt['environ.get']('MYVAR') }}"
Will print to stdout:
hello world
Another good thing to know is that the env variable also gets passed on to any included salt states as well.