In my template (XXX.sql) I have:
{{ params.etl_date if params.etl_date is not none else execution_date.strftime('%Y%m%d') }}
Then in my BigQueryOperator:
BigQueryOperator(task_id='XXX',
bigquery_conn_id=GOOGLE_CLOUD_PLATFORM_CONNECTION_ID,
write_disposition='WRITE_TRUNCATE',
create_disposition='CREATE_IF_NEEDED',
sql='XXX.sql',
schema=None,
use_legacy_sql=False,
destination_dataset_table=XXX,
params={
'etl_date': '20200605'
})
it works but with:
BigQueryOperator(task_id='XXX',
bigquery_conn_id=GOOGLE_CLOUD_PLATFORM_CONNECTION_ID,
write_disposition='WRITE_TRUNCATE',
create_disposition='CREATE_IF_NEEDED',
sql='XXX.sql',
schema=None,
use_legacy_sql=False,
destination_dataset_table=XXX,
# NO PARAMS
)
I can seem to access and format the execution date? It just returns an empty string. Any advice?
To add to Sergiy's response, it depends on where you want to make your intervention. Sergiy's is the only way for it to work with your template:
{{ params.etl_date if params.etl_date is not None else execution_date.strftime('%Y%m%d') }}
The reason why you have to provide it in this case as expressly None is owing to the the safe way your template is interpreted and rendered. That is there is an equivalent to dictionary.get(key, "") underneath params.etl_date... simply because this is safe when building string or text templates.
What you may want to do is use this expressly and change your template to be:
{{ params.etl_date if params.etl_date else execution_date.strftime('%Y%m%d') }}
This is because the empty string returned by default by params.etl_date will be evaluated by Python to be False.
In the second case you can set:
params={ 'etl_date': None}
Related
I am trying to pass data as JSON from a Flask route to a Jinja template rendering JavaScript. I want to iterate over the data using JavaScript. The browser shows SyntaxError: Unexpected token '&'. Expected a property name. when JSON.parse is called on the rendered data. How do I use rendered JSON data in JavaScript?
var obj = JSON.parse({{ data }})
for (i in obj){
document.write(obj[i].text + "<br />");
}
def get_nodes(node):
d = {}
if node == "Root":
d["text"] = node
else:
d["text"] = node.name
getchildren = get_children(node)
if getchildren:
d["nodes"] = [get_nodes(child) for child in getchildren]
return d
tree = get_nodes("Root")
return render_template("folder.html", data=tree)
If I just put {{ data }} in the HTML part, what I see looks correct.
{'text': 'Root', 'nodes': [{'text': u'Prosjekt3'}, {'text': u'Prosjekt4', 'nodes': [{'text': u'mappe8'}]}]}
Flask's Jinja environment automatically escapes data rendered in HTML templates. This is to avoid security issues if the dev tries to render untrusted user input.
Since you are passing a Python object to be treated as JSON, Flask provides the tojson filter which automatically dumps the data to JSON and marks it safe.
return render_template('tree.html', tree=tree)
var tree = {{ tree|tojson }};
When you just look at the data rendered in HTML, it looks correct because the browser displays the escaped symbols as the real symbols (although in this case you're seeing the string representation of a Python dict, not JSON, so there's still some issues like u markers).
Previous versions of Flask didn't mark the dumped data safe, so you might come across examples like {{ tree|tojson|safe }}, which isn't required anymore.
If you're not rendering JSON (or you already dumped the JSON to a string), you can tell Jinja that data is safe to render without escaping by using the safe filter.
# already dumped to json, so tojson would double-encode it
return render_template('tree.html', tree=json.dumps(tree))
var tree = {{ tree|safe }};
You can also wrap the string in Markup before rendering it, it's equivalent to the safe filter.
# already dumped and marked safe
return render_template('tree.html', tree=Markup(json.dumps(tree)))
var tree = {{ tree }};
If you're not passing this data to JavaScript, but using it in Jinja instead, you don't need JSON. Pass the actual Python data, don't call tojson on it, and use it as you would any other data in the template.
return render_template('tree.html', tree=tree)
{% for item in tree %}
<li>{{ item }}</li>
{% endfor %}
I could archive it using the following code sample.
<script>
console.log(JSON.parse({{json|safe}}))
</script>
The problem is that your server returns not JSON, but rendered HTML, which escapes some of the symbols with & notation.
Instead of using
return render_template("folder.html", data=tree)
try
return flask.jsonify(**tree)
I am trying to pass data as JSON from a Flask route to a Jinja template rendering JavaScript. I want to iterate over the data using JavaScript. The browser shows SyntaxError: Unexpected token '&'. Expected a property name. when JSON.parse is called on the rendered data. How do I use rendered JSON data in JavaScript?
var obj = JSON.parse({{ data }})
for (i in obj){
document.write(obj[i].text + "<br />");
}
def get_nodes(node):
d = {}
if node == "Root":
d["text"] = node
else:
d["text"] = node.name
getchildren = get_children(node)
if getchildren:
d["nodes"] = [get_nodes(child) for child in getchildren]
return d
tree = get_nodes("Root")
return render_template("folder.html", data=tree)
If I just put {{ data }} in the HTML part, what I see looks correct.
{'text': 'Root', 'nodes': [{'text': u'Prosjekt3'}, {'text': u'Prosjekt4', 'nodes': [{'text': u'mappe8'}]}]}
Flask's Jinja environment automatically escapes data rendered in HTML templates. This is to avoid security issues if the dev tries to render untrusted user input.
Since you are passing a Python object to be treated as JSON, Flask provides the tojson filter which automatically dumps the data to JSON and marks it safe.
return render_template('tree.html', tree=tree)
var tree = {{ tree|tojson }};
When you just look at the data rendered in HTML, it looks correct because the browser displays the escaped symbols as the real symbols (although in this case you're seeing the string representation of a Python dict, not JSON, so there's still some issues like u markers).
Previous versions of Flask didn't mark the dumped data safe, so you might come across examples like {{ tree|tojson|safe }}, which isn't required anymore.
If you're not rendering JSON (or you already dumped the JSON to a string), you can tell Jinja that data is safe to render without escaping by using the safe filter.
# already dumped to json, so tojson would double-encode it
return render_template('tree.html', tree=json.dumps(tree))
var tree = {{ tree|safe }};
You can also wrap the string in Markup before rendering it, it's equivalent to the safe filter.
# already dumped and marked safe
return render_template('tree.html', tree=Markup(json.dumps(tree)))
var tree = {{ tree }};
If you're not passing this data to JavaScript, but using it in Jinja instead, you don't need JSON. Pass the actual Python data, don't call tojson on it, and use it as you would any other data in the template.
return render_template('tree.html', tree=tree)
{% for item in tree %}
<li>{{ item }}</li>
{% endfor %}
I could archive it using the following code sample.
<script>
console.log(JSON.parse({{json|safe}}))
</script>
The problem is that your server returns not JSON, but rendered HTML, which escapes some of the symbols with & notation.
Instead of using
return render_template("folder.html", data=tree)
try
return flask.jsonify(**tree)
I am trying to pass data as JSON from a Flask route to a Jinja template rendering JavaScript. I want to iterate over the data using JavaScript. The browser shows SyntaxError: Unexpected token '&'. Expected a property name. when JSON.parse is called on the rendered data. How do I use rendered JSON data in JavaScript?
var obj = JSON.parse({{ data }})
for (i in obj){
document.write(obj[i].text + "<br />");
}
def get_nodes(node):
d = {}
if node == "Root":
d["text"] = node
else:
d["text"] = node.name
getchildren = get_children(node)
if getchildren:
d["nodes"] = [get_nodes(child) for child in getchildren]
return d
tree = get_nodes("Root")
return render_template("folder.html", data=tree)
If I just put {{ data }} in the HTML part, what I see looks correct.
{'text': 'Root', 'nodes': [{'text': u'Prosjekt3'}, {'text': u'Prosjekt4', 'nodes': [{'text': u'mappe8'}]}]}
Flask's Jinja environment automatically escapes data rendered in HTML templates. This is to avoid security issues if the dev tries to render untrusted user input.
Since you are passing a Python object to be treated as JSON, Flask provides the tojson filter which automatically dumps the data to JSON and marks it safe.
return render_template('tree.html', tree=tree)
var tree = {{ tree|tojson }};
When you just look at the data rendered in HTML, it looks correct because the browser displays the escaped symbols as the real symbols (although in this case you're seeing the string representation of a Python dict, not JSON, so there's still some issues like u markers).
Previous versions of Flask didn't mark the dumped data safe, so you might come across examples like {{ tree|tojson|safe }}, which isn't required anymore.
If you're not rendering JSON (or you already dumped the JSON to a string), you can tell Jinja that data is safe to render without escaping by using the safe filter.
# already dumped to json, so tojson would double-encode it
return render_template('tree.html', tree=json.dumps(tree))
var tree = {{ tree|safe }};
You can also wrap the string in Markup before rendering it, it's equivalent to the safe filter.
# already dumped and marked safe
return render_template('tree.html', tree=Markup(json.dumps(tree)))
var tree = {{ tree }};
If you're not passing this data to JavaScript, but using it in Jinja instead, you don't need JSON. Pass the actual Python data, don't call tojson on it, and use it as you would any other data in the template.
return render_template('tree.html', tree=tree)
{% for item in tree %}
<li>{{ item }}</li>
{% endfor %}
I could archive it using the following code sample.
<script>
console.log(JSON.parse({{json|safe}}))
</script>
The problem is that your server returns not JSON, but rendered HTML, which escapes some of the symbols with & notation.
Instead of using
return render_template("folder.html", data=tree)
try
return flask.jsonify(**tree)
play_hosts is a list of all machines for a play. I want to take these and use something like format() to rewrite them like rabbitmq#%s and then join them together with something like join(). So:
{{ play_hosts|format(???)|join(', ') }}
All the examples of format use piping where the input is the format string and not a list. Is there a way to use these (or something else) to accomplish what I want? The output should looks something like:
['rabbitmq#server1', 'rabbitmq#server2', rabbitmq#server3', ...]
The jinja2 doc describes format like this:
format(value, *args, **kwargs)
Apply python string formatting on an object:
{{ "%s - %s"|format("Hello?", "Foo!") }}
-> Hello? - Foo!
So it gives three kinds of input but doesn't describe those inputs in the example, which shows one in the pipe and the other two passed in via args. Is there a keyword arg to specify the string that's piped? Please help, python monks!
In ansible you can use regex_replace filter:
{{ play_hosts | map('regex_replace', '^(.*)$', 'rabbitmq#\\1') | list }}
I believe another way would be using the joiner global function, as you can read in http://jinja.pocoo.org/docs/2.9/templates/#list-of-global-functions:
A joiner is passed a string and will return that string every time it’s called, except the first time (in which case it returns an empty string). You can use this to join things
So your code would be something like:
[
{% set comma = joiner(",") %}
{% for host in play_hosts %}
{{ comma() }}
{{ "rabbitmq#%s"|format(host) }}
{% endfor %}
]
You can create custom filter
# /usr/share/ansible/plugins/filter/format_list.py (check filter_plugins path in ansible.cfg)
def format_list(list_, pattern):
return [pattern % s for s in list_]
class FilterModule(object):
def filters(self):
return {
'format_list': format_list,
}
and use it
{{ play_hosts | format_list('rabbitmq#%s') }}
You could simply join not only by , but also add the prefix together with it. Now that's not very pythonic or sophisticated but a very simple working solution:
[rabbitmq#{{ play_hosts | join(', rabbitmq#') }}]
If you want to format a string, through a list.
l: list = ["world", "stackoverflow"]
"Hello %s and %s"|format(*l)
I'm getting the response from server that is escaped:
'item':'<b> Some Data </b>'
I pass such data to template useing item= json.loads(response)
By default django templates (in Google App Engine) escapes it further,
so its double escaped in results.
I can use safe to remove one level of escaping like:
{{item|safe}}
How do i turn entities to their corresponding signs?
You can do this:
{% autoescape off %}
{{ your_text_var }}
{% endautoescape %}
Warning - THIS IS NOT A RECOMMENDED SOLUTION. You should be using autoescaping instead (check Rafael's answer).
Following should do the job.
response.replace('&', '&').replace('<', '<').replace('>', '>')
Update -
After suggestion by Jan Schär, you should rather use the following :
response.replace('<', '<').replace('>', '>').replace('&', '&')
Because, if response is >, it would result in > instead of the correct >. You should resolve & in the last.