How to remove select characters from xml parse in python / django? - python

Context
I am working on a django project and I need to loop through a nested dictionary to print the values
Here's the dictionary:
{body{u'#copyright': u'All data copyright Unitrans ASUCD/City of Davis 2015.', u'predictions': {u'#routeTitle': u'A', u'#dirTitleBecauseNoPredictions': u'Outbound to El Cemonte', u'#agencyTitle': u'Unitrans ASUCD/City of Davis', u'#stopTag': u'22258', u'#stopTitle': u'Silo Terminal & Haring Hall (WB)', u'#routeTag': u'A', u'message': [{u'#text': u'Weekend service is running Monday-Wednesday Dec. 28-30.', u'#priority': u'Normal'}, {u'#text': u'The A-line and Z-line do not run on weekends. Use O-line for weekend service.', u'#priority': u'Normal'}]}}}
I am parsing the dictionary from the following url:
http://webservices.nextbus.com/service/publicXMLFeed?command=predictions&a=unitrans&r=A&s=22258
Problem 1
I am getting trouble displaying the values of keys with '#' in them using django template tags, for example
{% for i in data%}
{% i.#copyright %}
{% endfor %}
This gives an error saying could not parse remainder.
Problem 2
One of the values has a nested dictionary in it with square brackets
[{u'#text': u'Weekend service is running Monday-Wednesday Dec. 28-30.', u'#priority': u'Normal'}, {u'#text': u'The A-line and Z-line do not run on weekends. Use O-line for weekend service.', u'#priority': u'Normal'}]
I cannot loop through this using for loop template tags
The solution I have in mind
In order to solve this and make it simpler I am looking to strip the characters '#', '[' and ']'from the xml, this would leave me with a much simpler dictionary which would be easy to loop through.
My Python Code Right Now in views.py
import xmltodict
import requests
def prediction(request, line, s_id):
url = "http://webservices.nextbus.com/service/publicXMLFeed? command=predictions&a=unitrans&r=" + line + "&s=" + s_id
data = requests.get(url)
data = xmltodict.parse(data, dict_constructor=dict)
data_dict = {}
data_dict["data"] = data
return render(request, 'routes/predictions.html', data_dict)
What I want to display on page predictions.html
Route Tag: A
Message : Weekend Service is running Monday-Wednesday Dec. 28-30.
The A-Line and Z-Line do not run on weekends. use O-Line for weekend service.
Priority: Normal
I would appreciate any inputs on this problem. Thank you for your time.

In xmltodict, the '#' symbols are there to indicate attributes of xml nodes, and the '[' and ']' are used to delimit element values that are themselves a list of values. (Here, it indicates the 'message' value is itself a list of two message objects). You can certainly try to read in the dict as raw text and scrape out what you need, but that won't take advantage of the reason most people are importing it to begin with: To organize the data and make it easier to access.
Instead of scraping the text, you can easily craft a template that would just pull the specific values from the dict that you want. Your data dict should be structured something like this:
{
body:
{
u'#copyright': u'All data copyright Unitrans ASUCD/City of Davis 2015.',
u'predictions':
{
u'#routeTitle': u'A',
u'#dirTitleBecauseNoPredictions': u'Outbound to El Cemonte',
u'#agencyTitle': u'Unitrans ASUCD/City of Davis',
u'#stopTag': u'22258',
u'#stopTitle': u'Silo Terminal & Haring Hall (WB)',
u'#routeTag': u'A',
u'message':
[
{
u'#text': u'Weekend service is running Monday-Wednesday Dec. 28-30.',
u'#priority': u'Normal'
},
{
u'#text': u'The A-line and Z-line do not run on weekends. Use O-line for weekend service.',
u'#priority': u'Normal'
}
]
}
}
}
To get the output you want, create a template tailored for this data and then just insert directly the values you need. Something like this: (apologies, I don't know django template syntax exactly)
Route Tag: {{ data_dict.body.predictions.routeTitle }}
Messages :
<ul>
{% for msg in data_dict.body.predictions.message %}
<li>{{ msg.text }} (Priority: {{ msg.priority }})</li>
{% endfor %}
</ul>

Related

python flask app, using dictionary with variables

So I'm trying to reach dictionary parameters with a variable but looks like you can't use it with the traditional way (mydict[var])
I'm passing a dictionary and a list to HTML:
#app.route('/')
def setup():
jinda = {
'first':'1234'
'second':'4567'
'third': '7890'
}
zirna = ['first', 'third']
return render_template('table.html', zirna=zirna, jinda=jinda)
Now in HTML I want to use a for loop two reach specific elements in the dictionary which is also in the list but I don't know how to do that also this one is not working 👇:
{% for i in zirna %}
<p> {{ jinda[i] }}<p>
{% endfor %}
This works for me and outputs:
<p> 1234<p>
<p> 7890<p>
However there is a syntax error in your python code. Commas should be used between each key:value when defining the dictionary:
jinda = {
'first':'1234',
'second':'4567',
'third': '7890',
}
Perhaps this is your issue, although I would expect this to throw a syntax error when launching your app.

Flask how to pass list of list to javascript [duplicate]

I am trying to pass data as JSON from a Flask route to a Jinja template rendering JavaScript. I want to iterate over the data using JavaScript. The browser shows SyntaxError: Unexpected token '&'. Expected a property name. when JSON.parse is called on the rendered data. How do I use rendered JSON data in JavaScript?
var obj = JSON.parse({{ data }})
for (i in obj){
document.write(obj[i].text + "<br />");
}
def get_nodes(node):
d = {}
if node == "Root":
d["text"] = node
else:
d["text"] = node.name
getchildren = get_children(node)
if getchildren:
d["nodes"] = [get_nodes(child) for child in getchildren]
return d
tree = get_nodes("Root")
return render_template("folder.html", data=tree)
If I just put {{ data }} in the HTML part, what I see looks correct.
{'text': 'Root', 'nodes': [{'text': u'Prosjekt3'}, {'text': u'Prosjekt4', 'nodes': [{'text': u'mappe8'}]}]}
Flask's Jinja environment automatically escapes data rendered in HTML templates. This is to avoid security issues if the dev tries to render untrusted user input.
Since you are passing a Python object to be treated as JSON, Flask provides the tojson filter which automatically dumps the data to JSON and marks it safe.
return render_template('tree.html', tree=tree)
var tree = {{ tree|tojson }};
When you just look at the data rendered in HTML, it looks correct because the browser displays the escaped symbols as the real symbols (although in this case you're seeing the string representation of a Python dict, not JSON, so there's still some issues like u markers).
Previous versions of Flask didn't mark the dumped data safe, so you might come across examples like {{ tree|tojson|safe }}, which isn't required anymore.
If you're not rendering JSON (or you already dumped the JSON to a string), you can tell Jinja that data is safe to render without escaping by using the safe filter.
# already dumped to json, so tojson would double-encode it
return render_template('tree.html', tree=json.dumps(tree))
var tree = {{ tree|safe }};
You can also wrap the string in Markup before rendering it, it's equivalent to the safe filter.
# already dumped and marked safe
return render_template('tree.html', tree=Markup(json.dumps(tree)))
var tree = {{ tree }};
If you're not passing this data to JavaScript, but using it in Jinja instead, you don't need JSON. Pass the actual Python data, don't call tojson on it, and use it as you would any other data in the template.
return render_template('tree.html', tree=tree)
{% for item in tree %}
<li>{{ item }}</li>
{% endfor %}
I could archive it using the following code sample.
<script>
console.log(JSON.parse({{json|safe}}))
</script>
The problem is that your server returns not JSON, but rendered HTML, which escapes some of the symbols with & notation.
Instead of using
return render_template("folder.html", data=tree)
try
return flask.jsonify(**tree)

Passing in variable template inputs for render_template() (Flask) [duplicate]

I am trying to pass data as JSON from a Flask route to a Jinja template rendering JavaScript. I want to iterate over the data using JavaScript. The browser shows SyntaxError: Unexpected token '&'. Expected a property name. when JSON.parse is called on the rendered data. How do I use rendered JSON data in JavaScript?
var obj = JSON.parse({{ data }})
for (i in obj){
document.write(obj[i].text + "<br />");
}
def get_nodes(node):
d = {}
if node == "Root":
d["text"] = node
else:
d["text"] = node.name
getchildren = get_children(node)
if getchildren:
d["nodes"] = [get_nodes(child) for child in getchildren]
return d
tree = get_nodes("Root")
return render_template("folder.html", data=tree)
If I just put {{ data }} in the HTML part, what I see looks correct.
{'text': 'Root', 'nodes': [{'text': u'Prosjekt3'}, {'text': u'Prosjekt4', 'nodes': [{'text': u'mappe8'}]}]}
Flask's Jinja environment automatically escapes data rendered in HTML templates. This is to avoid security issues if the dev tries to render untrusted user input.
Since you are passing a Python object to be treated as JSON, Flask provides the tojson filter which automatically dumps the data to JSON and marks it safe.
return render_template('tree.html', tree=tree)
var tree = {{ tree|tojson }};
When you just look at the data rendered in HTML, it looks correct because the browser displays the escaped symbols as the real symbols (although in this case you're seeing the string representation of a Python dict, not JSON, so there's still some issues like u markers).
Previous versions of Flask didn't mark the dumped data safe, so you might come across examples like {{ tree|tojson|safe }}, which isn't required anymore.
If you're not rendering JSON (or you already dumped the JSON to a string), you can tell Jinja that data is safe to render without escaping by using the safe filter.
# already dumped to json, so tojson would double-encode it
return render_template('tree.html', tree=json.dumps(tree))
var tree = {{ tree|safe }};
You can also wrap the string in Markup before rendering it, it's equivalent to the safe filter.
# already dumped and marked safe
return render_template('tree.html', tree=Markup(json.dumps(tree)))
var tree = {{ tree }};
If you're not passing this data to JavaScript, but using it in Jinja instead, you don't need JSON. Pass the actual Python data, don't call tojson on it, and use it as you would any other data in the template.
return render_template('tree.html', tree=tree)
{% for item in tree %}
<li>{{ item }}</li>
{% endfor %}
I could archive it using the following code sample.
<script>
console.log(JSON.parse({{json|safe}}))
</script>
The problem is that your server returns not JSON, but rendered HTML, which escapes some of the symbols with & notation.
Instead of using
return render_template("folder.html", data=tree)
try
return flask.jsonify(**tree)

Issue in storing Mysql Data From Flask to jinja (HTML) variable as json [duplicate]

I am trying to pass data as JSON from a Flask route to a Jinja template rendering JavaScript. I want to iterate over the data using JavaScript. The browser shows SyntaxError: Unexpected token '&'. Expected a property name. when JSON.parse is called on the rendered data. How do I use rendered JSON data in JavaScript?
var obj = JSON.parse({{ data }})
for (i in obj){
document.write(obj[i].text + "<br />");
}
def get_nodes(node):
d = {}
if node == "Root":
d["text"] = node
else:
d["text"] = node.name
getchildren = get_children(node)
if getchildren:
d["nodes"] = [get_nodes(child) for child in getchildren]
return d
tree = get_nodes("Root")
return render_template("folder.html", data=tree)
If I just put {{ data }} in the HTML part, what I see looks correct.
{'text': 'Root', 'nodes': [{'text': u'Prosjekt3'}, {'text': u'Prosjekt4', 'nodes': [{'text': u'mappe8'}]}]}
Flask's Jinja environment automatically escapes data rendered in HTML templates. This is to avoid security issues if the dev tries to render untrusted user input.
Since you are passing a Python object to be treated as JSON, Flask provides the tojson filter which automatically dumps the data to JSON and marks it safe.
return render_template('tree.html', tree=tree)
var tree = {{ tree|tojson }};
When you just look at the data rendered in HTML, it looks correct because the browser displays the escaped symbols as the real symbols (although in this case you're seeing the string representation of a Python dict, not JSON, so there's still some issues like u markers).
Previous versions of Flask didn't mark the dumped data safe, so you might come across examples like {{ tree|tojson|safe }}, which isn't required anymore.
If you're not rendering JSON (or you already dumped the JSON to a string), you can tell Jinja that data is safe to render without escaping by using the safe filter.
# already dumped to json, so tojson would double-encode it
return render_template('tree.html', tree=json.dumps(tree))
var tree = {{ tree|safe }};
You can also wrap the string in Markup before rendering it, it's equivalent to the safe filter.
# already dumped and marked safe
return render_template('tree.html', tree=Markup(json.dumps(tree)))
var tree = {{ tree }};
If you're not passing this data to JavaScript, but using it in Jinja instead, you don't need JSON. Pass the actual Python data, don't call tojson on it, and use it as you would any other data in the template.
return render_template('tree.html', tree=tree)
{% for item in tree %}
<li>{{ item }}</li>
{% endfor %}
I could archive it using the following code sample.
<script>
console.log(JSON.parse({{json|safe}}))
</script>
The problem is that your server returns not JSON, but rendered HTML, which escapes some of the symbols with & notation.
Instead of using
return render_template("folder.html", data=tree)
try
return flask.jsonify(**tree)

Unescape html entities from JSON in Django templates

I'm getting the response from server that is escaped:
'item':'<b> Some Data </b>'
I pass such data to template useing item= json.loads(response)
By default django templates (in Google App Engine) escapes it further,
so its double escaped in results.
I can use safe to remove one level of escaping like:
{{item|safe}}
How do i turn entities to their corresponding signs?
You can do this:
{% autoescape off %}
{{ your_text_var }}
{% endautoescape %}
Warning - THIS IS NOT A RECOMMENDED SOLUTION. You should be using autoescaping instead (check Rafael's answer).
Following should do the job.
response.replace('&', '&').replace('<', '<').replace('>', '>')
Update -
After suggestion by Jan Schär, you should rather use the following :
response.replace('<', '<').replace('>', '>').replace('&', '&')
Because, if response is &gt;, it would result in > instead of the correct >. You should resolve & in the last.

Categories