We Keep Coding

How to filter out data from a print out message

I have this one problem, where I print out a message response from a website(JSON response), and the response I get is this.
Here is my model with fake data:
{"token": "MTAxOTAwNjM4NjEyMzg0OTkwMQ.8hkyLV.n0ir2UA4qFE5pXen9YnPtFzgn4xP8tHmVmmkrl", "user_settings": {"locale": "en-US", "theme": "dark"}, "user_id": "101900638614857883"}
And, if I only want the value of "token" data which are this (MTAxOTAwNjM4NjEyMzg0OTkwMQ.8hkyLV.n0ir2UA4qFE5pXen9YnPtFzgn4xP8tHmVmmkrl) and I want to store it into a txt file, is there any good way to do it?
Thank you, guys!
I tried print(r.text('token')) but it did not work, since it only works on printing the category of the data's (like : Category : {"token" : 'daefafa', "user-id" : 'er121231231', more})

In python, JSON is treated as a dictionary.
To filter it use dictionary comprehension
tokenData = {key: val for key,val in data_json.items() if key == 'token'}
Full Code Snippet :
from urllib.request import urlopen
import json
url = "enter-your-url"
response = urlopen(url)
data_json = json.loads(response.read())
print(type(data_json)) # <class 'dict'>
#use dict comprehension
jsonToken = {key: val for key,val in data_json.items() if key == 'result'}
strToken = json.dumps(jsonToken)
# Only string json can be written to files
with open('data.txt','w') as file:
file.write(strToken)
file.close()

You need to parse the JSON into a dictionary using json.loads(). Like this:
import json
# ...
# request-getting code
# ...
data = json.loads(r.text)
print(data['token'])

stop flask duplicating a loaded variable

I'm building a basic cloud infrastructure management site and have a problem with the page that lists virtual machines.
The flask app pulls a list that is generated via various cloud platform's APIs, and is in the below format:
vm_list = {
'vmid': [],
'name': [],
'state': [],
'platform': []
}
the list is populated by looping through the API output and appending each value like so:
def zip_list():
...
for node in driver.list_nodes():
vm_list["vmid"].append(node.uuid)
vm_list["name"].append(node.name)
vm_list["state"].append(node.state)
vm_list["platform"].append(driver.name)
...
myVms = zip(vm_list['name'], vm_list['vmid'], vm_list['platform'], vm_list['state'])
return myVms
I'm loading this via my flask app like this:
#app.route('/vms/')
def vms():
myVms = {}
myVms = vm.zip_list()
return render_template('VMs.html', vm_list=myVms)
The VMs.html page loads this data into a table:
<table class="tableClass">
<tr>
<th>Name</th>
<th>id</th>
<th>Plaform</th>
<th>State</th>
</tr>
{% for row in vm_list %}
<tr>
<td>{{ row[0] }}</td>
<td>{{ row[1] }}</td>
<td>{{ row[2] }}</td>
<td>{{ row[3] }}</td>
<tr>
{% endfor %}
</table>
And this works fine, loading the data as expected. However my problem is each time I refresh the page, the data is loaded and appended to the list once again, doubling the table size. Each refresh adds the whole vm_list list to the table once more.
I had thought this may be resolved by "nulling" the myVms variable each time it's called (i.e. myVms = {}) in the flask app script and/or the zip_list function but that doesn't seem to work; the issue still persists.
I also looked into flask-caching to see if clearing flask's cache each reload would fix it but it appears not to.
I'm thinking that I can change something in the html file to force this to only load once per session or something similar, but my front-end skills don't reach out that far.
Does anyone have any idea what I can do in this situation or where I'm going wrong? Any advice is greatly appreciated.

You are close - the variable you actually need to reset each time is not myVms but vm_list, as follows:
class Node:
counter = 0
def __init__(self):
c_str = str(Node.counter)
self.uuid = "asdf" + c_str
self.name = "test " + c_str
self.state = "wow " + c_str + " such state"
Node.counter += 1
class Driver:
def __init__(self, number_of_nodes):
self.nodes = []
for x in range(number_of_nodes):
self.nodes.append(Node())
self.name = "the greatest driver"
def list_nodes(self) -> list:
return self.nodes
driver = Driver(10)
def zip_list():
vm_list = {'vmid': [], 'name': [], 'state': [], 'platform': []}
for node in driver.list_nodes():
vm_list["vmid"].append(node.uuid)
vm_list["name"].append(node.name)
vm_list["state"].append(node.state)
vm_list["platform"].append(driver.name)
myVms = zip(vm_list['name'], vm_list['vmid'], vm_list['platform'], vm_list['state'])
return myVms
print("First time:")
my_list = zip_list()
for i in my_list:
print(i)
print("Second time:")
my_list = zip_list()
for i in my_list:
print(i)
If you initialise vm_list outside of the zip_list() function instead, you will see the doubling up that you are experiencing.

You need to initialise vm_list with an empty dict. And if a key exists, then append to its list, else set the dict[key] with an empty list. This is done by setdefault.
Try this:
def zip_list():
...
vm_list = {}
for node in driver.list_nodes():
vm_list.setdefault('vmid', []).append(node.uuid)
vm_list.setdefault('name', []).append(node.name)
vm_list.setdefault('state', []).append(node.state)
vm_list.setdefault('platform', []).append(node.platform)
...
myVms = zip(vm_list['name'], vm_list['vmid'], vm_list['platform'], vm_list['state'])
return myVms

Converting CSV to HTML Table in Python

I'm trying to take data from a .csv file and importing into a HTML table within python.
This is the csv file https://www.mediafire.com/?mootyaa33bmijiq
Context:
The csv is populated with data from a football team [Age group, Round, Opposition, Team Score, Opposition Score, Location]. I need to be able to select a specific age group and only display those details in separate tables.
This is all I've got so far....
infile = open("Crushers.csv","r")
for line in infile:
row = line.split(",")
age = row[0]
week = row [1]
opp = row[2]
ACscr = row[3]
OPPscr = row[4]
location = row[5]
if age == 'U12':
print(week, opp, ACscr, OPPscr, location)

First install pandas:
pip install pandas
Then run:
import pandas as pd
columns = ['age', 'week', 'opp', 'ACscr', 'OPPscr', 'location']
df = pd.read_csv('Crushers.csv', names=columns)
# This you can change it to whatever you want to get
age_15 = df[df['age'] == 'U15']
# Other examples:
bye = df[df['opp'] == 'Bye']
crushed_team = df[df['ACscr'] == '0']
crushed_visitor = df[df['OPPscr'] == '0']
# Play with this
# Use the .to_html() to get your table in html
print(crushed_visitor.to_html())
You'll get something like:
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th>age</th>
<th>week</th>
<th>opp</th>
<th>ACscr</th>
<th>OPPscr</th>
<th>location</th>
</tr>
</thead>
<tbody>
<tr>
<th>34</th>
<td>U17</td>
<td>1</td>
<td>Banyo</td>
<td>52</td>
<td>0</td>
<td>Home</td>
</tr>
<tr>
<th>40</th>
<td>U17</td>
<td>7</td>
<td>Aspley</td>
<td>62</td>
<td>0</td>
<td>Home</td>
</tr>
<tr>
<th>91</th>
<td>U12</td>
<td>7</td>
<td>Rochedale</td>
<td>8</td>
<td>0</td>
<td>Home</td>
</tr>
</tbody>
</table>

Firstly, install pandas:
pip install pandas
Then,
import pandas as pd
a = pd.read_csv("Crushers.csv")
# to save as html file
# named as "Table"
a.to_html("Table.htm")
# assign it to a
# variable (string)
html_file = a.to_html()

Below function takes filename, headers(optional) and delimiter(optional) as input and converts csv to html table and returns as string.
If headers are not provided, assumes header is already present in csv file.
Converts csv file contents to HTML formatted table
def csv_to_html_table(fname,headers=None,delimiter=","):
with open(fname) as f:
content = f.readlines()
#reading file content into list
rows = [x.strip() for x in content]
table = "<table>"
#creating HTML header row if header is provided
if headers is not None:
table+= "".join(["<th>"+cell+"</th>" for cell in headers.split(delimiter)])
else:
table+= "".join(["<th>"+cell+"</th>" for cell in rows[0].split(delimiter)])
rows=rows[1:]
#Converting csv to html row by row
for row in rows:
table+= "<tr>" + "".join(["<td>"+cell+"</td>" for cell in row.split(delimiter)]) + "</tr>" + "\n"
table+="</table><br>"
return table
In your case, function call will look like this, but this will not filter out entries in csv but directly convert whole csv file to HTML table.
filename="Crushers.csv"
myheader='age,week,opp,ACscr,OPPscr,location'
html_table=csv_to_html_table(filename,myheader)
Note: To filter out entries with certain values add conditional statement in for loop.

Before you begin printing the desired rows, output some HTML to set up an appropriate table structure.
When you find a row you want to print, output it in HTML table row format.
# begin the table
print("<table>")
# column headers
print("<th>")
print("<td>Week</td>")
print("<td>Opp</td>")
print("<td>ACscr</td>")
print("<td>OPPscr</td>")
print("<td>Location</td>")
print("</th>")
infile = open("Crushers.csv","r")
for line in infile:
row = line.split(",")
age = row[0]
week = row [1]
opp = row[2]
ACscr = row[3]
OPPscr = row[4]
location = row[5]
if age == 'U12':
print("<tr>")
print("<td>%s</td>" % week)
print("<td>%s</td>" % opp)
print("<td>%s</td>" % ACscr)
print("<td>%s</td>" % OPPscr)
print("<td>%s</td>" % location)
print("</tr>")
# end the table
print("</table>")

First some imports:
import csv
from html import escape
import io
Now the building blocks - let's make one function for reading the CSV and another function for making the HTML table:
def read_csv(path, column_names):
with open(path, newline='') as f:
# why newline='': see footnote at the end of https://docs.python.org/3/library/csv.html
reader = csv.reader(f)
for row in reader:
record = {name: value for name, value in zip(column_names, row)}
yield record
def html_table(records):
# records is expected to be a list of dicts
column_names = []
# first detect all posible keys (field names) that are present in records
for record in records:
for name in record.keys():
if name not in column_names:
column_names.append(name)
# create the HTML line by line
lines = []
lines.append('<table>\n')
lines.append(' <tr>\n')
for name in column_names:
lines.append(' <th>{}</th>\n'.format(escape(name)))
lines.append(' </tr>\n')
for record in records:
lines.append(' <tr>\n')
for name in column_names:
value = record.get(name, '')
lines.append(' <td>{}</td>\n'.format(escape(value)))
lines.append(' </tr>\n')
lines.append('</table>')
# join the lines to a single string and return it
return ''.join(lines)
Now just put it together :)
records = list(read_csv('Crushers.csv', 'age week opp ACscr OPPscr location'.split()))
# Print first record to see whether we are loading correctly
print(records[0])
# Output:
# {'age': 'U13', 'week': '1', 'opp': 'Waterford', 'ACscr': '22', 'OPPscr': '36', 'location': 'Home'}
records = [r for r in records if r['age'] == 'U12']
print(html_table(records))
# Output:
# <table>
# <tr>
# <th>age</th>
# <th>week</th>
# <th>opp</th>
# <th>ACscr</th>
# <th>OPPscr</th>
# <th>location</th>
# </tr>
# <tr>
# <td>U12</td>
# <td>1</td>
# <td>Waterford</td>
# <td>0</td>
# <td>4</td>
# <td>Home</td>
# </tr>
# <tr>
# <td>U12</td>
# <td>2</td>
# <td>North Lakes</td>
# <td>12</td>
# <td>18</td>
# <td>Away</td>
# </tr>
# ...
# </table>
A few notes:
csv.reader works better than line splitting because it also handles quoted values and even quoted values with newlines
html.escape is used to escape strings that could potentially contain character < or >
it is often times easier to worh with dicts than tuples
usually the CSV files contain header (first line with column names) and could be easily loaded using csv.DictReader; but the Crushers.csv has no header (the data start from very first line) so we build the dicts ourselves in the function read_csv
both functions read_csv and html_table are generalised so they can work with any data, the column names are not "hardcoded" into them
yes, you could use pandas read_csv and to_html instead :) But it is good to know how to do it without pandas in case you need some customization. Or just as a programming exercise.

This should be working as well:
from html import HTML
import csv
def to_html(csvfile):
H = HTML()
t=H.table(border='2')
r = t.tr
with open(csvfile) as csvfile:
reader = csv.DictReader(csvfile)
for column in reader.fieldnames:
r.td(column)
for row in reader:
t.tr
for col in row.iteritems():
t.td(col[1])
return t
and call the function by passing the csv file to it.

Other answers are suggesting pandas, but that's probably overkill if formatting CSV to an HTML table is all you need. If you want to use an existing package just for this purpose, there's tabulate:
import csv
from tabulate import tabulate
with open("Crushers.csv") as file:
reader = csv.reader(file)
u12_rows = [row for row in reader if row[0] == "U12"]
print(tabulate(u12_rows, tablefmt="html"))

How Turning List into Class Object for Flask Templating

How to I turn a list of lists into a class that I can call for each object like foo.bar.spam?
list of lists:
information =[['BlueLake1','MO','North','98812'], ['BlueLake2','TX','West','65343'], ['BlueLake3','NY','sales','87645'],['RedLake1','NY','sales','58923'],['RedLake2','NY','sales','12644'],['RedLake3','KY','sales','32642']]
This would be to create variables for a very large html table using jinja2 templating in Flask.
I would want to be able to to do something like this:
{% for x in information %}
<tr>
<td>{{x.name}}</td>
<td>Via: {{x.location}} | Loop: {{x.region}}</td>
<td>{{x.idcode}}</td>
</tr>
{% endfor %}
There will be other uses then just this one template with this information, hence why I want it to be a callable class to use in other places.

Using collections.namedtuple:
>>> from collections import namedtuple
>>> Info = namedtuple('Info', ['name', 'location', 'region', 'idcode'])
>>>
>>> information =[
... ['BlueLake1','MO','North','98812'],
... ['BlueLake2','TX','West','65343'],
... ['BlueLake3','NY','sales','87645'],
... ['RedLake1','NY','sales','58923'],
... ['RedLake2','NY','sales','12644'],
... ['RedLake3','KY','sales','32642']
... ]
>>> [Info(*x) for x in information]
[Info(name='BlueLake1', location='MO', region='North', idcode='98812'),
Info(name='BlueLake2', location='TX', region='West', idcode='65343'),
Info(name='BlueLake3', location='NY', region='sales', idcode='87645'),
Info(name='RedLake1', location='NY', region='sales', idcode='58923'),
Info(name='RedLake2', location='NY', region='sales', idcode='12644'),
Info(name='RedLake3', location='KY', region='sales', idcode='32642')]

Probably the most common way is to put each of the records into a dict
info = []
for r in information:
record = dict(name=r[0], location=r[1], region=r[2], idcode=r[3])
info.append(record)
Jinja2 then allows you to use x.name etc to access the properties exactly as you do in your example.
{% for x in info %}
<tr>
<td>{{x.name}}</td>
<td>Via: {{x.location}} | Loop: {{x.region}}</td>
<td>{{x.idcode}}</td>
</tr>
{% endfor %}
NOTE this way of indexing into the data (x.name) is a jinja2 specific shortcut (though it's stolen from django templates, which probably stole it from something else).
Within python itself you'd have to do:
for x in info:
print(x['name'])
# x.name will throw an error since name isn't an 'attribute' within x
# x['name'] works because 'name' is a 'key' that we added to the dict

Loop in dictionary in HTML

I have a Python script creating a dictionary and passing it to a html page to generate a report.
in Python:
data_query= {}
data_query["service1"] = "value1"
data_query["service2"] = "value2"
return data_query
in HTML:
% for name, count in data_query:
<tr>
<td>${name}</td>
<td>${count}</td>
</tr>
% endfor
it does not work, says that it does not return enough values.
I also tried (pointed out in a comment in the other question, that I deleted by mistake):
% for name, count in dict.iteritems():
It does not give any error, but does not work. Displays nothing.
${len(dict)}
gives the right dictionary length
${len(dict.iteritems())}
does not display anything and seem to have a weird effect on my table format.
Is there a way to iterate correctly a dictionart in HTMl to display both the key and value?
EDIT: How I transfer the dictionary to the html page.
from mako.lookup import TemplateLookup
from mako.runtime import Context
from mako.exceptions import text_error_template
html_lookup = TemplateLookup(directories=[os.path.join(self.dir_name)])
html_template = html_lookup.get_template('/templates/report.html')
html_data = { 'data_queries' : data_queries }
html_ctx = Context(html_file, **html_data)
try:
html_template.render_context(html_ctx)
except:
print text_error_template().render(full=False)
html_file.close()
return
html_file.close()

% for name, count in dict.items:
<tr>
<td>${name}</td>
<td>${count}</td>
</tr>
% endfor
should probably work ... typically you dont call the fn when you pass it to a templating language... alternatively
% for name in dict:
<tr>
<td>${name}</td>
<td>${dict[name]}</td>
</tr>
% endfor
would likely also work
as an aside ... dict is a terrible variable name as it shadows the builtin dict (which might be part of your problem if that is actually your variable name)