We Keep Coding

python new line from array

I am using Jinja template in the frontend and in my backend I am using python using which I have an array which is of type string:
#app.route('/', methods=['POST'])
def upload_image():
match = ''
if 'files[]' not in request.files:
flash('No file part')
#file stuff
discription = ""
for file in files:
if file and allowed_file(file.filename):
#OCR related stuffs
for i in range(0, output_ocr_len):
#preprocessing
for index, row in df.iterrows():
if cord_len > 0:
height = result_img.shape[0]
width = result_img.shape[1]
for elements in range(0, cord_len):
char_length = []
predicted_pattern_category = numpy.append(
predicted_pattern_category, 'Misdirection')
[char_length.append(x)
for x in predicted_pattern_category if x not in char_length]
your_predicted_pattern_category = str(char_length)
char_length = []
predicted_pattern_type = numpy.append(
predicted_pattern_type, 'Visual Interference')
[char_length.append(x) for x in predicted_pattern_type if x not in char_length]
your_predicted_pattern_type = "" + str(char_length)
for i in range(0,len(char_length)):
print("from ML :-",char_length[i])
index_of_key = key.index(char_length[i])
discription = discription + "" + value[index_of_key]
print(discription)
match = str(match) + " "
else:
char_length = []
[char_length.append(x)
for x in predicted_pattern_category if x not in char_length]
your_predicted_pattern_category = str(char_length)
if len(your_predicted_pattern_category) < 3:
your_predicted_pattern_category=''
char_length = []
[char_length.append(x) for x in predicted_pattern_type if x not in char_length]
your_predicted_pattern_type = str(char_length)
if len(your_predicted_pattern_type) < 3:
your_predicted_pattern_type=''
for i in range(0,len(char_length)):
print("from ML :-",char_length[i])
index_of_key = key.index(char_length[i])
discription = discription + '\r\n' + value[index_of_key]
print(discription)
return render_template('uploads/results.html',
msg='Processed successfully!',
match=match,
discription=discription
filenames=output_results
)
else:
return render_template('uploads/results.html',
msg='Processed successfully!',
match=match,
filenames=file_names)
To display the description, I am using Jinja template:
<tr>
<th class="table-info">Description</th>
<td>{{ description }}</td>
</tr>
I want that the description is printed on a new line whose content is present within the "value" variable.
currently the description renders together and not in a new line:
happy: is an emotionis: an extensionmy mood: relies on people
What I want is (every sentence in a new line)
happy: is an emotionis:
an extensionmy mood:
relies on people

From what I can tell, your code comes out on one line because you're repeatedly appending to the same variable. If you really want it all in one variable description, and on separate lines, I think you need a new line char, but I think HTML may ignore them...
For your fundamental problem, tables are designed to have information on separate rows. If you want the descriptions to be on separate lines, I think separate rows is the way to go. I would personally do the for-loop in the template instead of the backend.
{% for value in values %}
<tr>
<th class="table-info">Description</th>
<td>{{ value }}</td>
</tr>
{% endfor %}

stop flask duplicating a loaded variable

I'm building a basic cloud infrastructure management site and have a problem with the page that lists virtual machines.
The flask app pulls a list that is generated via various cloud platform's APIs, and is in the below format:
vm_list = {
'vmid': [],
'name': [],
'state': [],
'platform': []
}
the list is populated by looping through the API output and appending each value like so:
def zip_list():
...
for node in driver.list_nodes():
vm_list["vmid"].append(node.uuid)
vm_list["name"].append(node.name)
vm_list["state"].append(node.state)
vm_list["platform"].append(driver.name)
...
myVms = zip(vm_list['name'], vm_list['vmid'], vm_list['platform'], vm_list['state'])
return myVms
I'm loading this via my flask app like this:
#app.route('/vms/')
def vms():
myVms = {}
myVms = vm.zip_list()
return render_template('VMs.html', vm_list=myVms)
The VMs.html page loads this data into a table:
<table class="tableClass">
<tr>
<th>Name</th>
<th>id</th>
<th>Plaform</th>
<th>State</th>
</tr>
{% for row in vm_list %}
<tr>
<td>{{ row[0] }}</td>
<td>{{ row[1] }}</td>
<td>{{ row[2] }}</td>
<td>{{ row[3] }}</td>
<tr>
{% endfor %}
</table>
And this works fine, loading the data as expected. However my problem is each time I refresh the page, the data is loaded and appended to the list once again, doubling the table size. Each refresh adds the whole vm_list list to the table once more.
I had thought this may be resolved by "nulling" the myVms variable each time it's called (i.e. myVms = {}) in the flask app script and/or the zip_list function but that doesn't seem to work; the issue still persists.
I also looked into flask-caching to see if clearing flask's cache each reload would fix it but it appears not to.
I'm thinking that I can change something in the html file to force this to only load once per session or something similar, but my front-end skills don't reach out that far.
Does anyone have any idea what I can do in this situation or where I'm going wrong? Any advice is greatly appreciated.

You are close - the variable you actually need to reset each time is not myVms but vm_list, as follows:
class Node:
counter = 0
def __init__(self):
c_str = str(Node.counter)
self.uuid = "asdf" + c_str
self.name = "test " + c_str
self.state = "wow " + c_str + " such state"
Node.counter += 1
class Driver:
def __init__(self, number_of_nodes):
self.nodes = []
for x in range(number_of_nodes):
self.nodes.append(Node())
self.name = "the greatest driver"
def list_nodes(self) -> list:
return self.nodes
driver = Driver(10)
def zip_list():
vm_list = {'vmid': [], 'name': [], 'state': [], 'platform': []}
for node in driver.list_nodes():
vm_list["vmid"].append(node.uuid)
vm_list["name"].append(node.name)
vm_list["state"].append(node.state)
vm_list["platform"].append(driver.name)
myVms = zip(vm_list['name'], vm_list['vmid'], vm_list['platform'], vm_list['state'])
return myVms
print("First time:")
my_list = zip_list()
for i in my_list:
print(i)
print("Second time:")
my_list = zip_list()
for i in my_list:
print(i)
If you initialise vm_list outside of the zip_list() function instead, you will see the doubling up that you are experiencing.

You need to initialise vm_list with an empty dict. And if a key exists, then append to its list, else set the dict[key] with an empty list. This is done by setdefault.
Try this:
def zip_list():
...
vm_list = {}
for node in driver.list_nodes():
vm_list.setdefault('vmid', []).append(node.uuid)
vm_list.setdefault('name', []).append(node.name)
vm_list.setdefault('state', []).append(node.state)
vm_list.setdefault('platform', []).append(node.platform)
...
myVms = zip(vm_list['name'], vm_list['vmid'], vm_list['platform'], vm_list['state'])
return myVms

Converting CSV to HTML Table in Python

I'm trying to take data from a .csv file and importing into a HTML table within python.
This is the csv file https://www.mediafire.com/?mootyaa33bmijiq
Context:
The csv is populated with data from a football team [Age group, Round, Opposition, Team Score, Opposition Score, Location]. I need to be able to select a specific age group and only display those details in separate tables.
This is all I've got so far....
infile = open("Crushers.csv","r")
for line in infile:
row = line.split(",")
age = row[0]
week = row [1]
opp = row[2]
ACscr = row[3]
OPPscr = row[4]
location = row[5]
if age == 'U12':
print(week, opp, ACscr, OPPscr, location)

First install pandas:
pip install pandas
Then run:
import pandas as pd
columns = ['age', 'week', 'opp', 'ACscr', 'OPPscr', 'location']
df = pd.read_csv('Crushers.csv', names=columns)
# This you can change it to whatever you want to get
age_15 = df[df['age'] == 'U15']
# Other examples:
bye = df[df['opp'] == 'Bye']
crushed_team = df[df['ACscr'] == '0']
crushed_visitor = df[df['OPPscr'] == '0']
# Play with this
# Use the .to_html() to get your table in html
print(crushed_visitor.to_html())
You'll get something like:
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th>age</th>
<th>week</th>
<th>opp</th>
<th>ACscr</th>
<th>OPPscr</th>
<th>location</th>
</tr>
</thead>
<tbody>
<tr>
<th>34</th>
<td>U17</td>
<td>1</td>
<td>Banyo</td>
<td>52</td>
<td>0</td>
<td>Home</td>
</tr>
<tr>
<th>40</th>
<td>U17</td>
<td>7</td>
<td>Aspley</td>
<td>62</td>
<td>0</td>
<td>Home</td>
</tr>
<tr>
<th>91</th>
<td>U12</td>
<td>7</td>
<td>Rochedale</td>
<td>8</td>
<td>0</td>
<td>Home</td>
</tr>
</tbody>
</table>

Firstly, install pandas:
pip install pandas
Then,
import pandas as pd
a = pd.read_csv("Crushers.csv")
# to save as html file
# named as "Table"
a.to_html("Table.htm")
# assign it to a
# variable (string)
html_file = a.to_html()

Below function takes filename, headers(optional) and delimiter(optional) as input and converts csv to html table and returns as string.
If headers are not provided, assumes header is already present in csv file.
Converts csv file contents to HTML formatted table
def csv_to_html_table(fname,headers=None,delimiter=","):
with open(fname) as f:
content = f.readlines()
#reading file content into list
rows = [x.strip() for x in content]
table = "<table>"
#creating HTML header row if header is provided
if headers is not None:
table+= "".join(["<th>"+cell+"</th>" for cell in headers.split(delimiter)])
else:
table+= "".join(["<th>"+cell+"</th>" for cell in rows[0].split(delimiter)])
rows=rows[1:]
#Converting csv to html row by row
for row in rows:
table+= "<tr>" + "".join(["<td>"+cell+"</td>" for cell in row.split(delimiter)]) + "</tr>" + "\n"
table+="</table><br>"
return table
In your case, function call will look like this, but this will not filter out entries in csv but directly convert whole csv file to HTML table.
filename="Crushers.csv"
myheader='age,week,opp,ACscr,OPPscr,location'
html_table=csv_to_html_table(filename,myheader)
Note: To filter out entries with certain values add conditional statement in for loop.

Before you begin printing the desired rows, output some HTML to set up an appropriate table structure.
When you find a row you want to print, output it in HTML table row format.
# begin the table
print("<table>")
# column headers
print("<th>")
print("<td>Week</td>")
print("<td>Opp</td>")
print("<td>ACscr</td>")
print("<td>OPPscr</td>")
print("<td>Location</td>")
print("</th>")
infile = open("Crushers.csv","r")
for line in infile:
row = line.split(",")
age = row[0]
week = row [1]
opp = row[2]
ACscr = row[3]
OPPscr = row[4]
location = row[5]
if age == 'U12':
print("<tr>")
print("<td>%s</td>" % week)
print("<td>%s</td>" % opp)
print("<td>%s</td>" % ACscr)
print("<td>%s</td>" % OPPscr)
print("<td>%s</td>" % location)
print("</tr>")
# end the table
print("</table>")

First some imports:
import csv
from html import escape
import io
Now the building blocks - let's make one function for reading the CSV and another function for making the HTML table:
def read_csv(path, column_names):
with open(path, newline='') as f:
# why newline='': see footnote at the end of https://docs.python.org/3/library/csv.html
reader = csv.reader(f)
for row in reader:
record = {name: value for name, value in zip(column_names, row)}
yield record
def html_table(records):
# records is expected to be a list of dicts
column_names = []
# first detect all posible keys (field names) that are present in records
for record in records:
for name in record.keys():
if name not in column_names:
column_names.append(name)
# create the HTML line by line
lines = []
lines.append('<table>\n')
lines.append(' <tr>\n')
for name in column_names:
lines.append(' <th>{}</th>\n'.format(escape(name)))
lines.append(' </tr>\n')
for record in records:
lines.append(' <tr>\n')
for name in column_names:
value = record.get(name, '')
lines.append(' <td>{}</td>\n'.format(escape(value)))
lines.append(' </tr>\n')
lines.append('</table>')
# join the lines to a single string and return it
return ''.join(lines)
Now just put it together :)
records = list(read_csv('Crushers.csv', 'age week opp ACscr OPPscr location'.split()))
# Print first record to see whether we are loading correctly
print(records[0])
# Output:
# {'age': 'U13', 'week': '1', 'opp': 'Waterford', 'ACscr': '22', 'OPPscr': '36', 'location': 'Home'}
records = [r for r in records if r['age'] == 'U12']
print(html_table(records))
# Output:
# <table>
# <tr>
# <th>age</th>
# <th>week</th>
# <th>opp</th>
# <th>ACscr</th>
# <th>OPPscr</th>
# <th>location</th>
# </tr>
# <tr>
# <td>U12</td>
# <td>1</td>
# <td>Waterford</td>
# <td>0</td>
# <td>4</td>
# <td>Home</td>
# </tr>
# <tr>
# <td>U12</td>
# <td>2</td>
# <td>North Lakes</td>
# <td>12</td>
# <td>18</td>
# <td>Away</td>
# </tr>
# ...
# </table>
A few notes:
csv.reader works better than line splitting because it also handles quoted values and even quoted values with newlines
html.escape is used to escape strings that could potentially contain character < or >
it is often times easier to worh with dicts than tuples
usually the CSV files contain header (first line with column names) and could be easily loaded using csv.DictReader; but the Crushers.csv has no header (the data start from very first line) so we build the dicts ourselves in the function read_csv
both functions read_csv and html_table are generalised so they can work with any data, the column names are not "hardcoded" into them
yes, you could use pandas read_csv and to_html instead :) But it is good to know how to do it without pandas in case you need some customization. Or just as a programming exercise.

This should be working as well:
from html import HTML
import csv
def to_html(csvfile):
H = HTML()
t=H.table(border='2')
r = t.tr
with open(csvfile) as csvfile:
reader = csv.DictReader(csvfile)
for column in reader.fieldnames:
r.td(column)
for row in reader:
t.tr
for col in row.iteritems():
t.td(col[1])
return t
and call the function by passing the csv file to it.

Other answers are suggesting pandas, but that's probably overkill if formatting CSV to an HTML table is all you need. If you want to use an existing package just for this purpose, there's tabulate:
import csv
from tabulate import tabulate
with open("Crushers.csv") as file:
reader = csv.reader(file)
u12_rows = [row for row in reader if row[0] == "U12"]
print(tabulate(u12_rows, tablefmt="html"))

Loop in dictionary in HTML

I have a Python script creating a dictionary and passing it to a html page to generate a report.
in Python:
data_query= {}
data_query["service1"] = "value1"
data_query["service2"] = "value2"
return data_query
in HTML:
% for name, count in data_query:
<tr>
<td>${name}</td>
<td>${count}</td>
</tr>
% endfor
it does not work, says that it does not return enough values.
I also tried (pointed out in a comment in the other question, that I deleted by mistake):
% for name, count in dict.iteritems():
It does not give any error, but does not work. Displays nothing.
${len(dict)}
gives the right dictionary length
${len(dict.iteritems())}
does not display anything and seem to have a weird effect on my table format.
Is there a way to iterate correctly a dictionart in HTMl to display both the key and value?
EDIT: How I transfer the dictionary to the html page.
from mako.lookup import TemplateLookup
from mako.runtime import Context
from mako.exceptions import text_error_template
html_lookup = TemplateLookup(directories=[os.path.join(self.dir_name)])
html_template = html_lookup.get_template('/templates/report.html')
html_data = { 'data_queries' : data_queries }
html_ctx = Context(html_file, **html_data)
try:
html_template.render_context(html_ctx)
except:
print text_error_template().render(full=False)
html_file.close()
return
html_file.close()

% for name, count in dict.items:
<tr>
<td>${name}</td>
<td>${count}</td>
</tr>
% endfor
should probably work ... typically you dont call the fn when you pass it to a templating language... alternatively
% for name in dict:
<tr>
<td>${name}</td>
<td>${dict[name]}</td>
</tr>
% endfor
would likely also work
as an aside ... dict is a terrible variable name as it shadows the builtin dict (which might be part of your problem if that is actually your variable name)

indexError, searching within

I am writing a program that will read a CSV file with data that looks like this:
"10724_artifact11679.jpg","H. 3 1/4 in. (8.26 cm)","10.210.114","This artwork is currently on display in Gallery 171","11679"
And write it into an HTML table. I only want the files that say, in the 3rd position, "This artwork is not on display".. but I've been having issues with this set of data
import csv
metlist4 = []
newList = csv.reader(open("v2img_10724_list.csv", 'r'))
for row in newList:
metlist4.append(row)
artifact_template = """<td>
<div>
<img src= "%(image)s" alt = "artifact" />
<p>Dimensions: %(dimension)s </p>
<p>Accession #: %(accession)s </p>
<p>Display: %(display)s </p>
<p>index2: %(index2)s </p>
</div>
</td>"""
html_list = []
count = 5794
for artifact in metlist4:
if artifact[3] in ["This artwork is not on display"]:
artifactinfo = {}
artifactinfo["image"]=artifact[0]
artifactinfo["dimension"]=artifact[1]
artifactinfo["accession"]=artifact[2]
artifactinfo["display"]=artifact[3]
artifactinfo["index2"]=count
count = count + 1
html_list.append(artifact_template % artifactinfo)
else:
pass
f = open("v3display_test.txt", "w")
f.write("\n".join(html_list))
f.close()
I get this error, but only when I run the entire metlist4...
File "/Users/Rose/Documents/workspace/METProjectFOREAL/src/no_display_Met4.py", line 34, in <module>
if artifact[3] in ["This artwork is not on display"]:
IndexError: list index out of range
if I run just a section, for example metlist4[0:500], the error does not occur. Any ideas or suggestions would be greatly appreciated!! Thanks!

There is at least one row that doesn't have a 4th element. Perhaps the line is empty.
Test for the length, and print the row to test:
if len(artifact) < 4:
print 'short row', artifact
If it is an empty line, just skip it:
if not artifact: continue
You are using a lot of verbose and redundant code; there is no need to build a separate list when you can just loop over the csv.reader() object directly, and there is no need to add an empty else: pass block either.
Idiomatic Python code would be:
artifact_template = """<td>
<div>
<img src= "%(image)s" alt = "artifact" />
<p>Dimensions: %(dimension)s </p>
<p>Accession #: %(accession)s </p>
<p>Display: %(display)s </p>
<p>index2: %(index2)s </p>
</div>
</td>"""
html_list = []
fields = 'image dimension accession display'.split()
with open("v2img_10724_list.csv", 'rb') as inputfile:
reader = csv.DictReader(inputfile, fields=fields, restval='_ignored')
for count, artifact in enumerate(reader, 5794):
if artifact and artifact['display'] == "This artwork is not on display":
artifactinfo["index2"] = count
html_list.append(artifact_template % artifact)
This use a csv.DictReader() instead to create the dictionaries per row, a with statement to ensure the file is closed when done, and enumerate() with a start value to track count.