We Keep Coding

Flask: Python List is displaying values in HTML file with quote marks and round brackets

I have the below function which returns a list
def get_entered_datalist(firstname, lastname):
if firstname and lastname:
curr = db.session.execute(entered_data, {'firstname': firstname, 'lastname': lastname})
list_name = list() #this is a list contstructor statement so have () brackets, usually when declaring list we use [] brackets
for name in curr:
list_name.append(name)
return list_name
else:
return False
It is being called this way
entered_names = get_entered_datalist(session['v_firstname'], session['v_lastname'])
and passed to the html file this way
return render_template('monthlybudget.html', title='Enter Monthly Budget Data', form=form, entereddata=entered_names)
The display code in HTML file is as below
{% if entereddata %}
{% for x in entereddata %}
<p>{{ x }} </p>
{% endfor %}
{% else %}
<p>No data found</p>
{% endif %}
but it is being displayed with round brackets and quote marks as below
The query getting the values from database is as below
select concat([First Name], ' ', [Last Name] ) as name from [dbo].[EAMonthlyBudget]
where [Report to First Name]= :firstname and [Report to Last Name] = :lastname
and [budget month] = datename(month,getdate()) and [budget year] = year(getdate());
I added a print statement in my python code to check what gets returned by the function and that is as below (shows with the brackets and quote marks)
[12/Aug/2022 15:29:44] "GET /static/images/nu_logo2.png HTTP/1.1" 304 -
[('Janak Shah',), ('Julie Wang',)]
-
How do I display the data without the brackets and quote marks?

Your curr variable contains tuples. try this code:
for name in curr:
list_name.append(name[0])
I think you need to check of index out for range error.
I hope this helps.

Render variables from a file

There is a file api.yml containing a config for ansible:
config/application.properties:
server.port: 6081
system.default.lang: rus
api.pdd.url: "http://{{ stage['PDD'] }}"
api.policy.alias: "integration"
api.order.url: "http://{{ stage['Order'] }}
api.foo.url: "http://{{ stage['FOO'] }}
There is a stage.yml containing the key and the stage values:
default_node:
Order: '172.16.100.40:8811'
PDD: '172.16.100.41:8090'
FOO: '10.100.0.11:3165
In fact, these files are larger and the 'stage' variables are also many.
My task is to parse api.yml and turn it into properties-config. The problem is that I can not pull up the values ​​{{stage ['value']}} I'm trying to do it this way:
stream = yaml.load(open('api.yml'))
result={}
result.update(stream['config/application.properties'])
context= yaml.load(open('stage.yml'))
stage={}
stage.update(context['default_node'])
text = '{% for items in result | dictsort(true)%} {{ items[0] }} = {{
items[1] }} {%endfor%}'
template = Template(text)
properti = (template.render(result=result, stage=stage))
At the output I get this:
server.port = 6081
system.default.lang = rus
api.pdd.url = http://{{ stage['PDD'] }}
api.policy.alias = integration
api.order.url = http://{{ stage['Order'] }}
api.foo.url = http://{{ stage['FOO'] }}
And you need to get this:
server.port = 6081
system.default.lang = rus
api.pdd.url = 172.16.100.41:8090
api.policy.alias = "integration"
api.order.url = 172.16.100.40:8811
api.foo.url = 10.100.0.11:3165
Can I do it with jinja or ansible lib?
Sorry for my bad english

Following this approach, you would need to treat api.yml as a template itself and render it. Otherwise, jinja2 will treat it as a simple value of the property. Something like this would do:
import yaml
from jinja2 import Environment, Template
import json
stream = yaml.load(open('api.yml'))
result={}
result.update(stream['config/application.properties'])
context= yaml.load(open('stage.yml'))
stage={}
stage.update(context['default_node'])
text = """{% for items in result | dictsort(true)%} {{ items[0] }} = {{ items[1] }} {%endfor%}"""
#Then render the results dic as well
resultsTemplate = Template(json.dumps(result))
resultsRendered = json.loads( resultsTemplate.render(stage=stage) )
template = Template(text)
properti = (template.render(result=resultsRendered, stage=stage))
After this you will see the wanted values in the properti var:
' api.foo.url = http://10.100.0.11:3165 api.order.url = http://172.16.100.40:8811 api.pdd.url = http://172.16.100.41:8090 api.policy.alias = integration server.port = 6081 system.default.lang = rus'
It would be nice though if jinja2 was able to render recursively. Maybe spending some time working out with the globals and shared modes of the Environment this can be achieved.
Hope this helps.

Converting CSV to HTML Table in Python

I'm trying to take data from a .csv file and importing into a HTML table within python.
This is the csv file https://www.mediafire.com/?mootyaa33bmijiq
Context:
The csv is populated with data from a football team [Age group, Round, Opposition, Team Score, Opposition Score, Location]. I need to be able to select a specific age group and only display those details in separate tables.
This is all I've got so far....
infile = open("Crushers.csv","r")
for line in infile:
row = line.split(",")
age = row[0]
week = row [1]
opp = row[2]
ACscr = row[3]
OPPscr = row[4]
location = row[5]
if age == 'U12':
print(week, opp, ACscr, OPPscr, location)

First install pandas:
pip install pandas
Then run:
import pandas as pd
columns = ['age', 'week', 'opp', 'ACscr', 'OPPscr', 'location']
df = pd.read_csv('Crushers.csv', names=columns)
# This you can change it to whatever you want to get
age_15 = df[df['age'] == 'U15']
# Other examples:
bye = df[df['opp'] == 'Bye']
crushed_team = df[df['ACscr'] == '0']
crushed_visitor = df[df['OPPscr'] == '0']
# Play with this
# Use the .to_html() to get your table in html
print(crushed_visitor.to_html())
You'll get something like:
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th>age</th>
<th>week</th>
<th>opp</th>
<th>ACscr</th>
<th>OPPscr</th>
<th>location</th>
</tr>
</thead>
<tbody>
<tr>
<th>34</th>
<td>U17</td>
<td>1</td>
<td>Banyo</td>
<td>52</td>
<td>0</td>
<td>Home</td>
</tr>
<tr>
<th>40</th>
<td>U17</td>
<td>7</td>
<td>Aspley</td>
<td>62</td>
<td>0</td>
<td>Home</td>
</tr>
<tr>
<th>91</th>
<td>U12</td>
<td>7</td>
<td>Rochedale</td>
<td>8</td>
<td>0</td>
<td>Home</td>
</tr>
</tbody>
</table>

Firstly, install pandas:
pip install pandas
Then,
import pandas as pd
a = pd.read_csv("Crushers.csv")
# to save as html file
# named as "Table"
a.to_html("Table.htm")
# assign it to a
# variable (string)
html_file = a.to_html()

Below function takes filename, headers(optional) and delimiter(optional) as input and converts csv to html table and returns as string.
If headers are not provided, assumes header is already present in csv file.
Converts csv file contents to HTML formatted table
def csv_to_html_table(fname,headers=None,delimiter=","):
with open(fname) as f:
content = f.readlines()
#reading file content into list
rows = [x.strip() for x in content]
table = "<table>"
#creating HTML header row if header is provided
if headers is not None:
table+= "".join(["<th>"+cell+"</th>" for cell in headers.split(delimiter)])
else:
table+= "".join(["<th>"+cell+"</th>" for cell in rows[0].split(delimiter)])
rows=rows[1:]
#Converting csv to html row by row
for row in rows:
table+= "<tr>" + "".join(["<td>"+cell+"</td>" for cell in row.split(delimiter)]) + "</tr>" + "\n"
table+="</table><br>"
return table
In your case, function call will look like this, but this will not filter out entries in csv but directly convert whole csv file to HTML table.
filename="Crushers.csv"
myheader='age,week,opp,ACscr,OPPscr,location'
html_table=csv_to_html_table(filename,myheader)
Note: To filter out entries with certain values add conditional statement in for loop.

Before you begin printing the desired rows, output some HTML to set up an appropriate table structure.
When you find a row you want to print, output it in HTML table row format.
# begin the table
print("<table>")
# column headers
print("<th>")
print("<td>Week</td>")
print("<td>Opp</td>")
print("<td>ACscr</td>")
print("<td>OPPscr</td>")
print("<td>Location</td>")
print("</th>")
infile = open("Crushers.csv","r")
for line in infile:
row = line.split(",")
age = row[0]
week = row [1]
opp = row[2]
ACscr = row[3]
OPPscr = row[4]
location = row[5]
if age == 'U12':
print("<tr>")
print("<td>%s</td>" % week)
print("<td>%s</td>" % opp)
print("<td>%s</td>" % ACscr)
print("<td>%s</td>" % OPPscr)
print("<td>%s</td>" % location)
print("</tr>")
# end the table
print("</table>")

First some imports:
import csv
from html import escape
import io
Now the building blocks - let's make one function for reading the CSV and another function for making the HTML table:
def read_csv(path, column_names):
with open(path, newline='') as f:
# why newline='': see footnote at the end of https://docs.python.org/3/library/csv.html
reader = csv.reader(f)
for row in reader:
record = {name: value for name, value in zip(column_names, row)}
yield record
def html_table(records):
# records is expected to be a list of dicts
column_names = []
# first detect all posible keys (field names) that are present in records
for record in records:
for name in record.keys():
if name not in column_names:
column_names.append(name)
# create the HTML line by line
lines = []
lines.append('<table>\n')
lines.append(' <tr>\n')
for name in column_names:
lines.append(' <th>{}</th>\n'.format(escape(name)))
lines.append(' </tr>\n')
for record in records:
lines.append(' <tr>\n')
for name in column_names:
value = record.get(name, '')
lines.append(' <td>{}</td>\n'.format(escape(value)))
lines.append(' </tr>\n')
lines.append('</table>')
# join the lines to a single string and return it
return ''.join(lines)
Now just put it together :)
records = list(read_csv('Crushers.csv', 'age week opp ACscr OPPscr location'.split()))
# Print first record to see whether we are loading correctly
print(records[0])
# Output:
# {'age': 'U13', 'week': '1', 'opp': 'Waterford', 'ACscr': '22', 'OPPscr': '36', 'location': 'Home'}
records = [r for r in records if r['age'] == 'U12']
print(html_table(records))
# Output:
# <table>
# <tr>
# <th>age</th>
# <th>week</th>
# <th>opp</th>
# <th>ACscr</th>
# <th>OPPscr</th>
# <th>location</th>
# </tr>
# <tr>
# <td>U12</td>
# <td>1</td>
# <td>Waterford</td>
# <td>0</td>
# <td>4</td>
# <td>Home</td>
# </tr>
# <tr>
# <td>U12</td>
# <td>2</td>
# <td>North Lakes</td>
# <td>12</td>
# <td>18</td>
# <td>Away</td>
# </tr>
# ...
# </table>
A few notes:
csv.reader works better than line splitting because it also handles quoted values and even quoted values with newlines
html.escape is used to escape strings that could potentially contain character < or >
it is often times easier to worh with dicts than tuples
usually the CSV files contain header (first line with column names) and could be easily loaded using csv.DictReader; but the Crushers.csv has no header (the data start from very first line) so we build the dicts ourselves in the function read_csv
both functions read_csv and html_table are generalised so they can work with any data, the column names are not "hardcoded" into them
yes, you could use pandas read_csv and to_html instead :) But it is good to know how to do it without pandas in case you need some customization. Or just as a programming exercise.

This should be working as well:
from html import HTML
import csv
def to_html(csvfile):
H = HTML()
t=H.table(border='2')
r = t.tr
with open(csvfile) as csvfile:
reader = csv.DictReader(csvfile)
for column in reader.fieldnames:
r.td(column)
for row in reader:
t.tr
for col in row.iteritems():
t.td(col[1])
return t
and call the function by passing the csv file to it.

Other answers are suggesting pandas, but that's probably overkill if formatting CSV to an HTML table is all you need. If you want to use an existing package just for this purpose, there's tabulate:
import csv
from tabulate import tabulate
with open("Crushers.csv") as file:
reader = csv.reader(file)
u12_rows = [row for row in reader if row[0] == "U12"]
print(tabulate(u12_rows, tablefmt="html"))

Python list variable substitution within multiline string...almost(edited)

(Edited, previous WAAAY below) -
I got it to work by moving the string subst within the for loop, but the result automatically puts 5 tabs to the left that I can't seem to get rid of
I wanted to leave it up for a little bit to see if anyone has an answer and maybe to help someone that follows me...
Code:
for i in dns_list:
with open("output.txt", "a") as output:
alert_dns = textwrap.dedent("""
{
\"tests\":[
{{
\"token\":\"DNS\",
\"type\":\"text\",
\"operator\":\"contains\",
\"preservecase\":false,
\"value\":\"%s\"
}
]
""")%(i)
alert_dns=alert_dns.strip()
output.write(alert_dns.strip())
(Previous)
I have a list of domain names, I need to iterate through the list(dns_list) and place the variable 'insert' into a multiline string(alert_dns)-
alert_dns="""
{{
\"tests\":[
{{
\"token\":\"DNS\",
\"type\":\"text\",
\"operator\":\"contains\",
\"preservecase\":false,
\"value\":\"{insert}\"
}}
]
}}
"""
dns_list=[]
temp_file_name = 'daily.csv'
with open(temp_file_name, 'r') as temp_file:
lines = temp_file.read()
dns = re.findall(urlmarker.WEB_URL_REGEX,lines)
for i in dns:
dns_list.append(i)
with open("output.txt", "w") as output:
for i in dns_list:
for insert in alert_dns:
# i=insert
alert_dns.format(i)
output.write(alert_dns+'\n')
I keep getting --
alert_dns.format(i)
KeyError: 'insert'

Instead of alert_dns.format(i)
you should
alert_dns.format(insert=i)

Alright this is the answer if anyone looks for it...
with open("output.txt", "w") as output:
for i in dns_list:
alert_dns = textwrap.dedent("""\
{
\"tests\":[
{{
\"token\":\"DNS\",
\"type\":\"text\",
\"operator\":\"contains\",
\"preservecase\":false,
\"value\":\"%s\"
}
]
""")%(i)
output.write(alert_dns+'\n')

indexError, searching within

I am writing a program that will read a CSV file with data that looks like this:
"10724_artifact11679.jpg","H. 3 1/4 in. (8.26 cm)","10.210.114","This artwork is currently on display in Gallery 171","11679"
And write it into an HTML table. I only want the files that say, in the 3rd position, "This artwork is not on display".. but I've been having issues with this set of data
import csv
metlist4 = []
newList = csv.reader(open("v2img_10724_list.csv", 'r'))
for row in newList:
metlist4.append(row)
artifact_template = """<td>
<div>
<img src= "%(image)s" alt = "artifact" />
<p>Dimensions: %(dimension)s </p>
<p>Accession #: %(accession)s </p>
<p>Display: %(display)s </p>
<p>index2: %(index2)s </p>
</div>
</td>"""
html_list = []
count = 5794
for artifact in metlist4:
if artifact[3] in ["This artwork is not on display"]:
artifactinfo = {}
artifactinfo["image"]=artifact[0]
artifactinfo["dimension"]=artifact[1]
artifactinfo["accession"]=artifact[2]
artifactinfo["display"]=artifact[3]
artifactinfo["index2"]=count
count = count + 1
html_list.append(artifact_template % artifactinfo)
else:
pass
f = open("v3display_test.txt", "w")
f.write("\n".join(html_list))
f.close()
I get this error, but only when I run the entire metlist4...
File "/Users/Rose/Documents/workspace/METProjectFOREAL/src/no_display_Met4.py", line 34, in <module>
if artifact[3] in ["This artwork is not on display"]:
IndexError: list index out of range
if I run just a section, for example metlist4[0:500], the error does not occur. Any ideas or suggestions would be greatly appreciated!! Thanks!

There is at least one row that doesn't have a 4th element. Perhaps the line is empty.
Test for the length, and print the row to test:
if len(artifact) < 4:
print 'short row', artifact
If it is an empty line, just skip it:
if not artifact: continue
You are using a lot of verbose and redundant code; there is no need to build a separate list when you can just loop over the csv.reader() object directly, and there is no need to add an empty else: pass block either.
Idiomatic Python code would be:
artifact_template = """<td>
<div>
<img src= "%(image)s" alt = "artifact" />
<p>Dimensions: %(dimension)s </p>
<p>Accession #: %(accession)s </p>
<p>Display: %(display)s </p>
<p>index2: %(index2)s </p>
</div>
</td>"""
html_list = []
fields = 'image dimension accession display'.split()
with open("v2img_10724_list.csv", 'rb') as inputfile:
reader = csv.DictReader(inputfile, fields=fields, restval='_ignored')
for count, artifact in enumerate(reader, 5794):
if artifact and artifact['display'] == "This artwork is not on display":
artifactinfo["index2"] = count
html_list.append(artifact_template % artifact)
This use a csv.DictReader() instead to create the dictionaries per row, a with statement to ensure the file is closed when done, and enumerate() with a start value to track count.