I'm relatively new to Python and Flask. I've been trying to create a web application that reads data readings from a .txt file and plots them onto a matplotlib plot. In this web-app, I have a front page with 3 buttons. These buttons redirect to different routes with functions that read data and plot them onto a matplotlib plot. The web-app works perfectly only for the first time I go to either of these routes. After that nothing loads anymore. I guess I have an infinite loop of some sort but I can't figure it out. Also, after the website gets stuck the Python process starts consuming more resources.
The problem persists only when I open this route on the web-app:
#app.route("/temperature/")
This loads without problems on the web-page, but only for one time and the whole web-app gets stuck and I cannot access any of the other routes either.
Thanks in advance!
EDIT 1 - Whole code below
cloudapp.py (Python sourcecode that runs Flask and the functions)
from flask import Flask
from flask import render_template
from flask import request
import numpy as np
import matplotlib.pyplot as plt, mpld3
from datetime import datetime
app = Flask(__name__, template_folder='C:\Users\Valtteri\Desktop\cloudapp\Templates\HTML')
global all_lines
#app.route("/")
def frontpage():
return render_template('frontpage.html')
#app.route("/temperature/")
def temperature():
f = open('C:/Email/file.txt', 'r')
cpt = 0 # Line amount value
all_lines = [] # List that has every Nth value
for line in f:
cpt += 1 # Goes through every line and adds 1 to the counter
# How often values are plotted (every Nth value)
if cpt%100 == 0:
all_lines.append(line) # When 30th line is counted, add that line to all_lines[] list
if cpt == 500: # How many values are plotted (counts from first)
break
dates = [str(line.split(';')[0]) for line in all_lines]
date = [datetime.strptime(x,'%Y.%m.%d_%H:%M') for x in dates]
y = [float(line.split(';')[1]) for line in all_lines]
z = [float(line.split()[2]) for line in all_lines]
fig, ax = plt.subplots()
ax.plot_date(date, y, 'r-')
f.close()
return mpld3.fig_to_html(fig)
if __name__ == "__main__":
app.run()
frontpage.html (HTML template in folder ..\templates\html)
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Envic Oy Cloud</title>
</head>
<body>
<div class="headers">
<h1>Web-Application</h1>
<h2> Version 0.0.1 </h2>
</div>
<div class="buttons">
<h3 class="buttonheader">Logger 1</h3>
<a class="templink" href="http://127.0.0.1:5000/temperature/" target="_blank"> Check temperature </a>
</body>
</html>
I use Bash on windows to run the code with
export FLASK_APP=myCloud.py
flask run
EDIT 2
I tried to solve the issue for a long time, but couldn't find a solution. It has something to do with Flask/mpld3 compatibility for me. I made the very same web app, but this time using a simple pyramid WSGI. I can now refresh the plot as many times and redirect myself into any view without the server hanging. I'll still leave the post unsolved, because I would still like to use Flask. I will continue my research too. Here is the pyramid version that works for me:
from wsgiref.simple_server import make_server
from pyramid.config import Configurator
from pyramid.response import Response
import numpy as np
import matplotlib.pyplot as plt, mpld3
from datetime import datetime
def hello_world(request):
return Response('<h1>Testing the Pyramid version!</h1>Check temperature')
def second_view(request):
with open('C:/Email/file.txt') as f:
cpt = 0 # Line amount value
all_lines = [] # List that has every Nth value
for line in f:
cpt += 1 # Goes through every line and adds 1 to the counter
# How often values are plotted (every Nth value)
if cpt%100 == 0:
all_lines.append(line) # When 30th line is counted, add that line to all_lines[] list
if cpt == 500: # How many values are plotted (counts from first)
break
dates = [str(line.split(';')[0]) for line in all_lines]
date = [datetime.strptime(x,'%Y.%m.%d_%H:%M') for x in dates]
y = [float(line.split(';')[1]) for line in all_lines]
z = [float(line.split()[2]) for line in all_lines]
plt.figure(figsize=(10,5))
plt.title('Humidity', fontsize=15)
plt.ylabel('Humidity RH', fontsize=15)
fig = plt.figure()
plot = plt.plot_date(date, z, 'b-')
myfig = mpld3.fig_to_html(fig, template_type='simple')
return Response(myfig)
if __name__ == '__main__':
config = Configurator()
config.add_route('hello_world', '/hello_world')
config.add_route('second_view', '/second_view')
config.add_view(hello_world, route_name='hello_world')
config.add_view(second_view, route_name='second_view')
app = config.make_wsgi_app()
server = make_server('', 8888, app)
server.serve_forever()
Here is what I attempted. I didn't get any issues with the server hanging after refreshing or opening a new tab of the link. However, after I shutdown the server (Ctrl + C), the console threw some exceptions that suggests that mpld3 or matplotlib had opened some new threads. Specifically, the exception is RuntimeError: main thread is not in main loop.
I did some googling and came across this link. The guy suggested using fig_to_dict with json. I tried his solution and I was still getting the exception.
Now, I'm going to write down both approaches and let you decide which to use. I don't know if either of them will work for you. For my setup, the app runs fine despite the exceptions after closing the server.
I'm also going to use your example that didn't read the txt file. I have debug set to True so I can make sure the GET requests are being processed when I refresh the chart or open a new instance of the link.
Approach 1 (fig_to_html)
app.py
from flask import Flask
from flask import render_template
import matplotlib.pyplot as plt
import mpld3
app = Flask(__name__, template_folder='/path/to/templates')
#app.route("/")
#app.route("/index")
def index():
return render_template('frontpage.html')
#app.route('/temperature')
def temperature():
date = ([1, 2, 3, 4])
y = ([1, 2, 3, 4])
fig = plt.figure(figsize=(10, 5))
plt.title('Temperature', fontsize=15)
plt.ylabel('Temperature' + u'\u2103', fontsize=15)
plt.plot(date, y, 'b-')
plt.ylim([0, 40])
myfig = mpld3.fig_to_html(fig, template_type='simple')
plt.clf() # clear figure
plt.cla() # clear axes
plt.close('all') # close all figures
# Print as HTML
return myfig
if __name__ == "__main__":
app.run(debug=True) # run on debug mode
Approach 2 (fig_to_dict)
app.py
from flask import Flask
from flask import render_template
import matplotlib.pyplot as plt
import mpld3
import json
app = Flask(__name__, template_folder='/path/to/templates')
#app.route("/")
#app.route("/index")
def index():
return render_template('frontpage.html')
#app.route('/temperature')
def temperature():
date = ([1, 2, 3, 4])
y = ([1, 2, 3, 4])
fig = plt.figure(figsize=(10, 5))
plt.title('Temperature', fontsize=15)
plt.ylabel('Temperature' + u'\u2103', fontsize=15)
plt.plot(date, y, 'b-')
plt.ylim([0, 40])
single_chart = dict()
single_chart['id'] = "temp_figure"
single_chart['json'] = json.dumps(mpld3.fig_to_dict(fig))
plt.clf() # clear figure
plt.cla() # clear axes
plt.close('all') # close figure
# Print as HTML
return render_template('temperature.html', single_chart=single_chart)
if __name__ == "__main__":
app.run(debug=True) # run on debug mode
And here are the template files. I noticed you used a static link in your frontpage.html so I replaced that with a placeholder that allows Flask to auto-populate the URL. You also had a div tag that you didn't close.
frontpage.html
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Envic Oy Cloud</title>
</head>
<body>
<div class="headers">
<h1>Web-Application</h1>
<h2> Version 0.0.1 </h2>
</div>
<div class="buttons"></div>
<h3 class="buttonheader">Logger 1</h3>
<a class="templink" href="{{ url_for('temperature') }}" target="_blank"> Check temperature </a>
</body>
</html>
temperature.html (for 2nd approach only)
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<title>Sample Page</title>
<script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.1.10/require.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.3/jquery.min.js"></script>
<script type="text/javascript" src="http://d3js.org/d3.v3.min.js"></script>
<script type="text/javascript" src="http://mpld3.github.io/js/mpld3.v0.2.js"></script>
</head>
<body>
<div id="{{single_chart.id}}">
</div>
<script type="text/javascript">
var figureId = "{{single_chart.id}}";
var json01 = {{single_chart.json|safe}};
mpld3.draw_figure(figureId, json01);
</script>
</body>
</html>
BTW, I'm using Python 3.5.4, Flask==0.12.2, matplotlib==2.1.2, and mpld3==0.3. I tested using Chrome Version 67.0.3396.87.
Related
I am attempting to make an app where a user can upload a CSV file of data and view the data thru the browser on a different flask route which would also a show the plot of the data.
I having issues in my code trying to show the plot of the data. (referencing the static file .png) I can get the pandas dataframe to HTML to work, but in my table.html file I am trying to reference a png plot created with matplot lib and saved to a static directory.
<img src="{{url_for('static', filename=filename)}}" />
All of this experimenting is due to cache issues with the browser so I am creating a unique filename with next_file_name function, and I think this is where I am getting screwed up in the HTML & Jinja trying to reference this unique file name.. I am hoping that a unique filename may be a fix for the cache issues I am observing. The png files are saving properly with the function plot0, plot1, plot2, plot3, etc...
I was hoping to be able to create something where I can repeat the process over & over of analyzing new data and getting retrieving a fresh new plot of the data. Any tips help, thanks
from flask import Flask, make_response, request, render_template
from werkzeug.utils import secure_filename
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
import time
app = Flask(__name__, static_url_path='/static')
num = 0
def next_file_name(num):
return 'static/plot%d.png' % num
#app.route('/')
def form():
return render_template('form.html')
#app.route('/transform', methods=["POST"])
def transform_view():
global num
f = request.files['data_file']
filename = secure_filename(f.filename)
f.save(filename)
df = pd.read_csv(filename, index_col='Date', parse_dates=True)
OAT = pd.Series(df['OAT'])
RAT = pd.Series(df['RAT'])
MAT = pd.Series(df['MAT'])
df_OATrat = (OAT - RAT)
df_MATrat = (MAT - RAT)
plt.scatter(df_OATrat,df_MATrat, color='grey', marker='+')
plt.xlabel('OAT-RAT')
plt.ylabel('MAT-RAT')
plt.title('Economizer Diagnostics')
plt.plot([0,-18],[0,-18], color='green', label='100% OSA during ideal conditions')
plt.plot([0,20],[0,5], color='red', label='Minimum OSA in cooling mode')
plt.plot([0,-38],[0,-9.5], color='blue', label='Minimum OSA in heating mode')
plt.plot([0,0],[-20,10], color='black')
plt.plot([-30,20],[0,0], color='black')
#plt.legend()
plt.text(-3, -28, time.ctime(), fontsize=9)
pic = next_file_name(num)
plt.savefig(pic)
num+=1
resp = make_response(render_template('table.html', tables=[df.to_html(classes='data')], titles=df.columns.values, filename='pic'))
resp.cache_control.no_cache = True
return resp
if __name__ == '__main__':
app.run(debug=True)
table.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv='cache-control' content='no-cache'>
<meta http-equiv='expires' content='0'>
<meta http-equiv='pragma' content='no-cache'>
<title>Title</title>
</head>
<body>
<h1>Economizer Data Plot</h1>
<img src="{{url_for('static', filename=filename)}}" />
</form>
{% for table in tables %}
{{ table|safe }}
{% endfor %}
</body>
</html>
You made pic a string instead of using it's value
... filename=pic
If it's a caching issue, it would be easier to either add automatic cache busting to your image files or disable cache with an after-request callback, which triggers the execution of a function at the end of a request.
#app.after_request
def add_header(response):
response.headers["Cache-Control"] = "no-cache, no-store, must-revalidate"
response.headers["Pragma"] = "no-cache"
response.headers["Expires"] = "0"
response.headers['Cache-Control'] = 'public, max-age=0'
return response
See this solution for more information.
I am trying to display the plotted figure using the web browser. But the render template in return statement is throwing error. Below is the code used and the error received.
The python code:
from flask import Flask, render_template, Response
import io
import matplotlib.pyplot as plt
from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
import numpy as np
app = Flask(__name__,template_folder = 'D:/DATA_SETS')
#app.route('/html_example')
def myplot():
with app.app_context():
plt.figure()
t = range(0,6,1)
x= np.sin(t)
plt.plot(t,x)
plt.savefig('D:/DATA_SETS/new_plot.png')
return Response(render_template('template_1.html', name = 'new_plot', url ='D:/DATA_SETS/new_plot.png'))
f = myplot()
if 1 == 1:
app.run(debug=True)
The template template_1.html
<!doctype html>
<html>
<body>
<h1>Performance</h1>
<p>{{ name }}</p>
<img src={{ url}} alt="Chart" height="42" width="42">
</body>
</html>
Expected : The image return from function should be displayed in the HTML format written.
Current : The webpage is displaying 'Not Found Error'.
Webpage looking at is 'http://localhost:5000/html_example'
You need to wrap your code into with app.app_context function. Your code will be similar to this:
app = Flask(__name__,template_folder = 'templates')
#app.route('/')
def myplot():
with app.app_context():
plt.figure()
t = range(0,6,1)
x= np.sin(t)
plt.plot(t,x)
plt.savefig('D:/DATA_SETS/new_plot.png')
return Response(render_template('template_1.html', name = 'new_plot',
url ='D:/DATA_SETS/new_plot.png'))
if __name__ == '__main__':
app.run(debug=True)
The problem is in the return statement. It should be
return render_template('template_1.html', name = 'new_plot', url ='D:/DATA_SETS/new_plot.png'))
and you should not be doing
f= myplot()
for producing the image.
when you will hit the URL localhost:5000/ it will automatically do the process of producing image and serve it to frontend.
Update:
Directory Structure:
main.py
from flask import Flask, render_template, Response
import io
import matplotlib.pyplot as plt
from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
import numpy as np
app = Flask(__name__,static_url_path="/folder/",static_folder="folder")
#app.route('/')
def myplot():
plt.figure()
t = range(0,6,1)
x= np.sin(t)
plt.plot(t,x)
plt.savefig(app.static_folder+'/'+'new_plot.png')
return render_template('template_1.html', name = 'new_plot', filename ='new_plot.png')
if __name__ == "__main__":
app.run(debug=True)
template_1.html
<!doctype html>
<html>
<body>
<h1>Performance</h1>
<p>{{ name }}</p>
<img src="{{ url_for('static', filename=filename) }}" alt="Chart" height="100" width="100">
</body>
</html>
Output:
Following this tutorial, I am trying to visualise a dataset using Holoviews instead of Bokeh (sample data available here as a CSV file), serving the results using Flask. I decided to use Flask and not Bokeh Server because I am building a larger workflow using the former.
My code is the following:
from flask import Flask, render_template, request
import numpy as np
import pandas as pd
from datetime import datetime
from bokeh.embed import components
from bokeh.io import curdoc
import holoviews as hv
hv.extension("bokeh")
app = Flask(__name__)
renderer = hv.renderer('bokeh')
infile = "./uploads/test.csv"
def loadRegionData(regionProperty, **kwargs):
df = pd.read_csv(infile, parse_dates=['Datetime'])
df1 = df[regionProperty]
df = pd.concat([df['Datetime'],df1], axis=1)
return hv.Curve(df)
colNames = ((pd.read_csv(infile, nrows=1)).drop(['Datetime'], axis=1)).columns.values
dmap = hv.DynamicMap(loadRegionData, kdims='RegionProperty').redim.values(RegionProperty=colNames)
hvplot = renderer.get_plot(dmap)
plot = hvplot.state
plot.name = 'plot'
curdoc().add_root(plot)
#app.route("/")
def index():
# Embed plot into HTML via Flask Render
script, div = components(plot)
return render_template("index.html", script=script, div=div)
if __name__ == '__main__':
app.run(port=5000, debug=True)
I am running into the following (unrelated issues)
When I deploy using Flask, the dropdowns to select the columns do not appear. I suspect that is because I am not returning/referring to the correct variables from the index() function into my index.html:
<html>
<head>
<link
href="http://cdn.bokeh.org/bokeh/release/bokeh-1.0.2.min.css"
rel="stylesheet" type="text/css">
<link
href="http://cdn.bokeh.org/bokeh/release/bokeh-widgets-1.0.2.min.css"
rel="stylesheet" type="text/css">
<script src="http://cdn.bokeh.org/bokeh/release/bokeh-1.0.2.min.js"></script>
<script src="http://cdn.bokeh.org/bokeh/release/bokeh-widgets-1.0.2.min.js"></script>
</head>
<body>
<h1>Holoview test</h1>
{{ script|safe }}
{{ div|safe }}
</body>
</html>
How can I get Flask to also show the dropdown selector?
An unrelated issue which I found when I tested this app using Bokeh Server, and which could also arise in the Flask implementation, is that the scales do not adjust dynamically based on my column selection. Perhaps this can go as a separate question on SO, but I thought to include it here for now to keep things together.
I'm new to Bokeh and Flask python in general but i have managed to create a graph and then output it on my browser via flask. The only problem is that im not getting the "Bokeh tool bar" next to my graph.
My code looks like this
from flask import Flask, render_template, request
import pandas as pd
import csv
from bokeh.plotting import figure
from bokeh.io import show
from bokeh.embed import components
from bokeh.models import Range1d
from bokeh.resources import CDN
app = Flask(__name__)
# Create the main plot
def create_figure():
xvals = []
yvals = []
with open('test.csv') as csvfile:
readCSV = csv.reader(csvfile, delimiter=',')
for row in readCSV:
xvalue = row[0]
yvalue = row[1]
xvals.append(xvalue)
yvals.append(yvalue)
p = figure(plot_width=400, plot_height=400, x_range=(0, 20))
p.y_range = Range1d(0, 15)
p.circle(xvals, yvals, size=10)
return p
# Index page
#app.route('/')
def index():
plot = create_figure()
script, div = components(plot)
cdn_js = CDN.js_files[0]
cdn_css = CDN.css_files[0]
return render_template("index.html", script=script, div=div,
cdn_js=cdn_js,
cdn_css=cdn_css)
# With debug=True, Flask server will auto-reload
# when there are code changes
if __name__ == '__main__':
app.run(port=5000, debug=True)
and my index.html code looks like this:
<html>
<head>
<link href={{ cdn_css|safe }} type="text/css" />
<script type="text/javascript" src={{ cdn_js|safe }}></script>
</head>
<body>
<H1>First</H1>
{{ script|safe }}
{{ div|safe }}
</body>
</html>
Am i missing something? When i output the graph to a output_file i get the toolbar. Any help would be appreciated.
Most likely this is this https://github.com/bokeh/bokeh/issues/7497 issue. The available workarounds, as I posted in the thread, are as follows:
There are two options. If you want to keep the toolbar as part of a plot, then you will have to create a ToolbarPanel manually and add it with add_layout() to a plot. Alternatively you can have a toolbar detached from a plot, as an element of a bigger layout, like it happens with grid plots. In both cases the key is to set plot.toolbar_location = None, to disable creation of the default ToolbarPanel.
Please follow this issue so that you can be aware of future developments.
I had the same problem. I can't explain the reason, but this example works: realpython github
This question already has answers here:
Converting matplotlib png to base64 for viewing in html template
(2 answers)
Save plot to image file instead of displaying it using Matplotlib
(24 answers)
How to serve static files in Flask
(24 answers)
Closed 4 years ago.
I'm very new to Flask and Matplotlib. I'd like to be able to show a simple chart I generated in some html, but I'm having a very hard time figuring out how. Here is my Python code:
from flask import Flask, render_template
import numpy as np
import pandas
import matplotlib.pyplot as plt
app = Flask(__name__)
variables = pandas.read_csv('C:\\path\\to\\variable.csv')
price =variables['price']
#app.route('/test')
def chartTest():
lnprice=np.log(price)
plt.plot(lnprice)
return render_template('untitled1.html', name = plt.show())
if __name__ == '__main__':
app.run(debug = True)
And here is my HTML:
<!doctype html>
<html>
<body>
<h1>Price Chart</h1>
<p>{{ name }}</p>
<img src={{ name }} alt="Chart" height="42" width="42">
</body>
</html>
You can generate the image on-the-fly in Flask URL route handler:
import io
import random
from flask import Response
from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
from matplotlib.figure import Figure
#app.route('/plot.png')
def plot_png():
fig = create_figure()
output = io.BytesIO()
FigureCanvas(fig).print_png(output)
return Response(output.getvalue(), mimetype='image/png')
def create_figure():
fig = Figure()
axis = fig.add_subplot(1, 1, 1)
xs = range(100)
ys = [random.randint(1, 50) for x in xs]
axis.plot(xs, ys)
return fig
Then you need to include the image in your HTML template:
<img src="/plot.png" alt="my plot">
As #d parolin pointed out, the figure generated by matplotlib will need to be saved before being rendered by the HTML. In order to serve images in flask by HTML, you will need to store the image in your flask file directory:
static/
images/
plot.png --> store plots here
templates/
Therefore, in your application, use plt.savefig:
#app.route('/test')
def chartTest():
lnprice=np.log(price)
plt.plot(lnprice)
plt.savefig('/static/images/new_plot.png')
return render_template('untitled1.html', name = 'new_plot', url ='/static/images/new_plot.png')
Then in untitled1.html:
<p>{{ name }}</p>
<img src={{ url}} alt="Chart" height="42" width="42">