I have a view that generates data and streams it in real time. I can't figure out how to send this data to a variable that I can use in my HTML template. My current solution just outputs the data to a blank page as it arrives, which works, but I want to include it in a larger page with formatting. How do I update, format, and display the data as it is streamed to the page?
import flask
import time, math
app = flask.Flask(__name__)
#app.route('/')
def index():
def inner():
# simulate a long process to watch
for i in range(500):
j = math.sqrt(i)
time.sleep(1)
# this value should be inserted into an HTML template
yield str(i) + '<br/>\n'
return flask.Response(inner(), mimetype='text/html')
app.run(debug=True)
You can stream data in a response, but you can't dynamically update a template the way you describe. The template is rendered once on the server side, then sent to the client.
One solution is to use JavaScript to read the streamed response and output the data on the client side. Use XMLHttpRequest to make a request to the endpoint that will stream the data. Then periodically read from the stream until it's done.
This introduces complexity, but allows updating the page directly and gives complete control over what the output looks like. The following example demonstrates that by displaying both the current value and the log of all values.
This example assumes a very simple message format: a single line of data, followed by a newline. This can be as complex as needed, as long as there's a way to identify each message. For example, each loop could return a JSON object which the client decodes.
from math import sqrt
from time import sleep
from flask import Flask, render_template
app = Flask(__name__)
#app.route("/")
def index():
return render_template("index.html")
#app.route("/stream")
def stream():
def generate():
for i in range(500):
yield "{}\n".format(sqrt(i))
sleep(1)
return app.response_class(generate(), mimetype="text/plain")
<p>This is the latest output: <span id="latest"></span></p>
<p>This is all the output:</p>
<ul id="output"></ul>
<script>
var latest = document.getElementById('latest');
var output = document.getElementById('output');
var xhr = new XMLHttpRequest();
xhr.open('GET', '{{ url_for('stream') }}');
xhr.send();
var position = 0;
function handleNewData() {
// the response text include the entire response so far
// split the messages, then take the messages that haven't been handled yet
// position tracks how many messages have been handled
// messages end with a newline, so split will always show one extra empty message at the end
var messages = xhr.responseText.split('\n');
messages.slice(position, -1).forEach(function(value) {
latest.textContent = value; // update the latest value in place
// build and append a new item to a list to log all output
var item = document.createElement('li');
item.textContent = value;
output.appendChild(item);
});
position = messages.length - 1;
}
var timer;
timer = setInterval(function() {
// check the response for new data
handleNewData();
// stop checking once the response has ended
if (xhr.readyState == XMLHttpRequest.DONE) {
clearInterval(timer);
latest.textContent = 'Done';
}
}, 1000);
</script>
An <iframe> can be used to display streamed HTML output, but it has some downsides. The frame is a separate document, which increases resource usage. Since it's only displaying the streamed data, it might not be easy to style it like the rest of the page. It can only append data, so long output will render below the visible scroll area. It can't modify other parts of the page in response to each event.
index.html renders the page with a frame pointed at the stream endpoint. The frame has fairly small default dimensions, so you may want to to style it further. Use render_template_string, which knows to escape variables, to render the HTML for each item (or use render_template with a more complex template file). An initial line can be yielded to load CSS in the frame first.
from flask import render_template_string, stream_with_context
#app.route("/stream")
def stream():
#stream_with_context
def generate():
yield render_template_string('<link rel=stylesheet href="{{ url_for("static", filename="stream.css") }}">')
for i in range(500):
yield render_template_string("<p>{{ i }}: {{ s }}</p>\n", i=i, s=sqrt(i))
sleep(1)
return app.response_class(generate())
<p>This is all the output:</p>
<iframe src="{{ url_for("stream") }}"></iframe>
5 years late, but this actually can be done the way you were initially trying to do it, javascript is totally unnecessary (Edit: the author of the accepted answer added the iframe section after I wrote this). You just have to include embed the output as an <iframe>:
from flask import Flask, render_template, Response
import time, math
app = Flask(__name__)
#app.route('/content')
def content():
"""
Render the content a url different from index
"""
def inner():
# simulate a long process to watch
for i in range(500):
j = math.sqrt(i)
time.sleep(1)
# this value should be inserted into an HTML template
yield str(i) + '<br/>\n'
return Response(inner(), mimetype='text/html')
#app.route('/')
def index():
"""
Render a template at the index. The content will be embedded in this template
"""
return render_template('index.html.jinja')
app.run(debug=True)
Then the 'index.html.jinja' file will include an <iframe> with the content url as the src, which would something like:
<!doctype html>
<head>
<title>Title</title>
</head>
<body>
<div>
<iframe frameborder="0"
onresize="noresize"
style='background: transparent; width: 100%; height:100%;'
src="{{ url_for('content')}}">
</iframe>
</div>
</body>
When rendering user-provided data render_template_string() should be used to render the content to avoid injection attacks. However, I left this out of the example because it adds additional complexity, is outside the scope of the question, isn't relevant to the OP since he isn't streaming user-provided data, and won't be relevant for the vast majority of people seeing this post since streaming user-provided data is a far edge case that few if any people will ever have to do.
Originally I had a similar problem to the one posted here where a model is being trained and the update should be stationary and formatted in Html. The following answer is for future reference or people trying to solve the same problem and need inspiration.
A good solution to achieve this is to use an EventSource in Javascript, as described here. This listener can be started using a context variable, such as from a form or other source. The listener is stopped by sending a stop command. A sleep command is used for visualization without doing any real work in this example. Lastly, Html formatting can be achieved using Javascript DOM-Manipulation.
Flask Application
import flask
import time
app = flask.Flask(__name__)
#app.route('/learn')
def learn():
def update():
yield 'data: Prepare for learning\n\n'
# Preapre model
time.sleep(1.0)
for i in range(1, 101):
# Perform update
time.sleep(0.1)
yield f'data: {i}%\n\n'
yield 'data: close\n\n'
return flask.Response(update(), mimetype='text/event-stream')
#app.route('/', methods=['GET', 'POST'])
def index():
train_model = False
if flask.request.method == 'POST':
if 'train_model' in list(flask.request.form):
train_model = True
return flask.render_template('index.html', train_model=train_model)
app.run(threaded=True)
HTML Template
<form action="/" method="post">
<input name="train_model" type="submit" value="Train Model" />
</form>
<p id="learn_output"></p>
{% if train_model %}
<script>
var target_output = document.getElementById("learn_output");
var learn_update = new EventSource("/learn");
learn_update.onmessage = function (e) {
if (e.data == "close") {
learn_update.close();
} else {
target_output.innerHTML = "Status: " + e.data;
}
};
</script>
{% endif %}
Related
I have an arduino program and set-up which detects the number of cars from a parking lot. The number of cars is printed on the serial every time when the sensor detects a car in front of the barrier.
I want to print the number of cars from the parking lot into a small web app. I use Tera Term to scan my serial bus and put the output data into a file text ( data.txt) . Then i use python to read the value from that text file and render it into a HTML page on a web app.
Here is the python code :
from flask import Flask, render_template
import logging
import requests
# Initialize the Flask application
app = Flask(__name__)
# Define a route for the default URL, which loads the form
#app.route("/")
def form():
with open('date.txt') as f:
data = []
lines = f.read().splitlines()
data.append(lines)
for i in data:
return render_template('index.html', variable=data)
if __name__ == "__main__":
app.run()
here is index.html
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Arduino Project</title>
<style>
body{
background: antiquewhite;
}
h1 {color:red;
text-align: center;}
h2 {color:blue;
text-align: center;}
</style>
</head>
<body>
<h1> - Arduino - </h1>
<h2> Number of cars in the parking place: {{ variable }}<br></h2>
<script> setTimeout(function(){
window.location.reload(1);
}, 5000);
</script>
</body>
</html>
It works fine, but i want to have only a variable which updates every 5 seconds,when the page is refreshed.I dont want to see all values from the date.txt,just the last one.
This is how my web app looks like with this code until now :
(ignore the error message)
enter image description here
Returning multiples times is not possible.
A return statement is used to end the execution of the function call
and “returns” the result (value of the expression following the return
keyword) to the caller. The statements after the return statements are
not executed.
So this part could be adjusted:
for i in data:
return render_template('index.html', variable=data)
You said you want the last line only. So replace above with this:
return render_template('index.html', variable=data[-1])
Documentation
Getting the last element of a list
https://www.geeksforgeeks.org/python-return-statement/
I have been struggling trying to find a way to update graphs on a flask webserver. I stored the images in the static file of my directory and accessed them by {{url_for('static',filname=ph_plot.png) }} but everytime I would send a post request to fetch a new range of data the graph would not update on my webserver but on my filesystem it would. I know I can change the name of the file everytime I save it to make it appear but I dont know if that is an optimal way to display a dynamically changing photo.
Currently I have been using the send_from_directory method in flask but with it hasnt worked for me either. Below is my code.
I have been working on this for a while and would love some help! Thank you
Notes: all_plots.ph_plot() is calling a function from another python program.
FLASK CODE:
#app.route('/read_ph', methods=["GET", "POST"])
def ph_plot():
if request.method == "POST":
a = request.form['read_ph']
all_plots.ph_plot(a)
time.sleep(3)
ph_plot = os.path.join(os.getcwd(), "images/ph_plot.png")
return render_template('live_stream.html', image_name=ph_plot)
#app.route('/read_ph/<ph_plot>', methods=["GET", "POST"])
def send_ph(ph_plot):
return send_from_directory("images", ph_plot)
HTML:
<html>
<body>
<h1>Data Monitoring Station</h1>
<h2>PH</h2>
<form method="POST" action="read_ph" >
<input name="read_ph" placeholder="Instances" type="text">
</form>
<button type="button">PH Graph</button>
<img src="{{ url_for('send_ph',ph_plot=image_name) }}" id="plot" width ="220" height ="220">
<hr>
<h5> Return to main page RETURN</h5>
<hr>
</body>
</html>
send_from_directory is generally for files that have actually been uploaded into a directory from a user. This is not what you're actually trying to do; you're:
Generating the plot data
Creating a plot and spending time with matplotlib rendering it
Saving this plot image to disk
Loading that image back off disk and sending it to a user
Cut out the middleman here of disk storage: create the data and send it straight to the template. Here's a crude example using plotly.js in a single file to get the data rendered on the front end. You can keep refreshing the page to get different graphs. But note, each plot is interactive; you can zoom, for example, export etc. things with the menu in the top right, show/hide the plot (which would make more sense if there were multiple traces). You don't get any of that by rendering a plot image.
from flask import Flask, render_template_string
import random
app = Flask(__name__)
# Normally you'd have this in the templates directory but for simplicity of
# putting everything into one file for an example, I'm using a template string
template = """
<script src="https://cdn.plot.ly/plotly-latest.min.js"></script>
<div id="plot_div" style="width: 100%; height: 100%"></div>
<script type="text/javascript">
var trace1 = {
x: {{ plot_data.x_axis }},
y: {{ plot_data.y_axis }},
type: 'scatter',
name: 'Example'
};
var layout = {
title: "Test",
titlefont: {
family: 'Poppins',
size: 18,
color: '#7f7f7f'
},
showlegend: true,
xaxis: {
title: 'Axis Unit',
},
yaxis: {
title: 'Other Axis Unit',
},
margin: {
l: 70,
r: 40,
b: 50,
t: 50,
pad: 4
}
};
var data = [trace1];
Plotly.newPlot("plot_div", data, layout);
</script>
"""
def get_plot_data():
x = list(range(10))
y = [random.randint(0, 100) for i in range(10)]
return {'x_axis': x, 'y_axis': y}
#app.route('/')
def home():
plot_data = get_plot_data()
return render_template_string(template,
plot_data=plot_data)
if __name__ == '__main__':
app.run()
A common issue here is in passing datetime strings because they will be escaped. In this case, you'll need to use x: {{ plot_data.x_axis | safe }}. See Passing HTML to template using Flask/Jinja2 and How to make html markup show up?
Problem
I am trying to display image-files using AJAX in Flask. More specifically, I want to display an image once a button is clicked and display the next image once the button is clicked again (like a slide-show). The filenames of the images are stored in my database. I query the database to get a list of filenames for the current user and combine each filename with the rest of the path (path to where the images are stored on disk) in order to display the image.
So far, I manage to get the first image of the current user. However, I can't figure out a way to keep track of which image is the next one to show.
I tried using a global variable as a counter (file_counter) which should serve as an index. I want to increase file_counter by 1 each time an ajax request is made in order to get the next file but the counter does not increase upon subsequent calls nor does it throw an error.
Question
How do I need to initialize the global variable (file_counter) in order for it to store its value across multiple calls? Furthermore, is the usage of global variables the correct way of doing this?
HTML
<div id="ajax-field"></div>
<button class="btn btn-block" id="next-button"><p>Next Image!</p></button>
AJAX:
$('#next-button').click(function(){
$("#ajax-field").text("");
$.ajax({
url: "/get_data",
type: "POST",
success: function(resp){
$('#ajax-field').append(resp.data);
}
});
});
Routing:
global filenames
global file_count
#app.route("/get_data", methods=['POST'])
def get_data():
try: # Is intended to fail on the first run in order for the global variables to be initialized. However it keeps failing on subsequent runs
display_img = filenames[file_count]
file_count +=1
except:
filenames = []
# current_user.uploads returns all file-objects of the current user
user_uploads = current_user.uploads
for file in user_uploads:
# file.filename returns the respective filename of the image
filenames.append(file.filename)
#filenames is now a list of filenames i.e. ['a.jpg','b.jpg','c.jpg'...]
display_img = filenames[0]
file_count = 1
path = "image_uploads/4_files/"+display_img
return jsonify({'data': render_template('ajax_template.html', mylist = path)})
ajax_template.html:
<ul>
{% block content %}
<li>
<img id="selected-image-ajax" src="{{url_for('static',filename=mylist)}}" class="img-thumbnail" style="display:block; margin:auto;"></img>
</li>
{% endblock content %}
</ul>
As #roganjosh pointed out, a session is the optimal way to store information across multiple requests. This solution presents an implementation of the photo display using flask.session to store the counter:
import flask, random, string
app = flask.Flask(__name__)
app.secret_key = ''.join(random.choice(string.printable) for _ in range(20))
#to use flask.session, a secret key must be passed to the app instance
#app.route('/display_page', methods=['GET'])
def display_page():
'''function to return the HTML page to display the images'''
flask.session['count'] = 0
_files = [i.filename for i in current_user.uploads]
return flask.render_template('photo_display.html', photo = _files[0])
#app.route('/get_photo', methods=['GET'])
def get_photo():
_direction = flask.request.args.get('direction')
flask.session['count'] = flask.session['count'] + (1 if _direction == 'f' else - 1)
_files = [i.filename for i in current_user.uploads]
return flask.jsonify({'photo':_files[flask.session['count']], 'forward':str(flask.session['count']+1 < len(_files)), 'back':str(bool(flask.session['count']))})
The display_page function will be called when a user accesses the /display_page route and will set the count to 0. get_photo is bound to the /get_photo route and will be called when the ajax request is sent.
photo_display.html:
<html>
<head>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script>
</head>
<body>
<div class='image_display'>
<img src="{{photo}}" id='photo_display' height="100" width="100">
<table>
<tr>
<td class='back'></td>
<td class='forward'><button id='go_forward' class='navigate'>Forward</button></td>
</tr>
</table>
</div>
</body>
<script>
$(document).ready(function(){
$('.image_display').on('click', '.navigate', function(){
var direction = 'b';
if ($(this).prop('id') === 'go_forward'){
direction = 'f';
}
$.ajax({
url: "/get_photo",
type: "get",
data: {direction: direction},
success: function(response) {
$('#photo_display').attr('src', response.photo);
if (response.back === "True"){
$('.back').html("<button id='go_back' class='navigate'>Back</button>")
}
else{
$('#go_back').remove();
}
if (response.forward === "True"){
$('.forward').html("<button id='go_forward' class='navigate'>Forward</button>")
}
else{
$('#go_forward').remove();
}
},
});
});
});
</script>
</html>
The javascript in display_page.html communicates with the backend, and updates the img tag src accordingly. The script adds or removes the navigation buttons, depending on the current count value.
Demo:
To test the solution above, I created an image folder to store random photographs to display:
I need your help, In my view i run a bash process that takes time depending on the size of the image; while it process i want to display a loading gif image and a sentence (such as "Please wait, the image is being processed"). I tried to did that with a template but it is rendered at the end of execution of the script not while it process.
Can someone help me to do that ? I've found a similar question "https://stackoverflow.com/questions/8317219/django-show-loading-message-during-long-processing" but the answer wasn't very clear for me because i never used ajax.
This is my view :
def process(request):
var = Image.objects.order_by('id').last()
subprocess.call("./step1.sh %s" % (str(var)), shell=True)
subprocess.call("./step2.sh %s" % (str(var)), shell=True)
return render(request, 'endexecut.html')
Template that will be displayed at the end of processing: "endexecut.html"
{% extends 'base.html' %}
{% block content %}
<div class="container">
<div class="row">
<div class="jumbotron">
<div class="row">
<center>
<p> Your image is processed succesfully ! </p>
</center>
</div>
</div>
</div>
</div>
{% endblock %}
You should have a look at Celery which allows you to create jobs and monitor them. This would replace your process(request). With your own process(request) you would have to implement that by yourself, and why do that if solutions for it exist already.
The state of your running subprocess can be queried like this using JS from your browser (AJAX):
def task_progress_json(request, job_id):
"""
A view to report the progress to the user
:param job_id:
:param request:
"""
job = AsyncResult(job_id)
data = job.result or job.state
if job.state == states.FAILURE or isinstance(data, Exception):
data = {'failure': "{}: {}".format(_('Error'), data)}
if job_id in request.session:
request.session[job_id] = None
else:
# see http://docs.celeryproject.org/en/latest/reference/celery.states.html#std:state-PROPAGATE_STATES
# and http://docs.celeryproject.org/en/latest/getting-started/next-steps.html#calling-tasks
if data == states.PENDING:
last_state = request.session.get(job_id, None)
if last_state == states.PENDING:
data = {'failure': "Error: No Job Running"}
request.session[job_id] = None
else:
request.session[job_id] = data
if isinstance(data, dict):
next_url = data.pop('next_url', None)
next_kwargs = data.pop('next_kwargs', {})
if next_url:
data['next_url'] = reverse(next_url, kwargs=next_kwargs)
return HttpResponse(json.dumps(data), content_type='application/json')
In the view that triggers the process, for example when the user clicks on the Submit button of a form, you start the job:
def form_valid(self, form):
# this is just an example, implement as you like
fs = FileSystemStorage(location=settings.IMAGE_ROOT)
stored_file = fs.save(form.cleaned_data['file'].name, form.cleaned_data['file'])
job = import_imag_task.delay(stored_file, self.request.user.id, self.spectre.id)
# this is another view that shows the progress bar
# the process of the progress bar is displayed according to
# results fetched from the above task_progress_json request via AJAX
return HttpResponseRedirect(reverse('import_image_progress') + '?job=' + job.id)
Your best shot is to use ajax. You can simply rewrite your view as:
def process(request):
return render(request, "endexecut.html")
Display the gif image in your endexecut template by default with whatever text you want to use.
In the script section of the template page, you can then have an ajax call like this:
$.ajax({
url: '/some-app/some-view/',
type: 'GET',
dataType: 'json',
success: function(result){
$('#image-id').attr('src', result.image);
},
error: function(xhr){
alert(xhr.responseText); //Remove this when all is fine.
}
});
Ajax is pointing to a url (Which you should have defined in your url.py), the url would point to the view that would do your long bash processing:
def bash_processing(request):
# Do your bash processing here
return JsonResponse({'image': 'image url here'})
That should work fine for you.
However, if the processing would take a long time, you may consider using celery to do the processing in background.
I had the same issues and used jquery and an ajaxcall. This way you can display the gif when you launch the upload and remove the gif when it's done.
I have 1 very simple web application I am building right now but am very new to flask and jinja (and web development as a whole actually).
I have a watch folder, which will be getting an image sent to it via ftp on a pulse for ever. This wtch folder will only ever have one image in. Every 1 minute, the old image is replaced by a new image, with a new timestamp.
I would like to dynamically update the page, (and displayed timestamp) on a pulse as well, without having to reload any banners or static images that I will add later. I only want to update the following two lines out of the "Channels.Jinja" sample to follow.
<br>{{screenshot_datetime}}<br/>
<img src={{screenshot_location}} width="100%"/>
Channels.Jinja
<!DOCTYPE HTML>
<html>
<head>
<title>Training</title>
</head>
<body bgcolor=white>
<div id=main>
<br>Date and Time of Screenshot <br/>
<br>{{screenshot_datetime}}<br/>
<img src={{screenshot_location}} width="100%"/>
</div>
<div id='test'>
<p>
<script>
var myVar=setInterval(function(){get_image()},1000);
function get_image() {
$.ajax({
type: 'GET',
cache: false,
url: 'get_data',
success: function({{data}}) {
$('img').attr('src', data);
}
});
}
</script>
</p>
</div>
</body>
</html>
Channels.py
def render_channel_route(cr):
static_folder = os.path.join('static',cr)
file_list = os.listdir(static_folder)
channel_files = [f for f in file_list if f.startswith(cr)]
if not channel_files :
logger.error('Could not find image file for Channel. File should start with {0}'.format(cr))
abort(404)
img = os.path.join(static_folder,file_list[0])
ts = get_time_from_filename(file_list[0],cr)
return render_template('Channels.jinja',screenshot_datetime=time.strftime('%c',ts),screenshot_location=img)
#app.route('/channel01-10')
def first_tab():
return render_channel_route('channel01-10')
#app.route('/get_data', methods=['GET'])
def get_data():
return render_template('Channels.jinja',
screenshot_datetime=time.strftime('%c',ts),screenshot_location=img)
Im at a loss, Ive been bumbling around for a while now. Any and all advice is welcome! I am seeing a 304 response upon refresh, but not even the timer i am trying to put on it is working. Pardon sloppy code, highly volatile code is getting changed often -_-
I don't know it there is a "special" way to deal with Ajax using some Flask extension, but in the "normal" Ajax flow first you need to use url_for to put the correct url in your Ajax call and return the data formatted in some way (in my example in JSON) and not to render the template again:
$.ajax({
type: 'GET',
cache: false,
url: "{{ url_for('get_data') }}",
success: function(resp){
$('img').attr('src', resp.url);
$('#sst').html(resp.time);
}
});
So, in your get_data function in your controller you have to get the time and the path again for your image an then return some like this (to fit in my example before):
from flask import json
#app.route('/get_data', methods=['GET'])
def get_data():
#get time and path
time=...
path=...
return json.dumps({time:time,url:path}), 200, {'Content-Type':'application/json'}
Look that I use $('#sst') so you have to put in your HTML:
<br><span id='sst'>{{screenshot_datetime}}</span><br/>