I can't seem to figure out how to using Flask's streaming. Here's my code:
#app.route('/scans/')
def scans_query():
url_for('static', filename='.*')
def generate():
yield render_template('scans.html')
for i in xrange(50):
sleep(.5)
yield render_template('scans.html', **locals())
return Response(stream_with_context(generate()))
and in my template:
<p>{% i %}</p>
I would like to see a counter on the page that changes every half second. Instead, the closest I've gotten is the page printing out each number on the next line.
To replace existing content on the page you might need javascript i.e., you could send it or make it to make requests for you, use long polling, websockets, etc. There are many ways to do it, here's one that uses server send events:
#!/usr/bin/env python
import itertools
import time
from flask import Flask, Response, redirect, request, url_for
app = Flask(__name__)
#app.route('/')
def index():
if request.headers.get('accept') == 'text/event-stream':
def events():
for i, c in enumerate(itertools.cycle('\|/-')):
yield "data: %s %d\n\n" % (c, i)
time.sleep(.1) # an artificial delay
return Response(events(), content_type='text/event-stream')
return redirect(url_for('static', filename='index.html'))
if __name__ == "__main__":
app.run(host='localhost', port=23423)
Where static/index.html:
<!doctype html>
<title>Server Send Events Demo</title>
<style>
#data {
text-align: center;
}
</style>
<script src="http://code.jquery.com/jquery-latest.js"></script>
<script>
if (!!window.EventSource) {
var source = new EventSource('/');
source.onmessage = function(e) {
$("#data").text(e.data);
}
}
</script>
<div id="data">nothing received yet</div>
The browser reconnects by default in 3 seconds if the connection is lost. if there is nothing more to send the server could return 404 or just send some other than 'text/event-stream' content type in response to the next request. To stop on the client side even if the server has more data you could call source.close().
Note: if the stream is not meant to be infinite then use other techniques (not SSE) e.g., send javascript snippets to replace the text (infinite <iframe> technique):
#!/usr/bin/env python
import time
from flask import Flask, Response
app = Flask(__name__)
#app.route('/')
def index():
def g():
yield """<!doctype html>
<title>Send javascript snippets demo</title>
<style>
#data {
text-align: center;
}
</style>
<script src="http://code.jquery.com/jquery-latest.js"></script>
<div id="data">nothing received yet</div>
"""
for i, c in enumerate("hello"):
yield """
<script>
$("#data").text("{i} {c}")
</script>
""".format(i=i, c=c)
time.sleep(1) # an artificial delay
return Response(g())
if __name__ == "__main__":
app.run(host='localhost', port=23423)
I've inlined the html here to show that there is nothing more to it (no magic). Here's the same as above but using templates:
#!/usr/bin/env python
import time
from flask import Flask, Response
app = Flask(__name__)
def stream_template(template_name, **context):
# http://flask.pocoo.org/docs/patterns/streaming/#streaming-from-templates
app.update_template_context(context)
t = app.jinja_env.get_template(template_name)
rv = t.stream(context)
# uncomment if you don't need immediate reaction
##rv.enable_buffering(5)
return rv
#app.route('/')
def index():
def g():
for i, c in enumerate("hello"*10):
time.sleep(.1) # an artificial delay
yield i, c
return Response(stream_template('index.html', data=g()))
if __name__ == "__main__":
app.run(host='localhost', port=23423)
Where templates/index.html:
<!doctype html>
<title>Send javascript with template demo</title>
<style>
#data {
text-align: center;
}
</style>
<script src="http://code.jquery.com/jquery-latest.js"></script>
<div id="data">nothing received yet</div>
{% for i, c in data: %}
<script>
$("#data").text("{{ i }} {{ c }}")
</script>
{% endfor %}
I think if you're going to use templates like that, you might need to use the stream_template function given here: http://flask.pocoo.org/docs/patterns/streaming/#streaming-from-templates
I didn't test this, but it might look like:
def stream_template(template_name, **context):
app.update_template_context(context)
t = app.jinja_env.get_template(template_name)
rv = t.stream(context)
rv.enable_buffering(5)
return rv
#app.route('/scans/')
def scans_query():
url_for('static', filename='.*')
def generate():
for i in xrange(50):
sleep(.5)
yield i
return Response(stream_template('scans.html', i=generate()))
Related
I have a view that generates data and streams it in real time. I can't figure out how to send this data to a variable that I can use in my HTML template. My current solution just outputs the data to a blank page as it arrives, which works, but I want to include it in a larger page with formatting. How do I update, format, and display the data as it is streamed to the page?
import flask
import time, math
app = flask.Flask(__name__)
#app.route('/')
def index():
def inner():
# simulate a long process to watch
for i in range(500):
j = math.sqrt(i)
time.sleep(1)
# this value should be inserted into an HTML template
yield str(i) + '<br/>\n'
return flask.Response(inner(), mimetype='text/html')
app.run(debug=True)
You can stream data in a response, but you can't dynamically update a template the way you describe. The template is rendered once on the server side, then sent to the client.
One solution is to use JavaScript to read the streamed response and output the data on the client side. Use XMLHttpRequest to make a request to the endpoint that will stream the data. Then periodically read from the stream until it's done.
This introduces complexity, but allows updating the page directly and gives complete control over what the output looks like. The following example demonstrates that by displaying both the current value and the log of all values.
This example assumes a very simple message format: a single line of data, followed by a newline. This can be as complex as needed, as long as there's a way to identify each message. For example, each loop could return a JSON object which the client decodes.
from math import sqrt
from time import sleep
from flask import Flask, render_template
app = Flask(__name__)
#app.route("/")
def index():
return render_template("index.html")
#app.route("/stream")
def stream():
def generate():
for i in range(500):
yield "{}\n".format(sqrt(i))
sleep(1)
return app.response_class(generate(), mimetype="text/plain")
<p>This is the latest output: <span id="latest"></span></p>
<p>This is all the output:</p>
<ul id="output"></ul>
<script>
var latest = document.getElementById('latest');
var output = document.getElementById('output');
var xhr = new XMLHttpRequest();
xhr.open('GET', '{{ url_for('stream') }}');
xhr.send();
var position = 0;
function handleNewData() {
// the response text include the entire response so far
// split the messages, then take the messages that haven't been handled yet
// position tracks how many messages have been handled
// messages end with a newline, so split will always show one extra empty message at the end
var messages = xhr.responseText.split('\n');
messages.slice(position, -1).forEach(function(value) {
latest.textContent = value; // update the latest value in place
// build and append a new item to a list to log all output
var item = document.createElement('li');
item.textContent = value;
output.appendChild(item);
});
position = messages.length - 1;
}
var timer;
timer = setInterval(function() {
// check the response for new data
handleNewData();
// stop checking once the response has ended
if (xhr.readyState == XMLHttpRequest.DONE) {
clearInterval(timer);
latest.textContent = 'Done';
}
}, 1000);
</script>
An <iframe> can be used to display streamed HTML output, but it has some downsides. The frame is a separate document, which increases resource usage. Since it's only displaying the streamed data, it might not be easy to style it like the rest of the page. It can only append data, so long output will render below the visible scroll area. It can't modify other parts of the page in response to each event.
index.html renders the page with a frame pointed at the stream endpoint. The frame has fairly small default dimensions, so you may want to to style it further. Use render_template_string, which knows to escape variables, to render the HTML for each item (or use render_template with a more complex template file). An initial line can be yielded to load CSS in the frame first.
from flask import render_template_string, stream_with_context
#app.route("/stream")
def stream():
#stream_with_context
def generate():
yield render_template_string('<link rel=stylesheet href="{{ url_for("static", filename="stream.css") }}">')
for i in range(500):
yield render_template_string("<p>{{ i }}: {{ s }}</p>\n", i=i, s=sqrt(i))
sleep(1)
return app.response_class(generate())
<p>This is all the output:</p>
<iframe src="{{ url_for("stream") }}"></iframe>
5 years late, but this actually can be done the way you were initially trying to do it, javascript is totally unnecessary (Edit: the author of the accepted answer added the iframe section after I wrote this). You just have to include embed the output as an <iframe>:
from flask import Flask, render_template, Response
import time, math
app = Flask(__name__)
#app.route('/content')
def content():
"""
Render the content a url different from index
"""
def inner():
# simulate a long process to watch
for i in range(500):
j = math.sqrt(i)
time.sleep(1)
# this value should be inserted into an HTML template
yield str(i) + '<br/>\n'
return Response(inner(), mimetype='text/html')
#app.route('/')
def index():
"""
Render a template at the index. The content will be embedded in this template
"""
return render_template('index.html.jinja')
app.run(debug=True)
Then the 'index.html.jinja' file will include an <iframe> with the content url as the src, which would something like:
<!doctype html>
<head>
<title>Title</title>
</head>
<body>
<div>
<iframe frameborder="0"
onresize="noresize"
style='background: transparent; width: 100%; height:100%;'
src="{{ url_for('content')}}">
</iframe>
</div>
</body>
When rendering user-provided data render_template_string() should be used to render the content to avoid injection attacks. However, I left this out of the example because it adds additional complexity, is outside the scope of the question, isn't relevant to the OP since he isn't streaming user-provided data, and won't be relevant for the vast majority of people seeing this post since streaming user-provided data is a far edge case that few if any people will ever have to do.
Originally I had a similar problem to the one posted here where a model is being trained and the update should be stationary and formatted in Html. The following answer is for future reference or people trying to solve the same problem and need inspiration.
A good solution to achieve this is to use an EventSource in Javascript, as described here. This listener can be started using a context variable, such as from a form or other source. The listener is stopped by sending a stop command. A sleep command is used for visualization without doing any real work in this example. Lastly, Html formatting can be achieved using Javascript DOM-Manipulation.
Flask Application
import flask
import time
app = flask.Flask(__name__)
#app.route('/learn')
def learn():
def update():
yield 'data: Prepare for learning\n\n'
# Preapre model
time.sleep(1.0)
for i in range(1, 101):
# Perform update
time.sleep(0.1)
yield f'data: {i}%\n\n'
yield 'data: close\n\n'
return flask.Response(update(), mimetype='text/event-stream')
#app.route('/', methods=['GET', 'POST'])
def index():
train_model = False
if flask.request.method == 'POST':
if 'train_model' in list(flask.request.form):
train_model = True
return flask.render_template('index.html', train_model=train_model)
app.run(threaded=True)
HTML Template
<form action="/" method="post">
<input name="train_model" type="submit" value="Train Model" />
</form>
<p id="learn_output"></p>
{% if train_model %}
<script>
var target_output = document.getElementById("learn_output");
var learn_update = new EventSource("/learn");
learn_update.onmessage = function (e) {
if (e.data == "close") {
learn_update.close();
} else {
target_output.innerHTML = "Status: " + e.data;
}
};
</script>
{% endif %}
I am using this documentation to get the user to login and return me code and state. However, when I run my app It gives me a 500 error.
This is my app.py
import flask
import SpotifyOAuth
app = flask.Flask(__name__)
#app.route('/')
def index():
flask.render_template("index.html")
#app.route('/login')
def accessSpotify():
SpotifyOAuth.RedirectTologin(redirect_uri='https://localhost:7001/authorized')
#app.route('/authorized')
def SP_redirect_uri():
return "spotify connected"
def main():
print("nothin")
if __name__ == '__main__':
app.run(use_reloader=True,port=7001)
and this is my SpotifyOAuth.py
import requests
def RedirectTologin(redirect_uri="https://localhost:7001/authorized"):
token_uri="https://accounts.spotify.com/authorize"
method="GET"
params={
"client_id" : '<id>',
"response_type" : 'code',
"redirect_uri" : redirect_uri,
"scope" : 'user-read-email'
}
client_secret='<secret>'
r = requests.get(token_uri,params=params)
print(r)
if __name__=='__main__':
RedirectTologin()
I know my RedirectTLogin() is working because when I print r it gives me response code 200. Not sure where I am going wrong in app.py
Here is my index.html for reference
<html>
<body>
<div>
<div id="login">
<h1>First, log in to spotify</h1>
Log in
</div>
<div id="loggedin">
</div>
</div>
</body>
</html>
Does anyone see any issue? I don't have much experience working with APIs
Flask routes need to return something: a rendered template, a redirect, plain text with a status code, etc.
To fix your issue you need to add returns to index() and accessSpotify() as follows:
#app.route('/')
def index():
return flask.render_template("index.html")
#app.route('/login')
def accessSpotify():
SpotifyOAuth.RedirectTologin(redirect_uri='https://localhost:7001/authorized')
return flask.redirect(url_for('SP_redirect_uri'))
from flask import Flask, jsonify, request
from flask_restful import Resource, Api
app = Flask(__name__)
api = Api(app)
class display(Resource):
def get(self):
return jsonify({"message":"hello world"})
def post(self):
data = request.get_json() # status code
return jsonify({'data': data}), 201
class square(Resource):
def get(self, num):
return jsonify({'square': num**2})
api.add_resource(display, '/display')
api.add_resource(square, '/square/<int:num>')
driver function
if __name__ == '__main__':
app.run(port = '5008')
This is the html file created using angularjs.This sends http request to the url given and receives data and displays it on th html page.
<!DOCTYPE html>
<html>
<script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.6.9/angular.min.js"></script>
<body>
<div ng-app="myApp" ng-controller="myCtrl">
<p>Today's welcome message is:</p>
<h1>{{message}}</h1>
</div>
<p>The $http service requests a page on the server, and the response is set as the value of "myWelcome" variable.</p>
<script>
var app = angular.module('app', []);
app.controller("myCtrl", function($http) {
var app = this;
$http.get("http://127.0.0.1:5008/display")
.success(function(data) {
app.message = data;
})
});
</script>
</body>
</html>
I am getting this error while running the code:
Today's welcome message is:
{{message}}
The $http service requests a page on the server, and the response is set as the value of "myWelcome" variable.
Program description: I already have a functioning program that runs on console window, but I'd like to present its output on a locally hosted web page. The program consists on getting lyrics for currently playing songs by making requests to Spotify's API. I store the current lyrics in a "lyrics.txt" file.
What I want:
Change the web page from the running lyrics program when it detects the song has changed.
[EDIT:]
Is there a way to make the flask page display a variable, that is updated by a python request.post of the lyrics app to the flask url with the updated variable as the data?
What I have:
I'm using Flask as the framework since its a one local web page.
import os, io
from flask import Flask, render_template
app = Flask(__name__)
#app.route("/")
def test():
'''reads the current lyrics file and passes it to the web page
manually reload the page to update lyrics'''
with io.open('lyrics.txt', 'r') as f:
HEAD = f.readline().strip("\n")
BODY = f.read().split('\n')
lyrics = {"HEAD": HEAD, "BODY": BODY}
return render_template("home.html", lyrics=lyrics)
if __name__ == "__main__":
app.run(debug=1)
link to lyrics app github
You would need JavaScript/AJAX on page which periodically sends request for new content and Flask should send current content from file.
In this example I use jQuery.get() to send request to server, and setTimeout() to repeat it periodically.
Flask sends current time to show different content.
import datetime
from flask import Flask, render_template_string
app = Flask(__name__)
#app.route("/")
def index():
return render_template_string("""<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<title>Test</title>
<script type="text/javascript" src="http://code.jquery.com/jquery-1.8.0.min.js"></script>
<script type="text/javascript">
function updater() {
$.get('/data', function(data) {
$('#time').html(data); // update page with new data
});
};
setInterval(updater, 1000); // run `updater()` every 1000ms (1s)
</script>
</head>
<body>
Date & Time: <span id="time"><span>
</body>
</html>""")
#app.route('/data')
def data():
"""send current content"""
return datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
if __name__ == "__main__":
app.run(debug=True)
EDIT:
The same using standard fetch() without external libraries.
Code has to be after <span>
import datetime
from flask import Flask, render_template_string
app = Flask(__name__)
#app.route("/")
def index():
return render_template_string("""<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<title>Test</title>
</head>
<body>
Date & Time: <span id="time"><span>
<script type="text/javascript">
var time_span = document.getElementById("time");
function updater() {
fetch('/data')
.then(response => response.text())
.then(text => (time_span.innerHTML = text)); // update page with new data
}
setInterval(updater, 1000); // run `updater()` every 1000ms (1s)
</script>
</body>
</html>""")
#app.route('/data')
def data():
"""send current content"""
return datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
if __name__ == "__main__":
app.run(debug=True)
I have a view that generates data and streams it in real time. I can't figure out how to send this data to a variable that I can use in my HTML template. My current solution just outputs the data to a blank page as it arrives, which works, but I want to include it in a larger page with formatting. How do I update, format, and display the data as it is streamed to the page?
import flask
import time, math
app = flask.Flask(__name__)
#app.route('/')
def index():
def inner():
# simulate a long process to watch
for i in range(500):
j = math.sqrt(i)
time.sleep(1)
# this value should be inserted into an HTML template
yield str(i) + '<br/>\n'
return flask.Response(inner(), mimetype='text/html')
app.run(debug=True)
You can stream data in a response, but you can't dynamically update a template the way you describe. The template is rendered once on the server side, then sent to the client.
One solution is to use JavaScript to read the streamed response and output the data on the client side. Use XMLHttpRequest to make a request to the endpoint that will stream the data. Then periodically read from the stream until it's done.
This introduces complexity, but allows updating the page directly and gives complete control over what the output looks like. The following example demonstrates that by displaying both the current value and the log of all values.
This example assumes a very simple message format: a single line of data, followed by a newline. This can be as complex as needed, as long as there's a way to identify each message. For example, each loop could return a JSON object which the client decodes.
from math import sqrt
from time import sleep
from flask import Flask, render_template
app = Flask(__name__)
#app.route("/")
def index():
return render_template("index.html")
#app.route("/stream")
def stream():
def generate():
for i in range(500):
yield "{}\n".format(sqrt(i))
sleep(1)
return app.response_class(generate(), mimetype="text/plain")
<p>This is the latest output: <span id="latest"></span></p>
<p>This is all the output:</p>
<ul id="output"></ul>
<script>
var latest = document.getElementById('latest');
var output = document.getElementById('output');
var xhr = new XMLHttpRequest();
xhr.open('GET', '{{ url_for('stream') }}');
xhr.send();
var position = 0;
function handleNewData() {
// the response text include the entire response so far
// split the messages, then take the messages that haven't been handled yet
// position tracks how many messages have been handled
// messages end with a newline, so split will always show one extra empty message at the end
var messages = xhr.responseText.split('\n');
messages.slice(position, -1).forEach(function(value) {
latest.textContent = value; // update the latest value in place
// build and append a new item to a list to log all output
var item = document.createElement('li');
item.textContent = value;
output.appendChild(item);
});
position = messages.length - 1;
}
var timer;
timer = setInterval(function() {
// check the response for new data
handleNewData();
// stop checking once the response has ended
if (xhr.readyState == XMLHttpRequest.DONE) {
clearInterval(timer);
latest.textContent = 'Done';
}
}, 1000);
</script>
An <iframe> can be used to display streamed HTML output, but it has some downsides. The frame is a separate document, which increases resource usage. Since it's only displaying the streamed data, it might not be easy to style it like the rest of the page. It can only append data, so long output will render below the visible scroll area. It can't modify other parts of the page in response to each event.
index.html renders the page with a frame pointed at the stream endpoint. The frame has fairly small default dimensions, so you may want to to style it further. Use render_template_string, which knows to escape variables, to render the HTML for each item (or use render_template with a more complex template file). An initial line can be yielded to load CSS in the frame first.
from flask import render_template_string, stream_with_context
#app.route("/stream")
def stream():
#stream_with_context
def generate():
yield render_template_string('<link rel=stylesheet href="{{ url_for("static", filename="stream.css") }}">')
for i in range(500):
yield render_template_string("<p>{{ i }}: {{ s }}</p>\n", i=i, s=sqrt(i))
sleep(1)
return app.response_class(generate())
<p>This is all the output:</p>
<iframe src="{{ url_for("stream") }}"></iframe>
5 years late, but this actually can be done the way you were initially trying to do it, javascript is totally unnecessary (Edit: the author of the accepted answer added the iframe section after I wrote this). You just have to include embed the output as an <iframe>:
from flask import Flask, render_template, Response
import time, math
app = Flask(__name__)
#app.route('/content')
def content():
"""
Render the content a url different from index
"""
def inner():
# simulate a long process to watch
for i in range(500):
j = math.sqrt(i)
time.sleep(1)
# this value should be inserted into an HTML template
yield str(i) + '<br/>\n'
return Response(inner(), mimetype='text/html')
#app.route('/')
def index():
"""
Render a template at the index. The content will be embedded in this template
"""
return render_template('index.html.jinja')
app.run(debug=True)
Then the 'index.html.jinja' file will include an <iframe> with the content url as the src, which would something like:
<!doctype html>
<head>
<title>Title</title>
</head>
<body>
<div>
<iframe frameborder="0"
onresize="noresize"
style='background: transparent; width: 100%; height:100%;'
src="{{ url_for('content')}}">
</iframe>
</div>
</body>
When rendering user-provided data render_template_string() should be used to render the content to avoid injection attacks. However, I left this out of the example because it adds additional complexity, is outside the scope of the question, isn't relevant to the OP since he isn't streaming user-provided data, and won't be relevant for the vast majority of people seeing this post since streaming user-provided data is a far edge case that few if any people will ever have to do.
Originally I had a similar problem to the one posted here where a model is being trained and the update should be stationary and formatted in Html. The following answer is for future reference or people trying to solve the same problem and need inspiration.
A good solution to achieve this is to use an EventSource in Javascript, as described here. This listener can be started using a context variable, such as from a form or other source. The listener is stopped by sending a stop command. A sleep command is used for visualization without doing any real work in this example. Lastly, Html formatting can be achieved using Javascript DOM-Manipulation.
Flask Application
import flask
import time
app = flask.Flask(__name__)
#app.route('/learn')
def learn():
def update():
yield 'data: Prepare for learning\n\n'
# Preapre model
time.sleep(1.0)
for i in range(1, 101):
# Perform update
time.sleep(0.1)
yield f'data: {i}%\n\n'
yield 'data: close\n\n'
return flask.Response(update(), mimetype='text/event-stream')
#app.route('/', methods=['GET', 'POST'])
def index():
train_model = False
if flask.request.method == 'POST':
if 'train_model' in list(flask.request.form):
train_model = True
return flask.render_template('index.html', train_model=train_model)
app.run(threaded=True)
HTML Template
<form action="/" method="post">
<input name="train_model" type="submit" value="Train Model" />
</form>
<p id="learn_output"></p>
{% if train_model %}
<script>
var target_output = document.getElementById("learn_output");
var learn_update = new EventSource("/learn");
learn_update.onmessage = function (e) {
if (e.data == "close") {
learn_update.close();
} else {
target_output.innerHTML = "Status: " + e.data;
}
};
</script>
{% endif %}