I have two different pages, one (A) that displays data taken from a model object, and one (B) that changes its fields.
I would like that when the post data is sent from B to the server, the server changes the values in A.
What is the best way to do it?
This example could work for me but it's in PHP... there is a way to replicate it whit Python?
https://www.w3schools.com/html/html5_serversentevents.asp
This is working example from w3schools in Django:
template
<!DOCTYPE html>
<html>
<body>
<h1>Getting server updates</h1>
<div id="result"></div>
<script>
if(typeof(EventSource) !== "undefined") {
var source = new EventSource("stream/");
source.onmessage = function(event) {
document.getElementById("result").innerHTML += event.data + "<br>";
};
} else {
document.getElementById("result").innerHTML = "Sorry, your browser does not support server-sent events...";
}
</script>
</body>
</html>
views
import datetime
import time
from django.http import StreamingHttpResponse
def stream(request):
def event_stream():
while True:
time.sleep(3)
yield 'data: The server time is: %s\n\n' % datetime.datetime.now()
return StreamingHttpResponse(event_stream(), content_type='text/event-stream')
urls
urlpatterns = [
path('stream/', views.stream, name='stream')
]
Update:
If you want to manage your notifications you can create the model like:
from django.db import models
class Notification(models.Model):
text = models.CharField(max_length=200)
user = models.ForeignKey(User, on_delete=models.CASCADE)
sent = models.BooleanField(default=False)
Then create the view that is looking for the first unsent notification and sends it:
#login_required
def stream(request):
def event_stream():
while True:
time.sleep(3)
notification = Notification.objects.filter(
sent=False, user=request.user
).first()
text = ''
if notification:
text = notification.text
notification.sent = True
notification.save()
yield 'data: %s\n\n' % text
return StreamingHttpResponse(event_stream(), content_type='text/event-stream')
And the send_notification function that creates an entry in the Notification model (just call this function from anywhere in your code):
def send_notification(user, text):
Notification.objects.create(
user=user, text=text
)
After reading this, I think I understood the whole thing (please comment if I’m wrong).
Django does NOT natively support keep-alive connections.
This means, when the client gets the message from the server, the connection is immediately closed after (like any classic HTTP request/response cycle).
What’s different with text/event-stream request, is that the client automatically tries to reconnect to the server every second (the length can be changed with a retry parameter).
Unfortunately, it seems using SSE in that case has no interest since it has the same con’s that polling (ie. a request/response cycle occurs each X seconds).
As expected and mentioned in other answers, I would need django-channels to create a persistent connection that prevent HTTP request/response overheads and ensure the message is sent immediately.
As mentioned in other answers, you will need to use Django Channels to properly handle asynchronous communication without tying up threads.
For an example, see the django-eventstream library which uses Channels to implement SSE.
Related
I need some help understanding the concepts needed to get my application to work. I am running two instances of localhost, one on 5000: and one on 8000:. One is a Flask instance (backend) and the other is a VUE instance (frontend). On my frontend, I have an onSubmit() function that does two things:
Sends the form data to the Flask instance via axios.post
Redirects the frontend to a result webpage.
On my backend, my Flask instance is supposed to return the information I need.
if request.method == "GET":
#Accessing page for first time
response_object['msg'] = "Waiting for user input"
elif request.method == "POST":
#Form has been submitted
data = request.get_json()
user = parseFormData(data)
itinerary = getResults(user)
return itinerary
I am unsure / unable how to check if this is working. On the redirected page, I have the following method that is called when the page is created.
getResponse(){
const path = "http://127.0.0.1:5000";
axios.post(path)
.then((res) => {
this.tmp = res.data;
console.log(this.tmp);
})
.catch((err) => {
console.error(err);
});
},
I am unable to get anything from this page actually. Any advice on where to go from here will be appreciated!
Edit:
The problem appears to lie in the sequence of events. On my Frontend, the form is submitted then the page is immediately redirected to a result page. While this is happening, my Backend receives the data, processes it and returns the information. That information appears to be lost. I believe this is sort of an example of async but I am unfamiliar with that concept as of now.
I'm learning Django on the fly so bear with me. My goal here is for a user to be able to upload a file, the server script process it and send the results on each iteration to the client live.
My current structure is: User uploads a file (csv), this is read via views.py then renders the blank html view, JS in the html creates a new websocket, and well this is where I'm not able to pass data outside of consumers.py or process the data.
I'm also not sure the structure of view > JS script > consumers > ? is the best route for processing files but I've not found much documentation on file uploads and channels.
views.py
from django.shortcuts import render
import pandas as pd
def someFunc(request):
if request.method == "POST":
global csvUpload
csvUpload = pd.read_csv(request.FILES['filename'])
return render(request, 'app/appPost.html')
I render the view here first so that the blank page displays prior to the script running.
appPost.html JS script
var socket = new WebSocket('ws://localhost:8000/ws/app_ws/')
socket.onmessage = function(event){
var data = JSON.parse(event.data);
console.log(data);
var para = document.createElement("P");
para.innerText = data.message;
document.body.appendChild(para);
}
consumers.py
from channels.generic.websocket import WebsocketConsumer
from asgiref.sync import async_to_sync
class WSConsumer(WebsocketConsumer):
def connect(self):
self.accept()
self.render()
async_to_sync(self.add_group)('appDel_updates_group')
Ultimately I'd then like to pass what's in this alarm file to the websocket connection.
app_alarm.py
from requests.api import request
import pandas as pd
import requests as r
def app_process(csvUpload):
csvUpload=pd.read_csv(request.FILES['filename'])
csvFile = pd.DataFrame(csvUpload)
colUsernames = csvFile.usernames
for user in colUsernames:
req = r.get('https://reqres.in/api/users/2')
t = req.json()
data = t['data']['email']
message = user + " " + data
channel_layer = get_channel_layer()
async_to_sync(channel_layer.group_send)(
'appDel_updates_group',
{'type': 'render', 'message': message}
)
How do you do something after you render the view? (Django)
This seems somewhat relevant, I'm going to see about creating a thread to run the file processing only after the render is done in views.
Edit: Threading allowed me to render the page then pass the data over to the alarm.py file for processing. This has solved the issue I was having by adding the following code into the views.py:
x = threading.Thread(target=app_alarm.sub_process, args=[csvUpload])
x.setDaemon(False)
x.start()
return render(request, 'efax/appPost.html')
This allows views to grab the file using request.FILES and pass it to the script before the page was rendered without waiting on the completion of the processing. That would have rendered the entire point of channels in this situation pointless. Hopefully this helps someone.
Copying from the original post edit:
Threading allowed me to render the page then pass the data over to the alarm.py file for processing. This has solved the issue I was having by adding the following code into the views.py:
x = threading.Thread(target=app_alarm.sub_process, args=[csvUpload])
x.setDaemon(False)
x.start()
return render(request, 'efax/appPost.html')
This allows views to grab the file using request.FILES and pass it to the script before the page was rendered without waiting on the completion of the processing. That would have rendered the entire point of channels in this situation pointless. Hopefully this helps someone.
I have built a Django app which runs automated testing. I collect the inputs from the user, as to what all tests need to be run and when the user clicks on submit button, the tests run for a few hours to a couple of days (depending on the number of tests selected) and once all the tests are completed, the results are showed on a template.
Now the problem is, till all the tests are completed, the user is not updated with the progress. So I decided to use Django Channles to provide live update to the client as and when I have results for individual tests.
With my implementation, I call the Group('user-1').send({'text': i}) method in a for loop. But all I see in the template is only the output of the last Group('user-1').send({'text': i}) operation.
As per my requirement, once Group('user-1').send({'text': i}) is called, the socket.onmessage() function in the section of my template should receive the message and should again wait for messages sent by subsequent Group('user-1').send({'text': i}) calls.
I saw a very similar question here which is already answered, but the solution is not working for me. Please help me identify what needs to corrected in my code, shown below.
consumers.py
------------
#channel_session_user_from_http
def ws_connect(message, **kwargs):
http_user = message.user
if not http_user.is_anonymous:
message.reply_channel.send({'accept': True})
Group('user-'+str(http_user.id)).add(message.reply_channel)
def ws_message(message):
for i in my_results_list:
Group('user-1').send({'text': i})
sleep(1)
results.html
------------
<script>
// Note that the path doesn't matter for routing; any WebSocket
// connection gets bumped over to WebSocket consumers
socket = new WebSocket("ws://" + window.location.host + "/task/summary/");
socket.onopen = function() {
socket.send("Hello");
}
socket.onmessage = function(message) {
document.getElementById("demo").innerHTML = message.data;
}
socket.onmessage()
// Call onopen directly if socket is already open
// if (socket.readyState == WebSocket.OPEN) socket.onopen();
</script>
You are calling document.getElementById("demo").innerHTML = message.data; this overwrites the content of demo each time it is called. You want something like:
document.getElementById("demo").innerHTML += message.data
or
document.getElementById("demo").innerHTML += '<br />' + message.data
If this answer solves your problem please mark it correct.
I'm trying to wrap my head around the channel features of Google App Engine since they don't (easily) provide websockets.
My current situation is that I have a long work (file processing) that is being executed asynchronously via a worker.
This worker update the state of the file processing in the database at every lines in order to inform the customer.
From that current perspective, a F5 will indicate the evolution of the processing.
Now I'd like to implement a live update system. Of course I could do an XHR request every 5 seconds but a live connection seems better... introducing Channels since Websockets seems out of the possibilities.
From what I understood, I can channel.send_message to one client only, not to a "room". The issue here, is that the worker that process the file does not have any information which customer is currently connected (could be one, could be ten).
I could loop over all the customer and post to each client_id, suspecting that at least one of them will get the message, but this is awfully useless and too resourceful.
I was hoping there was a better way to achieve this ? Maybe a nice alternative to Google Channels feature without having to reconfigure my whole App Engine system (like for Websockets)?
One solution I can think of, which is not the absolute ideal but would be more suited, is to manage dedicated database tables (could also be implemented in Memcache) with :
A table that contains a list of rooms
A table that contains a list of client_id connected to the room
e.g. :
Rooms (id, name)
Clients (id, room_id, client_id)
Now, instead of posting to channel.send_message(client_id, Message), one would make a wrapper like this :
def send_to_room(room, message):
# Queries are SQLAlchemy like :
room = Rooms.query.filter(Rooms.name === room).first()
if not room:
raise Something
clients = Clients.query.filter(Rooms.room_id === room.id).all()
for client in clients:
channel.send_message(client.client_id, message)
And voilà, you have a Room like implementation in Google App Engine.
The drawback of this solution is to add two tables (or equivalent) in your database.
Does someone has better?
I am assuming that the long running task is being kicked off by the client.
So before you kick off the task make a ajax request from the client to a handler similar to this one. This handler has two things returned to the client. The token param which is used by the javascript api to create a channel, and a cid param which is used to determine which client created the channel.
from google.appengine.api import channel
#ae.route("/channel")
class CreateChannel(webapp2.RequestHandler):
def get(self):
cid = str(uuid.uuid4())
token = channel.create_channel(cid)
data = {
"cid":cid,
"token":token
}
self.response.write(json.dumps(data))
Now use the channel javascript api to create a new channel
https://cloud.google.com/appengine/docs/python/channel/javascript
var onClosed = function(resp){
console.log("channel closed");
};
var onOpened = function(resp){
console.log("channel created");
};
var onmessage = function(resp){
console.log("The client received a message from the backend task");
console.log(resp);
};
var channel_promise = $.ajax({
url: "/channel",
method: "GET"
});
channel_promise.then(function(resp){
//this channel id is important you need to get it to the backend process so it knows which client to send the message to.
var client_id = resp.data.cid;
var channel = new goog.appengine.Channel(resp.data.token);
handler = {
'onopen': $scope.onOpened,
'onmessage': $scope.onMessage,
'onerror': function () {
},
'onclose': function () {
alert("channel closed.")
}
};
socket = channel.open(handler);
//onOpened is the callback function to call after channel has been created
socket.onopen = onOpened;
//onClose is the callback function to call after channel has been closed
socket.onclose = onClosed;
//onmessage is the callback function to call when receiving messages from your task queue
socket.onmessage = onMessage;
});
Now we are all set up to listen for channel messages.
So when the user clicks the button we need to kickoff the backend task.
var letsDoSomeWorkOnClick = function(){
//make sure you pass the client id with every ajax request
$.ajax({
url: "/kickoff",
method: "POST",
params: {"cid":client_id}
});
}
Now the app engine handler to start the backend task queue. I use the deffered library to do this. https://cloud.google.com/appengine/articles/deferred
#ae.route("/kickoff")
KickOffHandler(webapp2.RequestHandler):
def post(self):
cid = self.request.get("cid")
req = {}
req['cid'] = cid
task = MyLongRunningTask()
deferred.defer(task.long_runner_1, req, _queue="my-queue")
example task:
class MyLongRunningTask:
def long_runner_1(self,req):
# do a whole bunch of stuff
channel.send_message(req["cid"], json.dumps({"test":"letting client know task is done"})
Is there was a easy way to validate unique elements in a user entered form? I found ways to do it in php(lookup values after user enters it in table) but not sure how to do it using jinja2
For example, say a user is signing up and I require all email addresses to be unique, how can I let them know its not unique before they submit the results(like this: https://github.com/signup/free if you are already a member if you enter your email it'll tell you it exists as soon as you go to the next line. I've seen many sites do this)? Right now I can catch the errors but it refreshes the page and gives them an error and I'd like to let them know their form will fail before they hit submit. In this example, I'm checking for something that does not exist but in some other examples I'll want to check if something in the database exists.
I understand I'll need to query my database but I'm unsure how to do it from the template itself or if there's another way to do it.
You can do an XMLHttpRequest ( https://developer.mozilla.org/en/using_xmlhttprequest ) with Javascript to POST the input box's text (even easier if you use jQuery, as in the upcoming example) and do a DB query server-side to see if the email (input box text) is unique. Accordingly, you would return a response from this view, where xhr=True in the #view_config() decorator for non-IE browsers and request.response.content_type='text/html' is set for IE browsers. For example:
#view_config(permission='view', route_name='check_email', renderer='json') ##for IE browsers
#view_config(permission='view', route_name='check_email', renderer='json', xhr=True) ##for non-IE
def check_email(request):
email= request.POST['email']
dbquery = DBSession.query(User).filter(User.email==email).first()
## if the email exists in the DB
if dbquery:
msg = 'used'
## else if the email is available
else:
msg = 'available'
if request.is_xhr:
return {'msg':msg}
else: # for IE browser
request.response.content_type = 'text/html'
return Response(json.dumps({'msg':msg}))
You can do a POST easily (client-side) by using a library such as jQuery to take care of the XMLHttpRequest. After including the jQuery library in your template, as well as the .js file with you script:
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.7.2/jquery.min.js" type="text/javascript"></script>
<script type='text/javascript' src="{{request.static_url('tutorial:static/myscript.js')}}"></script>
And then in myscript.js do:
$(function() {
$('#email').on('blur', postEmail)
})
// this function POSTs the entered email
function postEmail() {
var email = $(this).val()
$.post('/check_email', email, callback)
}
// this function is used to do something with the response returned from your view
function callback(data) {
if (data['msg'] === 'used') {
alert('used')
}
else if (data['msg'] === 'available') {
alert('available')
}
}
Instead of using javascript or Jquery, you could do a message system, passing it to the renderer through a dict. For example:
message = ''
if 'form.submitted' in request.params:#form.submitted being the name of the submit of the form
#check if email exists
if exists:
message = 'email already registered'
else:
message = 'success'
return dict(message = message)
Just a way of doing it without the JS