We Keep Coding

Send response to EventSource after webook (Flask + uWSGI + nginx)

I have Flask application with route (webhook) receiving POST requests (webhooks) from external phone application (incomming call = POST request). This route sets threading.Event.set() and based on this event, another route (eventsource) sends an event stream to opened EventSource connection on a webpage created by yet another route (eventstream).
telfa_called = Event()
telfa_called.clear()
call = ""
#telfa.route('/webhook', methods=['GET', 'POST'])
def webhook():
global call
print('THE CALL IS HERE')
x = request.data
y = ET.fromstring(x.decode())
caller_number = y.find('caller_number').text
telfa_called.set() # setting threading.Event for another route
return Response(status=200)
#telfa.route('/eventstream', methods = ['GET','POST'])
#login_required
def eventstream():
jsid = str(uuid.uuid4())
return render_template('telfa/stream.html', jsid=jsid)
def eventsource_gen():
while 1:
if telfa_called.wait(10):
telfa_called.clear()
print('JE TO TADY')
yield "data: {}\n\n".format(json.dumps(call))
#telfa.route('/eventsource', methods=['GET', 'POST'])
def eventsource():
return Response(eventsource_gen(), mimetype='text/event-stream')`
Everything works great when testing in pure Python application. The problem starts, when I move this to production server, where I use uWSGI with nginx. (Other parts of this Python application work without any troubles.)
When the eventSource connection is opened and incomming webhook should be processed, whole flask server stucks (for all other users, too), page stops to load and I cannot find, where the error is.
I only know, the POST request from external application is received, but the response to EventSource is not made.
I suspect it has something to do with processes - the EventSource connection from JavaScript is one process, the webhook route another - and they do not communicate. So or so, I suppose this has to have very trivial solution, but I didn't find it in past 3 days and nights. Any hints, please? Thanks in advance.
To be complete, this my uwsgi config file:
[uwsgi]
module = wsgi:app
enable-threads = true
master = true
processes = 5
threads = 2
uid = www-data
gid= www-data
socket = /tmp/myproject.sock
chmod-socket = 666
vacuum = true
die-on-term = true
limit-as=512
buffer-size = 512000
workers = 5
max-requests = 100
req-logger = file:/tmp/uwsg-req.log
logger = file:/tmp/uwsgi.log`

SafeConfigParser overriding files

I am making a chat application, and need a conf file for the server and another for the client. However, whenever I run the client its conf properties get transferred to the configuration file of the server even though I am using SafeConfigParser. Is there any way to fix this?
Thanks.
when server starts:
config = configparser.SafeConfigParser()
config.read('chatserver.conf')
config['PORT']['port'] = str(self.port)
with open("chatserver.conf","w") as configfile:
config.write(configfile)
when a client joins:
clientconf = configparser.SafeConfigParser()
clientconf.read('chatclient.conf')
clientconf['SERVER']['last_server_used'] = str(self.host)
clientconf['SERVER']['port_used'] = str(self.port)
with open("chatserver.conf","w") as confFile:
clientconf.write(confFile)
chatclient.conf:
[SERVER]
last_server_used = '127.0.0.1'
port_used = '50000'
default_debug_mode = False
log = True
default_log_file = chat.log
chatserver.conf:
[PORT]
port = '1000'
When I run the server, a client joins the chat, and close it, then run the server again, the chatserver.conf file becomes the same as chatclient.conf

In the line:
with open("chatserver.conf","w") as confFile:
clientconf.write(confFile)
You save clientconf as chatserver.conf. I think you mean to save as chatclient.conf instead.

How can I have my flask-app run on my sites subdomain?

For example, I want to run flask-app on http://api.domain.com . However, I have no idea how to do this and the flask documentation serves no help. I am using a shared namecheap web server via SSH to run python. I have ports 8080, 8181 and 8282 open.
Server-sided code:
from flask import Flask
from flask import Blueprint
app = Flask(__name__)
app.config['SERVER_NAME'] = 'domain.com'
#app.route('/status')
def status():
return 'Status : Online'
bp = Blueprint('subdomain', __name__, subdomain="api")
app.register_blueprint(bp)
if __name__ == '__main__':
app.run(host=app.config["SERVER_NAME"],port=8181,debug=True)
When I visit http://www.api.domain.com/status , it returns a 404 error.
Nothing displays on the SSH console.
Any help if very much appreciated.

First things first:
http (i.e. a web server without a SSL certificate) is insecure. You should set up a certificate and always use port 443 to the outside.
Then, on namecheap, you need to define a CNAME entry to point to the subdomain.
In Namecheap, click domain -> Manage, then Advanced DNS
Create a new record, select CNAME as the Type, and enter the subdomain name (just the top level) as the HOST, then the IP where you server is as the value (TTL (time to live) is the time it takes to change when you want to change it next time, 1, 10min is useful to debug stuff, but DNS may not honor that anyways...)
Wait a few minutes, and you should be able to reach your server at the subdomain name.
Now, if you use the same IP as a webserver for example, but a different port, that is basically not gonna do what you want. The DNS will forward subdomain traffic to your (same) server IP, so if your webserver is on port 443, you will also reach it with https://api.domain.com. If your API uses port 8080 or 8081, you will need to always specify the port to actually reach the API server at the subdomain (i.e api.domain.com:8080 ).
The DNS merely forwards the subdomain name to the IP you tell it to.

I solved this using the tornado in Python. I already answered this question and it's working fine
from tornado.wsgi import WSGIContainer
from tornado.httpserver import HTTPServer
from tornado.ioloop import IOLoop
from flask import Flask
app = Flask(__name__)
#app.route('/')
def hello_world():
return 'Hello, World!'
http_server = HTTPServer(WSGIContainer(app))
http_server.listen(int(5000),address="ip_address_of_your_server_machine")
IOLoop.instance().start()
Now you can access this page like www.example.com:5000

Actually Flask is not meant to run a server by itself, it's only for debugging, if you want to run an web app you should run behind a Apache or Nginx with some wsgi, here is an simple example on Ubuntu 18.04 LTS:
https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-uswgi-and-nginx-on-ubuntu-18-04

Session authentication with Django channels

Trying to get authentication working with Django channels with a very simple websockets app that echoes back whatever the user sends over with a prefix "You said: ".
My processes:
web: gunicorn myproject.wsgi --log-file=- --pythonpath ./myproject
realtime: daphne myproject.asgi:channel_layer --port 9090 --bind 0.0.0.0 -v 2
reatime_worker: python manage.py runworker -v 2
I run all processes when testing locally with heroku local -e .env -p 8080, but you could also run them all separately.
Note I have WSGI on localhost:8080 and ASGI on localhost:9090.
Routing and consumers:
### routing.py ###
from . import consumers
channel_routing = {
'websocket.connect': consumers.ws_connect,
'websocket.receive': consumers.ws_receive,
'websocket.disconnect': consumers.ws_disconnect,
}
and
### consumers.py ###
import traceback
from django.http import HttpResponse
from channels.handler import AsgiHandler
from channels import Group
from channels.sessions import channel_session
from channels.auth import channel_session_user, channel_session_user_from_http
from myproject import CustomLogger
logger = CustomLogger(__name__)
#channel_session_user_from_http
def ws_connect(message):
logger.info("ws_connect: %s" % message.user.email)
message.reply_channel.send({"accept": True})
message.channel_session['prefix'] = "You said"
# message.channel_session['django_user'] = message.user # tried doing this but it doesn't work...
#channel_session_user_from_http
def ws_receive(message, http_user=True):
try:
logger.info("1) User: %s" % message.user)
logger.info("2) Channel session fields: %s" % message.channel_session.__dict__)
logger.info("3) Anything at 'django_user' key? => %s" % (
'django_user' in message.channel_session,))
user = User.objects.get(pk=message.channel_session['_auth_user_id'])
logger.info(None, "4) ws_receive: %s" % user.email)
prefix = message.channel_session['prefix']
message.reply_channel.send({
'text' : "%s: %s" % (prefix, message['text']),
})
except Exception:
logger.info("ERROR: %s" % traceback.format_exc())
#channel_session_user_from_http
def ws_disconnect(message):
logger.info("ws_disconnect: %s" % message.__dict__)
message.reply_channel.send({
'text' : "%s" % "Sad to see you go :(",
})
And then to test, I go into Javascript console on the same domain as my HTTP site, and type in:
> var socket = new WebSocket('ws://localhost:9090/')
> socket.onmessage = function(e) {console.log(e.data);}
> socket.send("Testing testing 123")
VM481:2 You said: Testing testing 123
And my local server log shows:
ws_connect: test#test.com
1) User: AnonymousUser
2) Channel session fields: {'_SessionBase__session_key': 'chnb79d91b43c6c9e1ca9a29856e00ab', 'modified': False, '_session_cache': {u'prefix': u'You said', u'_auth_user_hash': u'ca4cf77d8158689b2b6febf569244198b70d5531', u'_auth_user_backend': u'django.contrib.auth.backends.ModelBackend', u'_auth_user_id': u'1'}, 'accessed': True, 'model': <class 'django.contrib.sessions.models.Session'>, 'serializer': <class 'django.core.signing.JSONSerializer'>}
3) Anything at 'django_user' key? => False
4) ws_receive: test#test.com
Which, of course, makes no sense. Few questions:
Why would Django see message.user as an AnonymousUser but have the actual user id _auth_user_id=1 (this is my correct user ID) in the session?
I am running my local server (WSGI) on 8080 and daphne (ASGI) on 9090 (different ports). And I didn't include session_key=xxxx in my WebSocket connection - yet Django was able to read my browser's cookie for the correct user, test#test.com? According to Channels docs, this shouldn't be possible.
Under my setup, what is the best / simplest way to carry out authentication with Django channels?

Note: This answer is explicit to channels 1.x, channels 2.x uses a different auth mechanism.
I had a hard time with django channels too, i had to dig into the source code to better understand the docs ...
Question 1:
The docs mention this kind of long trail of decorators relying on each other (http_session, http_session_user ...) that you can use to wrap your message consumers, in the middle of that trail it states this:
Now, one thing to note is that you only get the detailed HTTP information during the connect message of a WebSocket connection (you can read more about that in the ASGI spec) - this means we’re not wasting bandwidth sending the same information over the wire needlessly.
This also means we’ll have to grab the user in the connection handler and then store it in the session;....
Its easy to get lost in all that, at least we both did ...
You just have to remember that this happens when you use channel_session_user_from_http:
It calls http_session_user
a. calls http_session which will parse the message and give us a message.http_session attribute.
b. Upon returning from the call, it initiates a message.user based on the information it got in message.http_session ( this will bite you later)
It calls channel_session which will initiate a dummy session in message.channel_session and ties it to the message reply channel.
Now it calls transfer_user which will move the http_session into the channel_session
This happens during the connection handling of a websocket, so on subsequent messages you won't have acces to detailed HTTP information, so what's happening after the connect is that you're calling channel_session_user_from_http again, which in this situation (post-connect messages) calls http_session_user which will attempt reading the Http information but fails resulting in setting message.http_session to None and overriding message.user to AnonymousUser.
That's why you need to use channel_session_user in this case.
Question 2:
Channels can use Django sessions either from cookies (if you’re running your websocket server on the same port as your main site, using something like Daphne), or from a session_key GET parameter, which works if you want to keep running your HTTP requests through a WSGI server and offload WebSockets to a second server process on another port.
Remember http_session, that decorator that gets us the message.http_session data? it appears that if it doesn't find a session_key GET parameter it fails to settings.SESSION_COOKIE_NAME, which is the regular sessionid cookie, so whether you provide session_key or not, you'll still get connected if you're logged in, of course that happens only when your ASGI and WSGI servers are on the same domain (127.0.0.1 in this case), the port difference doesn't matter.
I think the difference that the docs are trying to communicate but didn't expand on is that you need to setup session_key GET parameter when having your ASGI and WSGI servers on different domains since cookies are restricted by domain not port.
Due to that lack of explanation i had to test running ASGI and WSGI on same port and different port and the result was the same, i was still getting authenticated, changed one server domain to 127.0.0.2 instead of 127.0.0.1 and the authentication was gone, set the session_key get parameter and the authentication was back again.
Update: a rectification of the docs paragraph was just pushed to the channels repo, it was meant to mention domain instead of port like i mentioned.
Question 3:
my answer is the same as turbotux's but longer, you should use #channel_session_user_from_http on ws_connect and #channel_session_user on ws_receive and ws_disconnect, nothing from what you showed tells that it won't work if you do that change, maybe try removing http_user=True from your receive consumer? even thou i suspect it has no effect since its undocumented and intended only to be used by Generic Consumers...
Hope this helps!

To answer your first question you need to use the:
channel_session_user
decorator in the receive and disconnect calls.
channel_session_user_from_http
calls the transfer_user session during the connect method to transfer the http session to the channel session. This way all future calls may access the channel session to retrieve user information.
To your second question I believe what you are seeing is that default web socket library passes the browser cookies over the connection.
Third, I think your setup will be working quite well once have changed the decorators.

I ran into this problem and I found that it was due to a couple of issues that might be the cause. I'm not suggesting this will solve your issue, but might give you some insight. Keep in mind I am using rest framework. First I was overriding the User model. Second when I defined the application variable in my root routing.py I didn't use my own AuthMiddleware. I was using the docs suggested AuthMiddlewareStack. So, per the Channels docs, I defined my own custom authentication middleware, which takes my JWT value from the cookies, authenticates it and assigns it to the scope["user"] like so:
routing.py
from channels.routing import ProtocolTypeRouter, URLRouter
import app.routing
from .middleware import JsonTokenAuthMiddleware
application = ProtocolTypeRouter(
{
"websocket": JsonTokenAuthMiddleware(
(URLRouter(app.routing.websocket_urlpatterns))
)
}
middleware.py
from http import cookies
from django.contrib.auth.models import AnonymousUser
from django.db import close_old_connections
from rest_framework.authtoken.models import Token
from rest_framework_jwt.authentication import BaseJSONWebTokenAuthentication
class JsonWebTokenAuthenticationFromScope(BaseJSONWebTokenAuthentication):
def get_jwt_value(self, scope):
try:
cookie = next(x for x in scope["headers"] if x[0].decode("utf-8")
== "cookie")[1].decode("utf-8")
return cookies.SimpleCookie(cookie)["JWT"].value
except:
return None
class JsonTokenAuthMiddleware(BaseJSONWebTokenAuthentication):
def __init__(self, inner):
self.inner = inner
def __call__(self, scope):
try:
close_old_connections()
user, jwt_value =
JsonWebTokenAuthenticationFromScope().authenticate(scope)
scope["user"] = user
except:
scope["user"] = AnonymousUser()
return self.inner(scope)
Hope this helps this helps!

Websocket connection between socket.io client and tornado python server

I'm trying to get websockets to work between two machines. One pc and one raspberry pi to be exact.
On the PC I'm using socket.io as a client to connect to the server on the raspberry pi.
With the following code I iniated the connection and try to send predefined data.
var socket = io.connect(ip + ':8080');
socket.send('volumes', { data: data });
On the raspberry pi, the websocket server looks like this:
from tornado import web, ioloop
from sockjs.tornado import SockJSRouter, SockJSConnection
class EchoConnection(SockJSConnection):
def on_message(self, msg):
self.send(msg)
def check_origin(self, origin):
return True
if __name__ == '__main__':
EchoRouter = SockJSRouter(EchoConnection, '/echo')
app = web.Application(EchoRouter.urls)
app.listen(8080)
ioloop.IOLoop.instance().start()
But the connection is never established. And I don't know why. In the server log I get:
WARNING:tornado.access:404 GET /socket.io/1/?t=1412865634790
(192.168.0.16) 9.01ms
And in the Inspector on the pc there is this error message:
XMLHttpRequest cannot load http://192.168.0.10:8080/socket.io/1/?t=1412865634790. Origin sp://793b6d4588ead99e1780e35b71d24d1b285328f8.hue is not allowed by Access-Control-Allow-Origin.
I am out of ideas and don't know what to do. Can you help me?
Thank you!

Well, the solution for your problem has to do with the internal design of the sockjs-tornado library more than with the socket.io library.
Basically, your problem has to do with cross origin request i.e. the html that is generating the request to the websocket server is not at the same origin as the websocket server. I can see from your code that you already identified the problem ( and you tried to solve it by redefining the method "check_origin") but you didn´t find the proper way to do it, basically because within this library is not the SockJSConnection class the one that extends tornado WebSocketHandler and so redefining its "check_origin" is useless. If you dig a little bit into the code, you will see that there exists one class defined, namely SockJSWebSocketHandler that has a redefinition of such method itself, which relies on the tornado implementation if it returns true, but that also allows you to avoid that check using a setting parameter :
class SockJSWebSocketHandler(websocket.WebSocketHandler):
def check_origin(self, origin):
***
allow_origin = self.server.settings.get("websocket_allow_origin", "*")
if allow_origin == "*":
return True
So, to summarize, you just need to include the setting "websocket_allow_origin"="*" in the server settings and everything should work properly =D
if __name__ == '__main__':
EchoRouter = SockJSRouter(EchoConnection, '/echo', user_settings={"websocket_allow_origin":"*"})