So I have written a websocket application in Twisted. The application is a basic game between a number of users, but trying to use the web socket for setup and record saving is painful, so I was looking into using Django based rendering for the supplementary information (as in standings, game setup, lobby list, etc) and leave the websockets for the real action. I know I can use some basic IPC functionality to have the Django requests signal the Twisted application, but I was curious if the Django signal system would also work across applications as a simple form of IPC...
No. Django signals are restricted to a single Python interpreter. You'll need to put together something else (sockets, JSON-RPC, XMPP, etc.) in order to perform IPC.
Related
I'm struggling with some architectural choices for a scalable internet-of-things application.
I've chosen to base my project on Twisted augmented with the Cyclone framework to provide many Tornado convenances (websockets, auth-decorators, secure-cookies, etc)
Using a Twisted core has worked beautifully for me. I have numerous IP protocol and hardware interfaces all of which turned out to have great library support inside of twisted (and adding new protocols and interfaces to my application are the most-likely angles I'll have project scope creep), all with Twisted needing very low CPU and providing for very high connection counts.
My problems are with second-order webapp functionality.
I pulled in Cyclone thinking that with it's auth goodies (OpenID, oauth, user-auth decorators and secure-cookies) it wouldn't take much to implement user/session/admin functionality in my webapp. After the 500+ lines of abstracting my database (via txmongo) and just building user logins it became clear I both:
Didn't understand how little Cyclone/Tornado bring in the user/session/admin space, and
Didn't understand the amount of code it takes to fill in the gaps if your trying to build a multi-user auth webapp
A friend pointed me at Flask, which initially I thought was completely redundant, until I found flask plugins. The combination of Flask-Login and Flask-Admin would completely cover my user, session and user-admin needs, negating me writing what I would guess to be about 2k lines of code. Unfortunately, the flask plugins are all rife with blocking code and calls to blocking libraries. I don't see them as compatible with my project even if WSGI containers are used given that the user/session functionality happens with every page load (additionally I don't see any short cuts that would allow me to port them to async world without work roughly equal to that of rewriting them)
My question is:
In the python async space (... hopefully in the Twisted space, given my protocol needs), are there any plugins or alternate frameworks that provide ready-to-go user/login/admin functionality similar to what is in Flask-Login and Flask-Admin?
P.S. I've looked at Klein as the obvious Twisted version of Flask, but it doesn't seem to have a plugin ecosystem, and I'm not finding any strong user/session/admin there.
P.P.S. By the time I wrote this question I had already written my own (crappy) user-login-session system. So what I'm really after is the "Admin" capability (automated CRUD functions on user-style records, including web UI rendering, all designed in a Twisted/async way). I asked about user/login in the question in case it turn out there is an already-integraded solution (such as flask-login and flask-admin) in which case I would happily drop my code and switch to that.
Do you really need everything async? Consider async WebSockets but sync page renders. If you must, add an async downstream proxy or load balancer which will virtually eliminate app server's IO overhead.
My issue comes as follows: I want to mount a game using only one port (80), and django.
For my game (it's not a MMORTS game, so I must have real-time interaction) I need to use Websockets. This means, I need to use Django (backend, profiles) + Websockets (The game itself). For the sake of this issue, think about this game as if it was like Habbo Hotel or Sims (I need that kind of interaction).
Question: What non-dead packages could I use for this?
If the recommended websockets-package is not integrable to Django but mountable as a separate server, I'd need to solve 2 additional issues:
Is there a way to bridge non-ws connections to the regular Django application handler?
Is there a way to integrate whatever I implement as the websocket handling (i.e. game logic) to the Django engine? (i.e. models).
Assume I'd use this in a shared hosting supporting Python 2.7/Django 1.6.
What are the implications of using webservices or message queues in an application?
In my case, I need to connect a Django web application with a python application, and I need two way communication between these applications. Sometimes the web app sends request to the python app to activate a few hardware devices, and sometimes the python app needs to be generally queried to obtain data.
The issue my entire application depends on instantaneous data received or sent to the python core application, so I cannot afford to waste system resources on querying everytime. I need to use something like a listener/receiver to send/receive data, without manually triggering a query every few seconds.
I am using Django for web application framework and Python for my core application.
I already have ZMQ being used internally for multiagent communication platform. If it is message queue, all I need to do is just connect to it, and send and receive data.
If it is a webservice, I need to freshly integrate the webservices. Again, what is the preferred method to create a webservice using Python?
What are some realtime "push" options for django that can install as a python package? I want to avoid having to do things like installing independent web-servers for realtime.
Essentially I am looking for something like pusher.com (cloud system) or this socket.io build for django (which has a build status:failing) for chat and other various push operations.
Ape was suggested here, but it seems it requires you to setup Ape as a server. If its not too much to ask for, are there any solutions that build right into django?
Since the time the answer was written (2012); a lot has changed.
The preferred method now to do realtime updates of the system is using websockets; which is being formalized and proposed as a standard RFC 6455. This page on MDN has a great overview of the technology.
The other emerging technology is Server Sent Events which is a W3C draft proposal.
Projects such as swampdragon and django-socketio make integrating realtime functionality easier in your project.
There are two main components to any realtime system:
A connection that remains open from the browser to a server.
A server that listens on this connection and then responds.
A system / standard to store and be notified of messages.
Okay so maybe three components.
Since django doesn't work in realtime any solution that offers realtime push/updates will require another server/service to accept messages and then notify listeners of pending messages.
Django would be the application that pushes messages (writes them) to this server on a channel (a queue/bucket). Listeners then subscribe to a channel to be notified of messages. Since the connection remains open; messages are retrieved in "realtime".
Django really has a minimal role in all of this. There are various implementations that provide the three components necessary for realtime notifications to work.
I really like juggernaut because it is super simple to set up, and uses node.js that doesn't require a lot in terms of server-side components. The other reason I prefer it is because it supports Adobe Flash Socket in addion to WebSocket (and others, see the link).
The api to access it also very simple - in fact, if you are already using redis (which you really should since its so easy to use), you don't need another API as you can drop messages to redis and juggernaut will read them, or you can use its Python API. A simple example from this flask snippet:
Send (write) a message to a channel:
>>> from juggernaut import Juggernaut
>>> jug = Juggernaut()
>>> jug.publish('channel', 'The message')
Listen to it:
<script type=text/javascript
src=http://localhost:8080/application.js></script>
<script type=text/javascript>
var jug = new Juggernaut();
jug.subscribe('channel', function(data) {
alert('Got message: ' + data);
});
</script>
Django is built to serve web pages and there is nothing out of the box to support websockets in django. The quickest/easiest option is pusher.com (I use it an really like it). You can start with something like pusher.com and if you write a quick wrapper around it, you can replace it with your own server using socket.io or any other web socket server by just changing the wrapper / interface to connect to the new server. Just make sure you write it with being able to switch out the backend at any time.
If you really want to start run your own socket server there are projects out there that will make it easy to use sockets in django:
django-websocket
django-socketio
You can actually serve up Django from Tornadio2, a working implementation of socketio in Tornado. If you want to build any degree of sophistication into your realtime app you will likely need a redis pubsub backend that maps sessions to channels and handles multicasting. For this you might like to take a look at Brukva. Also read up Yuval Adam's blog post on this subject. Finally, Tony Abou Assaleh's sample package and post will provide a useful base reference when setting up tornadio2 for django.
I have a Python-driven web interface powered by Apache 2.2 with mod_python and Python 2.4. I need to make an asynchronous process appear synchronous to users of this web interface.
When users access one module on this website:
An external SOAP interface will be contacted with a unique identifier and will respond with a number N
The external interface will respond asynchronously by contacting a SOAP server on my machine between 1 and 10 times (the number N tells us how many responses we will receive)
I need to somehow aggregate these responses and pass them to the original module which will display the information back to the user. The goal is to make the process appear synchronous to the user.
What is the best way to handle this synchronization issue? Is this something Twisted would be well-suited for?
I am not restricting myself to Python for the solution, though it is preferred because everything else on the server is in Python. I prefer a solution that is both scalable and will take a minimal amount of programming time (though I understand that these attributes are somewhat at odds).
Maybe you can use Orbited to get ajax push with long-lived HTTP connections to your web clients. Orbited is based on Twisted, so I think it makes sense to look at if you already know Twisted. Have a look at this tutorial to get started.