We Keep Coding

Redis and channels with Windows

I'm trying to get a var to my consumers.py to send data to the client in real time as a function does API calls and returns that to the browser.
I know channels needs Redis to function, but why? Why can we not just pass a list as it's built to the consumers class or any variable for that matter? From another answer: to store the necessary information required for different instances of consumers to communicate with one another. But what if I only will use one websocket connection, and only one user is allowed to be logged in at a time? This will be locally hosted only and the function is outside of consumers.py that returns the data so subscribing to groups may be where I need these.
Am I missing something or is redis / memurai a must here? I just can't help but to feel there's an easier way.

I ended up using server side events (SSE), specifically django-eventstream, and so far it's worked great as I didn't need the client to interact with the server, for a chat application this would not work.
Eventstream creates an endpoint at /events/ the client can connect to and receive a streaming http response.
Sending data from externalFunc.py:
send_event('test', 'message', {'text': 'Hello World'})
Event listener in HTML page:
var es = new ReconnectingEventSource('/events/');
es.addEventListener('message', function (e) {
console.log(e.data);
var source = new EventSource("/events/")
var para = document.createElement("P");
const obj = JSON.parse(event.data)
para.innerText = obj.text;
document.body.appendChild(para)
}, false);
es.addEventListener('stream-reset', function (e) {
}, false);

Write Admin Messages in Bearle / Django-Private-Chat (Django 2/Python)

I was wondering, if it is possible to connect (as an admin) with a user and to write messages? I already modified the message-table in the database, however, the message does only appear after reloading the whole website - which is not the optimal outcome. I appreciate every hint. Thank you.

You have to use a pub/sub service to get notification on the client side when a message is sent in server side
A good start is pusher.com service, in your server side code, you will run this example code whenever you send a message to your user:
pusher.trigger('my-channel', 'my-event', {
'message': 'hello world'
})
and in your client side, you add a code that listen to the above channel/topic and act accordingly
var channel = pusher.subscribe('my-channel');
channel.bind('my-event', function(data) {
alert('Received my-event with message: ' + data.message);
});

How to build a system to handle MQTT broker and Django

I am planning to build a home automation system where IoT devices communicate with the MQTT broker.The system also involves a Django web server that serves API for ios/android devices. I will describe an example of what I want to implement.
An API call is made from mobile app to Django server to turn ON a device. When such API request is made to Django, it should push 'Turn ON' data to IoT device via MQTT protocol.
Also, IoT devices send some real time data to the MQTT broker. Upon receiving such data I want to send push notifications to a mobile app via apns/fcm.
How can I implement this?. Will Django channels serve the purpose or should I code my Django server to act as an MQTT client and communicate with the MQTT broker?. Or is there any other methods to implement this.

well, i did a little project on paho-MQTT it's a nice experience with google chrome extension MQTTLens.(you should try this if u aren't using this already)
in your case, I think you can use rest-framework of Django for building an API and on the front-end, you can use crispy-form to make ON-OFF signals and this will directly communicate to the views of Django in which you can write the client and subscriber details.
lets focus on An API call is made from mobile app to Django server to turn ON a device. When such API request is made to Django, it should push 'Turn ON' data to IoT device via MQTT protocol.
you can make views which response to the API call from any devices for that you can check django-rest-framework this is the best option that we have.
and now IoT devices send some real time data to the MQTT broker
for this, you can check Google's this artical. MQTT broker can be handed with the Django views easily and this process not very complex if you use the modular structure with Django's DRY concpet.
on the other hand, you can also make different views for just client or for broker it's up to you but i think this approach will take a long time to devlope such application I don't know about mobile development so i can't help you with that :(.

You can handle the task using JavaScript. I have experience in implementing MQTT protocol in Django and Django-REST projects using JavaScript. You should embed a JavaScript code block in your frontend file (in my case, HTML). First, you should call the Paho-MQTT jQuery package in your file.
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/paho-mqtt/1.0.2/mqttws31.min.js"></script>
Then add this block of code.
#parameters
var hostname = "mqtt.eclipse.org"; #There are different brokers. You should enter the broker's hostname.
var port = 80; #The port number can be different based on a TLS or non-TLS connection.
var ClientID = "ClientID_" + parseInt(Math.random()*100);
#Create a client instance
var client = new Paho.MQTT.Client(hostname, Number(port), ClientID);
#Set callback handlers
client.onConnectionLost = onConnectionLost;
client.onMessageArrived = onMessageArrived;
#Connect the client
client.connect(
{onSuccess: onConnect}
);
#Called when client connects
function onConnect() {
#Once a connection has been established, make a subscription and send a message
console.log("onConnect");
client.subscribe("subTopic");
alert("Connected.");
}
#Called when the client loses its connection
function onConnectionLost(responseObject){
if(responseObject.errorCode != 0){
console.log("onConnectionLost:" + responseObject.errorMessage);
}
}
#Called when a message arrives
function onMessageArrived(message) {
console.log("Message arrived: topic=" + message.destinationName + ", message=" + message.payloadString);
if (message.destinationName == "subTopic") {
#Do something
}
By using the code your application will be connected to the broker and listen to one or multiple topics. It means you can get sensors' data realtime. This necessitates to publish sensor's data on your hardware device, say ESP module or Raspberry PI.
It is very likely that you want to send commands from your application to the actuators to turn them on or off. For doing that you need to publish some messages from your application that your hardware will listen to.
Assume you have a toggle switch that you want to publish a message by toggling that.
<label id="switch{{ device.unique_id }}" class="switch">
<input id="state{{ device.unique_id }}" type="checkbox" onclick="publish('{{ device.unique_id }}')">
<span class="slider round"></span>
</label>
The above HTML block should reside into a django for block. Then you should write the publish onclick function to call that on toggling the switch. You can see an example of such a function below.
function publish(x) {
if(!client){
return;
}
var status = document.getElementById(x);
if (status.innerHTML == 'ON'){
status.innerHTML = 'OFF';
var message = new Paho.MQTT.Message("TurnOFF");
message.destinationName = "pubTopic";
client.send(message);
} else {
status.innerHTML = 'ON';
var message = new Paho.MQTT.Message("TurnON");
message.destinationName = "pubTopic";
client.send(message);
}
}
x in the publish function is the id that is embedded in the HTML file. To receive your published messages you should listen to the specific topic(s) on your hardware device.

What is the best way to send messages to Socket.IO clients from various back-ends

My Setup:
I have an existing python script that is using Tweepy to access the Twitter Streaming API. I also have a website that shows aggregate real-time information from other sources from various back-ends.
My Ideal Scenario:
I want to publish real-time tweets as well as real-time updates of my other information to my connected users using Socket.IO.
It would be really nice if I could do something as simple as an HTTP POST (from any back-end) to broadcast information to all the connected clients.
My Problem:
The Socket.IO client implementation is super straight forward... i can handle that. But I can't figure out if the functionality I'm asking for already exists... and if not, what would be the best way to make it happen?
[UPDATE]
My Solution: I created a project called Pega.IO that does what I was looking for. Basically, it lets you use Socket.IO (0.8+) as usual, but you can use HTTP POST to send messages to connected users.
It uses the Express web server with a Redis back-end. Theoretically this should be pretty simple to scale -- I will continue contributing to this project going forward.
Pega.IO - github
To install on Ubuntu, you just run this command:
curl http://cloud.github.com/downloads/Gootch/pega.io/install.sh | sh
This will create a Pega.IO server that is listening on port 8888.
Once you are up and running, just:
HTTP POST http://your-server:8888/send
with data that looks like this:
channel=whatever&secretkey=mysecret&message=hello+everyone
That's all there is to it. HTTP POST from any back-end to your Pega.IO server.

The best way I've found for this sort of thing is using a message broker. Personally, I've used RabbitMQ for this, which seems to meet the requirements mentioned in your comment on the other answer (socket.io 0.7 and scalable). If you use RabbitMQ, I'd recommend the amqp module for node, available through npm, and the Pika module for Python.
An example connector for Python using pika. This example accepts a single json-serialized argument:
def amqp_transmit(message):
connection = pika.AsyncoreConnection(pika.ConnectionParameters(host=settings.AMQP_SETTINGS['host'],
port=settings.AMQP_SETTINGS['port'],
credentials=pika.PlainCredentials(settings.AMQP_SETTINGS['username'],
settings.AMQP_SETTINGS['pass'])))
channel = connection.channel()
channel.exchange_declare(exchange=exchange_name, type='fanout')
channel.queue_declare(queue=NODE_CHANNEL, auto_delete=True, durable=False, exclusive=False)
channel.basic_publish(exchange=exchange_name,
routing_key='',
body=message,
properties=pika.BasicProperties(
content_type='application/json'),
)
print ' [%s] Sent %r' %(exchange_name, message)
connection.close()
Very basic connection code on the node end might look like this:
var connection = amqp.createConnection(
{host: amqpHost,
port: amqpPort,
password: amqpPass});
function setupAmqpListeners() {
connection.addListener('ready', amqpReady)
connection.addListener('close', function() {
console.log('Uh oh! AMQP connection failed!');
});
connection.addListener('error', function(e) {throw e});
}
function amqpReady(){
console.log('Amqp Connection Ready');
var q, exc;
q = connection.queue(queueName,
{autoDelete: true, durable: false, exclusive: false},
function(){
console.log('Amqp Connection Established.');
console.log('Attempting to get an exchange named: '+exchangeName);
exc = connection.exchange(exchangeName,
{type: 'fanout', autoDelete: false},
function(exchange) {
console.log('Amqp Exchange Found. ['+exchange.name+']');
q.bind(exc, '#');
console.log('Amqp now totally ready.');
q.subscribe(routeAmqp);
}
);
}
);
}
routeAmqp = function(msg) {
console.log(msg);
doStuff(msg);
}
Edit: The example above uses a fan-out exchange that does not persist messages. Fan-out exchange is likely going to be your best option since scalability is a concern (ie: you are running more than one box running Node that clients can be connected to).

Why not write your Node app so that there are two parts:
The Socket.IO portion, which communicates directly with the clients, and
An HTTP API of some sort, which receives POST requests and then broadcasts appropriate messages with Socket.IO.
In this way, your application becomes a "bridge" between your non-Node apps and your users' browsers. The key here is to use Socket.IO for what it was made for--real time communication to browsers--and rely on other Node technologies for other parts of your application.
[Update]
I'm not on a development environment at the moment, so I can't get you a working example, but some pseudocode would look something like this:
http = require('http');
io = require('socket.io');
server = http.createServer(function(request, response) {
// Parse the HTTP request to get the data you want
io.sockets.emit("data", whatever); // broadcast the data to Socket.IO clients
});
server.listen(8080);
socket_server = io.listen(server);
With this, you'd have a web server on port 8080 that you can use to listen to web requests (you could use a framework such as Express or one of many others to parse the body of the POST request and extract the data you need).

Web sockets / Tornado - Notify client on database update

I'm trying to use a Tornado web socket server to notify my user when changes are made to a database in realtime. I was hoping to use HTML5 web sockets for this, even though most browsers don't support them. None of the demos that come with the Tornado package use web sockets and they are not mentioned in the documentation, so I have no idea how to get started. The few examples I could find on google either don't work or are poorly documented.
Does anyone have any examples of how I can use Tornado to push data to a client when a MySQL database has been updated or something similar I can use to learn from?

A Lee's answer is a good one, you probably want socket.io if you need to support older browsers.
Websockets are very easy in tornado though:
import tornado.websocket
class EchoWebSocket(tornado.websocket.WebSocketHandler):
def open(self):
print "WebSocket opened"
def on_message(self, message):
self.write_message(u"You said: " + message)
def on_close(self):
print "WebSocket closed"
Then route it as any other handler, and include Websocket Javascript in your views:
var ws = new WebSocket("ws://localhost:8888/websocket");
ws.onopen = function() {
ws.send("Hello, world");
};
ws.onmessage = function (evt) {
alert(evt.data);
};
For more info, see the source: https://github.com/facebook/tornado/blob/master/tornado/websocket.py

I've had success using the socket.io client and tornadio on the server end. Socket.IO provides an abstraction over websockets and provides fallbacks if websockets aren't supported by the browser (long polling, flash socket, etc.).
In order to use it you just need to write a tornadio script a la the tornadio documentation that monitors your database and then include the socket.io JavaScript in your web pages and have it establish a connection to wherever your tornadio server resides at the URL route you specified in your tornadio script.

This post using websockets and redis covers the basic idea pretty well.