Passing foreign characters via Socket.IO - python

I am doing a very simple thing, just sending a message to my Flask app via Socket.IO . It works like a charm with English, but some other languages break somewhere in the process.
Minimal working example follows.
testapp.py:
from flask import Flask
from flask_socketio import SocketIO
app = Flask('example')
socketio = SocketIO(app)
#socketio.on('test')
def on_connect(data):
print(data)
if __name__ == '__main__':
socketio.run(app)
index.html:
<!doctype html>
<html>
<body>
<script type="text/javascript" src="js/socket.io.js"></script>
<script>
var socket = io.connect('http://localhost:5000');
socket.on('connect', function() {
socket.emit('test', 'ASCII text');
socket.emit('test', 'Český text');
});
</script>
</body>
</html>
Instead of expected Český text, I am getting ÄŚeskĂ˝ text on the console.
I am using the newest versions of both the server packages (Flask-SocketIO 3.0.2, python-socketio 2.0.0, python-engineio 2.2.0) and the client (socket.io.js 2.1.1) and also checked that both of my files are UTF-8 encoded.
Some bug reports and questions mention a breaking change that happenned between 1.x and 2.x versions, so i tried using some older versions of the client (1.4.8, 1.7.4) instead of the newest one. The result was not much better: ÃÅeskÄË text.
This is beginners' stuff, so there must be a popular SO question covering it already. I probably just can't find it. So... what did I miss?

Related

"Flask-SocketIO is Running under Werkzeug" despite using socketio.run()

I'm having trouble running Flask & SocketIO with Eventlet despite using socketio.run(), any suggestions are appreciated. I'm currently on Python 3.9 and I've tried multiple different versions of each of these modules with no avail.
[2021-04-04 06:39:05,709] WARNING in __init__: Flask-SocketIO is Running under Werkzeug, WebSocket is not available.
"GET /socket.io/?EIO=4&transport=websocket HTTP/1.1" 400 -
index.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>SAR</title>
<script src="https://cdn.socket.io/3.1.3/socket.io.min.js" integrity="sha384-cPwlPLvBTa3sKAgddT6krw0cJat7egBga3DJepJyrLl4Q9/5WLra3rrnMcyTyOnh" crossorigin="anonymous"></script>
</head>
<body>
<button id="ping" onclick="send()">ping</button>
<script>
var socket = io.connect("http://" + document.domain + ":" + location.port, {transports: ['websocket']});
socket.on("connect", function(){
socket.emit("ping", "Established a connection, pinging!");
});
socket.on("pong", function(response){
console.log(response)
});
function send(){
socket.emit("ping", "ping_data");
}
</script>
</body>
</html>
app.py
from flask import Flask, render_template
from flask_socketio import SocketIO, emit
import eventlet
app = Flask(__name__)
socketio = SocketIO(app, logger=True)
#app.route( '/' )
def index():
return render_template( 'index.html')
def receivedCallback():
print('Pong received by user!')
#socketio.on( 'ping' )
def handle_ping(data):
print("received", data)
socketio.emit('pong', "pong_data", callback=receivedCallback)
if __name__ == '__main__':
socketio.run(app)
It seems like running my main file through the terminal resolves this issue. If anyone knows why this is please do share, thanks. :)
python app.py
To give a loose idea of why it runs with python app.py is that the if __name__ == '__main__' is executed only when it is run through the terminal, much like how the main function is called first in many other programming languages like Java or C when run from the terminal directly.
When a Flask app is run through Werkzeug, the flask app instance is imported and sort of wrapped into a module by Werkzeug in which the requests are routed into. (That is why the if __name__ == __main__ part is never executed – much like when you import another module in your code, the main function of that module is never called.) So, as far as my understanding goes, when you run your Flask app through Werkzeug, the requests are received and routed over HTTP by default and not over the WebSocket protocol that Flask-SocketIO uses when you run socketio.run(). The WebSocket protocol requires a socket to be always open, enabling asynchronous I/O which cannot work in HTTP as it is a client-server protocol.
I, however, do not have an answer to how to solve the problem and work with Flask-SocketIO through Werkzeug, but I hope the above explanation throws some light into your problem and drives you in the correct direction to look at.

Push message to all online users using Flask, WITHOUT creating threads for each users

Suppose ther is a chat group.
Is there a way to append new messages sent by any member of a particular group to the html page of all the online users in that group using FLASK.
WITHOUT: creating a thread for each users and monitoring for latest updates in the chats.
This is an example, with a very minimal code. If you are interested then you can explore more and do the experiments.
Firstly, you need the Flask-SocketIO module, run the below command to install it.
pip install flask-socketio
With this example, the project directory will be something like below:
ProjectDirectory
|
|-> templates
| |
| |-> index.html
|
|-> app.py
app.py
from flask import Flask, render_template
from flask_socketio import SocketIO, emit
app = Flask(__name__)
app.config['SECRET_KEY'] = 'randomSecretKEy#123'
socket = SocketIO(app)
#app.route('/')
def index():
return render_template('index.html')
#socket.on('message')
def message(data):
print(data)
emit('receive_message', data, broadcast=True)
if __name__ == '__main__':
socket.run(app)
index.html
<script src="//cdnjs.cloudflare.com/ajax/libs/socket.io/2.2.0/socket.io.js" integrity="sha256-yr4fRk/GU1ehYJPAs8P4JlTgu0Hdsp4ZKrx8bDEDC3I=" crossorigin="anonymous"></script>
<script src="//code.jquery.com/jquery-1.12.4.min.js" integrity="sha256-ZosEbRLbNQzLpnKIkEdrPv7lOy9C27hHQ+Xp8a4MxAQ=" crossorigin="anonymous"></script>
<script type="text/javascript" charset="utf-8">
var socket = io();
socket.on('connect', function() {
console.log('Connected to server');
});
socket.on('receive_message', function(msg) {
console.log('Received message: ', msg)
})
function sendMessage() {
msg = $('#message').val()
socket.emit('message', msg)
}
</script>
<input type="text" id="message"><button onclick="sendMessage()">Send</button>
Run the Flask app, open two browser tabs, with web developer tools opened side by side. AS soon as you open, you will see that Connected to server message has been logged. Now, type some message and hit the send button, in one tab, and you will see that, the message has been received in another tab's console.
I hope you get a direction with this and it is helpful for you.
Note: Again, I am telling this is a very basic example. Off course, you can add more logic and functionality, and make it more like a chatting app, but it will be out of the scope of this question. So, I will leave it to you and your imagination, to which extent you can learn and take this ahead. You can read about the module's documentation here.

How to use Ajax with CherryPy to run a python script to SSH into a Raspberry Pi Zero from a web interface

I'm trying to make a web interface to remotely control a GoPro. To do this so far, I've got a Raspberry Pi 3B which I'm planning to use as a web server which is connected to a Pi Zero via USB, which is connected via WiFi to the GoPro. So, whenever I SSH into the Pi Zero from the Pi 3B, I can control the GoPro. So now, I'm wanting to create a web server with CherryPy and Ajax so that I can create a web interface to control the GoPro when connected to the same WiFi network as the Pi 3B (web server).
I've been reading the CherryPy documentation to try and understand how to use Ajax with CherryPy. I was able to implement it before with a more basic CherryPy method (without Ajax), but it made it go onto a different webpage whereas for this, I want it to stay on the same webpage, which is why I'm using Ajax. I've never really used Ajax before so I'm quite unfamiliar and I basically tried my best to replicate the example on the CherryPy documentation but that didn't work, so forgive me if I'm missing something simple here.
Here's the HTML page code:
<html>
<head>
<script src="http://code.jquery.com/jquery-2.0.3.min.js"></script>
<script type="text/javascript">
$(document).ready(function(){
$("#takePhotos").click(function(e){
$.post("/take", {"count": $("input[name='count']").val()})
.done(alert("Running Take Photos!"));
})
e.preventDefault();
});
</script>
</head>
<body>
<input type="text" value="" name="count"/>
<button id="takePhotos">Take Photos</button>
</body>
And here's the python code for the web server:
import cherrypy
import os, os.path
import GoPro as gp
import requests
class RaspiServer(object):
#cherrypy.expose
def index(self):
return open('index.html')
#cherrypy.expose
class GPWebService(object):
#cherrypy.tools.accept(media="text/plain")
def GET(self,count):
print("running")
def POST(self,count):
print("running")
gp.connect("pi", "raspberrypizero.local", "raspberry")
gp.takePhotos(count)
return count
if __name__ == '__main__':
conf = {
'/': {
'tools.sessions.on': True,
'tools.staticdir.root': os.path.abspath(os.getcwd())
},
'/takePhotos':{
'request.dispatch': cherrypy.dispatch.MethodDispatcher(),
'tools.response_headers.on': True,
'tools.response_headers.headers': [('Content_Type', 'text/plain')]
},
'/static':{
'tools.staticdir.on': True,
'tools.staticdir.dir': "./public"
}
}
cherrypy.config.update({'server.socket_host': '0.0.0.0'})
webapp = RaspiServer()
webapp.takePhotos = GPWebService()
cherrypy.quickstart(webapp, '/', conf)
The problem I'm facing is that whenever I enter a value into the input field and click the button, the alert shows up saying 'Running Take Photos!' and in the console I can see that the web server has received the request, but the python scripts don't run in the 'POST' method.
What I want to happen is that the POST method executes and the two python methods 'connect' and 'takePhotos' execute (which is not happening right now).
Thanks very much in advance!
After playing about and learning more about Ajax all morning, I finally figured out the answer and it's as stupid and easy as I thought.
On line 7 of the index.html file, I had to change $.post("/take") to $.post("/takePhotos")

Server sent event data not displaying in browser

I am new to Python and networking. I am trying some Python code at server side to display some continous data on my client browser. I have taken this sample code from net itself. The problem is that this code works fine on local machine when i try to access it my browser using localhost but doesn't works when i try to access it from other machine. In Ubuntu i doesn't get any data on my browser but in windows i get all data pushed on my browser when i stop the server code. I am using Twisted and Flask in Python for this. Here is the code i am trying:
import random
from twisted.web.server import Site
from twisted.web.wsgi import WSGIResource
from twisted.internet import reactor
import time
from flask import Flask, request, Response
app = Flask(__name__)
def event_stream():
count = 0
while True:
count += 1
yield 'data: %d\n\n' % count
time.sleep(5)
print 'data sent...'
#app.route('/my_event_source')
def sse_request():
return Response(
event_stream(),
mimetype='text/event-stream')
#app.route('/')
def page():
return '''
<!DOCTYPE html>
<html>
<head>
<script type="text/javascript" src="//code.jquery.com/jquery-1.8.0.min.js"></script>
<script type="text/javascript" src="EventSource.js"></script>
<script type="text/javascript">
$(document).ready(
function() {
sse = new EventSource('/my_event_source');
sse.onmessage = function(message) {
console.log('A message has arrived!');
$('#output').append('<li>'+message.data+'</li>');
}
})
</script>
</head>
<body>
<h2>Hello World Example to implement Server Sent Event using Twisted in Python...</h2>
<ul id="output"></ul>
</body>
</html>
'''
if __name__ == '__main__':
print 'starting server...'
resource = WSGIResource(reactor, reactor.getThreadPool(), app)
site = Site(resource)
reactor.listenTCP(9999, site)
reactor.run()
Can someone help me why the data is not getting displayed when i am trying to access it from other machine (or how can i debug it). Any help is greatly appreciated.

minimal example of Python's bottle microframework using gevent-socketio and Socket.IO.js

Question: What would be a comparable solution to the example at this link, except implemented using gevent-socketio and Socket.io.js with bottle? I'm looking for the minimal solution that will simply pass some traffic in a loop from the client to the server and back to the client using gevent-socketio, Socket.io.js, and bottle.
Background: I have developed a simple web-app that provides a web-based terminal for a remote custom shell (cli) on the server. The browser (client) collects shell commands from a form input field, passes the command over a web-socket to a gevent.pywsgi.WSGIServer handling the requests via the geventwebsocket.WebSocketHandler handler, which supplies the command to the shell, while asynchronously returning output via the socket to a textarea field in a form in the client's browser. This is based on a great, little example provided by the bottle team:
http://bottlepy.org/docs/dev/async.html#finally-websockets
Provided here for redundancy:
example_server.py:
from bottle import request, Bottle, abort
app = Bottle()
#app.route('/websocket')
def handle_websocket():
wsock = request.environ.get('wsgi.websocket')
if not wsock:
abort(400, 'Expected WebSocket request.')
while True:
try:
message = wsock.receive()
wsock.send("Your message was: %r" % message)
except WebSocketError:
break
from gevent.pywsgi import WSGIServer
from geventwebsocket import WebSocketHandler, WebSocketError
server = WSGIServer(("0.0.0.0", 8080), app,
handler_class=WebSocketHandler)
server.serve_forever()
client.html:
<!DOCTYPE html>
<html>
<head>
<script type="text/javascript">
var ws = new WebSocket("ws://example.com:8080/websocket");
ws.onopen = function() {
ws.send("Hello, world");
};
ws.onmessage = function (evt) {
alert(evt.data);
};
</script>
</head>
</html>
Motivation: My existing app works great in the latest version of Firefox and Chrome. IE support is non-existent, and Safari compatibility is middlin'. I'm ultimately looking for a cross-browswer solution to communicate shell commands and output between the client and server. If I had a simple example for bottle, I think I could move forward more quickly.
Incidentally, I looked at the gevent-socketio examples and even a bottle example, but all of these examples are too different from the above simple example for me to make the leap in application. (The gevent-socketio examples look nothing like the bottle apps, which which I'm familiar. And, the bottle example doesn't actually show how to communicate with the client.)
Thanks! :)
Circus! the process runner and watcher built on top of zmq, use bottle and socketio for the web interfaces:
https://github.com/mozilla-services/circus/blob/master/circus/web/circushttpd.py
https://github.com/mozilla-services/circus/blob/master/circus/web/server.py
The source code is simple enough for helping you to get started to build a bigger app with bottle and socketio.
Otherwise, I advice you to move to sockjs! which a more generic implementation with better support for different backends.
This other thread can help you :
SockJS or Socket.IO? Worth to recode ajax-based page?

Categories