I'm trying to use a javascript plotting library to plot data I calculated in the python console (IPython) using Numpy.
I was thinking of a design to accomplish this. I thought it should look something like this:
This means I'm using a Flask app to sent a server-side event to the client as soon as the data is ready in the IPython process.
1) Is this the right approach?
2) How can I send data from the console to my Flask app? I think I need to run the Flask App with gevent so I can listen to commands from the other python process?
3) How can I send the data using a server-side event to the client? I found some examples of server-side events but I don't see how they can sent arbitrary data. Example:
import gevent
import gevent.monkey
from gevent.pywsgi import WSGIServer
gevent.monkey.patch_all()
from flask import Flask, request, Response, render_template
app = Flask(__name__)
def event_stream():
count = 0
while True:
gevent.sleep(2)
yield 'data: %s\n\n' % count
count += 1
#app.route('/my_event_source')
def sse_request():
return Response(
event_stream(),
mimetype='text/event-stream')
#app.route('/')
def page():
return render_template('sse.html')
if __name__ == '__main__':
http_server = WSGIServer(('127.0.0.1', 8001), app)
http_server.serve_forever()
This example uses a predefined event_stream() function. I am looking for a way to send the data from the python console with a server-side event to the client.
Maybe there is another and better approach?
Is it $ipython console(zmq console) of the plain IPython terminal ($ipython) ?
Why not directly use the IPython Notebook ?
(Notebook, QT and zmq console can allow a 3rd program to listen for event published by the kernel).
Have a look at IPEP 21 and current implementation proposal which is bidirectional kernel<-> js communication.
Did you had a look at bokeh from continuum that seem to do almost what you want ?
(also IPython not iPython in the images, but this is a detail)
Related
I'm building a Flask web app using the flask-socketio module to implement websockets. It generally works fine, but when I try to emit multiple messages from the server to the client in a for loop, all the messages are actually sent at once - that is, as soon as all of them have been created.
I read that the solution might be to use an eventlet server capable of asynchronous task handling:
"The simplest deployment strategy is to have eventlet or gevent installed, and start the web server by calling socketio.run(app) as shown in examples above. This will run the application on the eventlet or gevent web servers, whichever is installed." (taken from the Flask-SocketIO docs)
Sadly, that doesn't solve the problem. I've never worked with websockets before, so I'm a bit lost. Here is a simplified version of my code:
from flask import Flask
from flask_socketio import SocketIO, emit
import eventlet
app = Flask(__name__)
socketio = SocketIO(app)
#socketio.on("connect")
def handle_connect():
print("server and client connected")
#socketio.on("text")
def text(question):
for _ in range(3):
answer = my_module.generate_answer(question)
emit("message", {"msg": answer})
socketio.run(app)
Just assume that my_module.generate_answer() generates a sentence based on some user input sent via the websocket. Each generation takes 5-10 seconds. That's also the reason why I want the answers to be sent via a WebSocket once they're generated - my frontend could already display the first answer while waiting for the next ones.
Thank you so much for your help!
You need to monkey patch at the very top of your file.
Your import statements should look like this:
import eventlet
eventlet.monkey_patch()
from flask import Flask
from flask_socketio import SocketIO, emit
I have a Python script that pulls data from a 3 rd party API. Currently this Pyhton script is automated on server side.
There are few instances where I have to toggle the script manually for new data updates. For the manual toggle I have to login to the server each time and run it from command line. Is there a way where I can create web url or something similar and just run that URL to make that script run from the browser address bar.
One approach you could take is to use Flask, which is a minimal web framework. Here's an example of how you could use it:
from flask import Flask
from your_script import your_func
app = Flask(__name__)
#app.route('/run')
def run_command():
your_func()
return 'Executed your function!'
if __name__ == '__main__':
app.run(debug=False, port=8080)
If you run this code you'd get a web server running on port 8080 that executes your function when you access the url. Here's a tutorial in the Flask documentation to get you started.
I think the easiest way to do this is by using Flask.
from flask import Flask
app = Flask(__name__)
#app.route('/')
def hello_world():
# your code here
return 'Hello, World!'
I want to build a Webapp with Flask where some data is printed on a dynamic page in real time.
The data is taken from a Python script which connects to a Websocket, then it's printed on the frontend with Flask.
I have two problems:
1) I can't run both the scripts together
2) I don't know how to call parsed from test to yield
Here is the code:
from time import sleep
from flask import Flask, render_template
import websocket
from bitmex_websocket import Instrument
from bitmex_websocket.constants import InstrumentChannels
from bitmex_websocket.constants import Channels
import json
from threading import Thread, Event
app = Flask(__name__)
websocket.enableTrace(True)
channels = [
InstrumentChannels.trade,
]
XBTUSD = Instrument(symbol='XBTUSD',
channels=channels)
XBTUSD.on('action', lambda msg: test(msg))
def test(msg):
parsed = json.loads(json.dumps(msg))
print(parsed)
#app.route('/')
def index():
# render the template (below) that will use JavaScript to read the stream
return render_template('index.html')
#app.route('/stream_sqrt')
def stream():
def generate():
yield '{}\n'.format('test')
return app.response_class(generate(), mimetype='text/plain')
if __name__ == '__main__':
XBTUSD.run_forever()
app.run()
If i put XBTUSD.run_forever() before app.run() i will start the part supposed to retrieve the data but the Flask app won't start. If i do the opposite, the Flask app will run but not the other part. How can i run together the whole app? How could i "share" variables between test and generate?
An easier way to go, please use flask-socketio instead flask.
https://flask-socketio.readthedocs.io/en/latest/
Sample for sending messages using flask-socketio
https://flask-socketio.readthedocs.io/en/latest/#sending-messages
Just to give a context here, I'm a node.JS developer, but I'm on a project that I need to work with Python using Flask framework.
The problem is, when a client request to an endpoint of my rest flask app, I need to emit an event using socket.IO, and get some data from the socket server, then this data is the response of the endpoint. But I didn't figured out how to send this, because flask needs a "return" statement saying what is the response, and my callback is in another context.
Sample of what I'm trying to do: (There's some comments explaining)
import socketio
import eventlet
from flask import Flask, request
sio = socketio.Server()
app = Flask(__name__)
#app.route('/test/<param>')
def get(param):
def ack(data):
print (data) #Should be the response
sio.emit('event', param, callback=ack) # Socket server call my ack function
#Without a return statement, the endpoint return 500
if __name__ == '__main__':
app = socketio.Middleware(sio, app)
eventlet.wsgi.server(eventlet.listen(('', 8000)), app)
Maybe, the right question here is: Is this possible?
I'm going to give you one way to implement what you want specifically, but I believe you have an important design flaw in this, as I explain in a comment above. In the way you have this coded, your socketio.Server() object will broadcast to all your clients, so will not be able to get a callback. If you want to emit to one client (hopefully not the same one that sent the HTTP request), then you need to add a room=client_sid argument to the emit. Or, if you are contacting a Socket.IO server, then you need to use a Socket.IO client here, not a server.
In any case, to block your HTTP route until the callback function is invoked, you can use an Event object. Something like this:
from threading import Event
from flask import jsonify
#app.route('/test/<param>')
def get(param):
ev = threading.Event()
result = None
def ack(data):
nonlocal result
nonlocal ev
result = {'data': data}
ev.set() # unblock HTTP route
sio.emit('event', param, room=some_client_sid, callback=ack)
ev.wait() # blocks until ev.set() is called
return jsonify(result)
I had a similar problem using FastAPI + socketIO (async version) and I was stuck at the exact same point. No eventlet so could not try out the monkey patching option.
After a lot of head bangings it turns out that, for some reason, adding asyncio.sleep(.1) just before ev.wait() made everything work smoothly. Without that, emitted event actually never reach the other side (socketio client, in my scenario)
I have a python server that is currently keeping track of the location of all the buses in my university and generating predictions of arrivals to specific locations.
Now, I wanted to attach a lightweight REST API to this server but I have been running intro problems.
I tried using flask with the following code:
from flask import Flask, jsonify
from PredictionWrapper import *
import threading
class RequestHandler():
def __init__(self,predictionWrapper):
self.app = Flask(__name__)
self.predictor = predictionWrapper
self.app.debug = False
self.app.add_url_rule('/<route>/<int:busStop>','getSinglePrediction',self.getSinglePrediction)
t = threading.Thread(target=self.app.run, kwargs={'host':'0.0.0.0', 'port':80, 'threaded':True})
t.start()
def getSinglePrediction(self, route, busStop):
# TODO Get the actual prediction with given parameters
prediction = self.predictor.getPredictionForStop(route, busStop)
return jsonify({'busStop': busStop, 'prediction': prediction})
def getStopPrediction(self, busStop):
# TODO Get the actual prediction with given parameters
return jsonify({'busStop': busStop, 'prediction': 2})
def run(self):
self.app.run(host='0.0.0.0', port=80, threaded=True)
The problem is that I have been encountering the error below after about half a day of running the server. Note that no requests were made to the server around the time it failed with the following error:
ERROR:werkzeug: - - [01/May/2016 09:55:55] code 400, message Bad request syntax ('\x02\xfd\xb1\xc5!')
After investigating I believe I need to deploy to a WSGI production server. But I have no clue what it means in this specific approach given that 1)the flask server is being threaded in order to run the rest of the prediction application, and 2)I am using classes which none of the documentation uses.
Any help on how to setup the wsgi file with apache, gunicorn, or the technology of your choice would be appreciated. Also, any comments on a better approach on making a non-blocking REST API would be helpful.
Let me know if you need any further clarification!
Not sure if this can actually solve your problem but you can use the coroutine based web server gevent. They have a WSGI server that you can use if that's what you meant by saying that you need to deploy a WSGI production server.
If you want to implement the server to your flask application just do the following:
from gevent.pywsgi import WSGIServer
app = Flask(__name__)
http_server = WSGIServer(('', 5000), app)
http_server.serve_forever()
Gevent in general is a very powerful tool and by issuing context switches as necessary it can handle multiple clients very easily. Also, gevent fully supports flask.
First thing to do would be to put exception handling to deal with bad JSON request data (which maybe is what's happening) something like
def getSinglePrediction(self, route, busStop):
try:
prediction = self.predictor.getPredictionForStop(route, busStop)
return jsonify({'busStop': busStop, 'prediction': prediction})
except:
return jsonify({'busStop': 'error', 'prediction': 'error'})