I want to build a Webapp with Flask where some data is printed on a dynamic page in real time.
The data is taken from a Python script which connects to a Websocket, then it's printed on the frontend with Flask.
I have two problems:
1) I can't run both the scripts together
2) I don't know how to call parsed from test to yield
Here is the code:
from time import sleep
from flask import Flask, render_template
import websocket
from bitmex_websocket import Instrument
from bitmex_websocket.constants import InstrumentChannels
from bitmex_websocket.constants import Channels
import json
from threading import Thread, Event
app = Flask(__name__)
websocket.enableTrace(True)
channels = [
InstrumentChannels.trade,
]
XBTUSD = Instrument(symbol='XBTUSD',
channels=channels)
XBTUSD.on('action', lambda msg: test(msg))
def test(msg):
parsed = json.loads(json.dumps(msg))
print(parsed)
#app.route('/')
def index():
# render the template (below) that will use JavaScript to read the stream
return render_template('index.html')
#app.route('/stream_sqrt')
def stream():
def generate():
yield '{}\n'.format('test')
return app.response_class(generate(), mimetype='text/plain')
if __name__ == '__main__':
XBTUSD.run_forever()
app.run()
If i put XBTUSD.run_forever() before app.run() i will start the part supposed to retrieve the data but the Flask app won't start. If i do the opposite, the Flask app will run but not the other part. How can i run together the whole app? How could i "share" variables between test and generate?
An easier way to go, please use flask-socketio instead flask.
https://flask-socketio.readthedocs.io/en/latest/
Sample for sending messages using flask-socketio
https://flask-socketio.readthedocs.io/en/latest/#sending-messages
Related
The following scenario:
I have a Raspberry Pi running as a server. Currently I am using a Python script with Flask and I can also access the Raspberry Pi from my PC. (The flask server runs an react app.)
But the function should be extended. It should look like the following:
2nd Python script is running all the time. This Python script fetches data from an external API every second and processes it. If certain conditions are met, the data should be processed and then the data should be communicated to the Python Flask server. And the Flask server then forwards the data to the website running on the computer.
How or which method is best to program this "interprocess communication". Are there any libraries? I tried Celery, but then it throws up my second Python script whenever I want to access the external API, so I don't know if this is the right choice.
What else would be the best approach? Threading? Direct interprocess communication?
If important, this is how my server application looks so far:
from gevent import monkey
from flask import Flask, render_template
from flask_socketio import SocketIO
monkey.patch_all()
app = Flask(__name__, template_folder='./build', static_folder='./build/static')
socket_io = SocketIO(app)
#app.route('/')
def main():
return render_template('index.html')
#socket_io.on('fromFrontend')
def handleInput(input):
print('Input from Frontend: ' + input)
send_time()
#socket_io.on('time')
def send_time():
socket_io.emit('time', {'returnTime': "some time"})
if __name__ == '__main__':
socket_io.run(app, host='0.0.0.0', port=5000, debug=True)
Well i found a solution for my specific problem i implemented it with a thread as follows:
import gevent.monkey
gevent.monkey.patch_all()
from flask import Flask, render_template
from flask_socketio import SocketIO
import time
import requests
from threading import Thread
app = Flask(__name__, template_folder='./build', static_folder='./build/static')
socket_io = SocketIO(app)
#app.route('/')
def main():
thread = Thread(target=backgroundTask)
thread.daemon = True
thread.start()
return render_template('index.html')
#socket_io.on('fromFrontend')
def handleInput(input):
print('Input from Frontend: ' + input)
#socket_io.on('time')
def send_time():
socket_io.emit('time', {'returnTime': 'hi frontend'})
def backgroundTask():
# do something here
# access socket to push some data
socket_io.emit('time', {'returnTime': "some time"})
if __name__ == '__main__':
socket_io.run(app, host='0.0.0.0', port=5000, debug=True)
I'm using the Flask-SocketIO library which works fine but I need to send a notification with emit to the outside of a socket.io decorator and it's a real pain. Looking at the solutions, many people use rabbitmq or redis but I don't know how to use them.
Here's my code :
from flask import Flask, render_template
from flaskwebgui import FlaskUI
from flask_socketio import SocketIO, emit
app = Flask(__name__)
async_mode = None
app.config['SECRET_KEY'] = 'hello'
socketio = SocketIO(app, async_mode=async_mode, message_queue='amqp:///socketio')
def run_sock():
socketio.run(app, debug=True)
ui = FlaskUI(app, fullscreen=True, server=run_sock,)
#app.route("/")
def index():
return render_template('index.html')
#socketio.on('test', namespace='/test')
def test():
print("test")
if __name__ == "__main__":
ui.run()
io = SocketIO(message_queue='amqp:///socketio')
io.emit('test_emit', {'data': 'toto'}, namespace='/test')
My JS front-end never gets the test_emit message, how do I do?
The problem with your emit is that it appears below the ui.run() call, which does not return until you close the application. Move the emit to any function in your application that executes while the server is running (such as a Flask view function) and it should work just fine.
Why do you have two SocketIO objects in the same process? The socketio instance that you defined near the top of the script can be used anywhere within the process, no need to create a second instance. You do not need to use a message queue for this problem, since you have all the usages of Socket.IO within a single process.
I have been trying to follow the tutorials to get flask apps to run on Heroku, like this one: https://dev.to/emcain/how-to-set-up-a-twitter-bot-with-python-and-heroku-1n39.
They all tell you to put this in your code in a file server.py:
from flask import Flask
app = Flask(__name__)
app.run(host='0.0.0.0')
And then run the app via the following command:
python3 server.py
But the tutorials don't explain how to connect the actual function you want to run using the app. In my case, I have a File testbot.py that has the function test(arg1) that contains the code I want to execute:
def test(arg1):
while(1):
#do stuff with arg1 on twitter
I want to do something like this:
from flask import Flask
from testbot import test
from threading import Thread
app = Flask(__name__)
app.addfunction(test(arg1='hardcodedparameter'))
app.run(host='0.0.0.0')
So that when the app runs my test() function executes with the argument. Right now my server is starting, but nothing is happening.
Am I thinking about this correctly?
*Edit: I got it working with the solution, so my server.py now looks like this:
from flask import Flask
from testbot import test
def main_process():
test("hardcodeparam")
app = Flask(__name__)
Thread(target=main_process).start()
app.run(debug=True,host='0.0.0.0')
And now test runs as expected.
Before app.run, register the function with a path, e.g.
#app.route('/')
def test(): # no argument
... do one iteration
return 'ok'
Then visiting the URL will trigger the function. Sites such as https://cron-job.org/ can automate that visiting on a regular basis for free, as suggested here.
If the regular intervals aren't good enough, then you could try:
#app.route('/')
def index(): # no argument
return 'ok'
def test():
while True:
# do stuff
from threading import Thread
Thread(target=test).start()
app.run(...)
You will probably still need to have a job regularly visiting the URL so that Heroku sees that the server is alive and in use.
I have a script on python, which prints some data. The script is on Centos7, nginx.
How could I connect to the script via URL (GET query) to be able to parse the data?
You can use a framework like Django or flask to make an api out of it. I'll suggest flask since it's very light-weight, making it ideal for such small tasks.
E.g.
def your_function(input):
# do something
return output
from flask import Flask
from flask import request
app = Flask(__name__)
#app.route('/my_api')
def your_api_function():
input = request.args.get('my_query_string')
return your_function(input)
if __name__ == '__main__':
app.run(debug=True)
And then use the endpoint
/my_api?my_query_string=my_input
You can further play around with it to return JSON, take parameters from request body and so on and so forth.
Read more here http://flask.pocoo.org/
I'm developing a simple API in Flask, it isn't REST or anything.
There's a module that returns real-time data in a list.
The module in question
# module.py
def get_data():
do_something()
return [info_from_somewhere, info_from_other_place]
And the app
# app.py
import module
#app.route('/')
def get_data():
return jsonify(data=module.get_data()[0])
The thing is, this would run the function every time someones requests that route. And as the data is only one for all, I want to give it for every request but running the function just once.
Edit: I tried this:
got_data = module.get_data()
#app.route('/')
def get_data():
return jsonify(data=got_data[0])
Works, but don't refresh the list. So my question would be "how can I refresh it every second?" I tried sleep but it freezes my app
You could achieve this with celery. From project page.
Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.
Other solution could be done by spawning a thread, that will update the data every second, but this could get tricky really fast.
from threading import Timer
from flask import Flask
app = Flask(__name__)
DATA = "data"
def update_data(interval):
Timer(interval, update_data, [interval]).start()
global DATA
DATA = DATA + " updating..."
# update data every second
update_data(1)
#app.route("/")
def index():
return DATA
if __name__ == "__main__":
app.run(debug=True)