I am using a python script which uses cmdloop and takes input from the commandline:
import cmd
class SuperC2(cmd.Cmd):
def default(self, line):
f = open("command.txt", "w")
f.write(line)
f.close()
if __name__ == '__main__':
SuperC2().cmdloop('Shell')
Another script uses flask and reads the command.txt file and serves it (via GET request) to a client. It also listens for POSTs of clients that executed this command and posted the output:
from flask import Flask
from flask import request
app = Flask(__name__)
#app.route('/command')
def command():
str = open('command.txt', 'r').read()
return str
#app.route('/result', methods = ['POST'])
def result():
if request.method == 'POST':
data = request.form
print(data["command"])
print(data["output"])
return "ok"
if __name__ == '__main__':
app.run(host='0.0.0.0', port=8000)
What I really would like to achieve is to have 1 script that runs the flask app and the cmdloop at the same time, with only the cmdloop showing the command and the result (it got from the flask POSTs). Can someone give an example on how it can be done? I'm guessing Process Queues or pipes?
Generally, the simplest way to transfer data between two scripts/threads/what have you is to use a socket from the socket library. Just have one script/thread host a server bound to 127.0.0.1 and connect to it from the other process!
Related
The following scenario:
I have a Raspberry Pi running as a server. Currently I am using a Python script with Flask and I can also access the Raspberry Pi from my PC. (The flask server runs an react app.)
But the function should be extended. It should look like the following:
2nd Python script is running all the time. This Python script fetches data from an external API every second and processes it. If certain conditions are met, the data should be processed and then the data should be communicated to the Python Flask server. And the Flask server then forwards the data to the website running on the computer.
How or which method is best to program this "interprocess communication". Are there any libraries? I tried Celery, but then it throws up my second Python script whenever I want to access the external API, so I don't know if this is the right choice.
What else would be the best approach? Threading? Direct interprocess communication?
If important, this is how my server application looks so far:
from gevent import monkey
from flask import Flask, render_template
from flask_socketio import SocketIO
monkey.patch_all()
app = Flask(__name__, template_folder='./build', static_folder='./build/static')
socket_io = SocketIO(app)
#app.route('/')
def main():
return render_template('index.html')
#socket_io.on('fromFrontend')
def handleInput(input):
print('Input from Frontend: ' + input)
send_time()
#socket_io.on('time')
def send_time():
socket_io.emit('time', {'returnTime': "some time"})
if __name__ == '__main__':
socket_io.run(app, host='0.0.0.0', port=5000, debug=True)
Well i found a solution for my specific problem i implemented it with a thread as follows:
import gevent.monkey
gevent.monkey.patch_all()
from flask import Flask, render_template
from flask_socketio import SocketIO
import time
import requests
from threading import Thread
app = Flask(__name__, template_folder='./build', static_folder='./build/static')
socket_io = SocketIO(app)
#app.route('/')
def main():
thread = Thread(target=backgroundTask)
thread.daemon = True
thread.start()
return render_template('index.html')
#socket_io.on('fromFrontend')
def handleInput(input):
print('Input from Frontend: ' + input)
#socket_io.on('time')
def send_time():
socket_io.emit('time', {'returnTime': 'hi frontend'})
def backgroundTask():
# do something here
# access socket to push some data
socket_io.emit('time', {'returnTime': "some time"})
if __name__ == '__main__':
socket_io.run(app, host='0.0.0.0', port=5000, debug=True)
I have a RPC Server using zerorpc in Python, written this way
import zerorpc
from service import Service
print('RPC server - loading')
def main():
print('RPC server - main')
s = zerorpc.Server(Service())
s.bind("tcp://*:4242")
s.run()
if __name__ == "__main__" : main()
It works fine when I create a client
import zerorpc, sys
client_rpc = zerorpc.Client()
client_rpc.connect("tcp://127.0.0.1:4242")
name = sys.argv[1] if len(sys.argv) > 1 else "dude"
print(client_rpc.videos('138cd9e5-3c4c-488a-9b6f-49907b55a040.webm'))
and runs it. The print() outputs what this 'videos' function returns.
But when I try to use it this same code inside route from a Flask app, I receive the following error:
File "src/gevent/__greenlet_primitives.pxd", line 35, in
gevent.__greenlet_primitives._greenlet_switch
gevent.exceptions.LoopExit: This operation would block forever Hub:
The flask method/excerpt
import zerorpc, sys
client_rpc = zerorpc.Client()
client_rpc.connect("tcp://127.0.0.1:4242")
#app.route('/videos', methods=['POST'])
def videos():
global client_rpc
client_rpc.videos('138cd9e5-3c4c-488a-9b6f-49907b55a040.webm')
I can't find out what might be happening. I'm quite new to Python and I understand that this may have some relation with Flask and how it handles the thread, but I can't figure out how to solve it.
zerorpc depends on gevent, which provides async IO with cooperative coroutines. This means your flask application must use gevent for all IO operations.
In your specific case, you are likely starting your application with a standard blocking IO WSGI server.
Here is a snippet using the WSGI server from gevent:
import zerorpc
from gevent.pywsgi import WSGIServer
app = Flask(__name__)
client_rpc = zerorpc.Client()
client_rpc.connect("tcp://127.0.0.1:4242")
#app.route('/videos', methods=['POST'])
def videos():
global client_rpc
client_rpc.videos('138cd9e5-3c4c-488a-9b6f-49907b55a040.webm')
# ...
if __name__ == "__main__":
http = WSGIServer(('', 5000), app)
http.serve_forever()
Excerpt from https://sdiehl.github.io/gevent-tutorial/#chat-server
I want to build a Webapp with Flask where some data is printed on a dynamic page in real time.
The data is taken from a Python script which connects to a Websocket, then it's printed on the frontend with Flask.
I have two problems:
1) I can't run both the scripts together
2) I don't know how to call parsed from test to yield
Here is the code:
from time import sleep
from flask import Flask, render_template
import websocket
from bitmex_websocket import Instrument
from bitmex_websocket.constants import InstrumentChannels
from bitmex_websocket.constants import Channels
import json
from threading import Thread, Event
app = Flask(__name__)
websocket.enableTrace(True)
channels = [
InstrumentChannels.trade,
]
XBTUSD = Instrument(symbol='XBTUSD',
channels=channels)
XBTUSD.on('action', lambda msg: test(msg))
def test(msg):
parsed = json.loads(json.dumps(msg))
print(parsed)
#app.route('/')
def index():
# render the template (below) that will use JavaScript to read the stream
return render_template('index.html')
#app.route('/stream_sqrt')
def stream():
def generate():
yield '{}\n'.format('test')
return app.response_class(generate(), mimetype='text/plain')
if __name__ == '__main__':
XBTUSD.run_forever()
app.run()
If i put XBTUSD.run_forever() before app.run() i will start the part supposed to retrieve the data but the Flask app won't start. If i do the opposite, the Flask app will run but not the other part. How can i run together the whole app? How could i "share" variables between test and generate?
An easier way to go, please use flask-socketio instead flask.
https://flask-socketio.readthedocs.io/en/latest/
Sample for sending messages using flask-socketio
https://flask-socketio.readthedocs.io/en/latest/#sending-messages
My server.py is as follows,
from flask import Flask, jsonify, Response, redirect
import json
from UIAccess import UIAccess
app=Flask(__name__)
#app.route('/Hello/<username>')
def id_no(username):
id= obj.get_id(username)
return json.dumps(id)
if __name__ == '__main__':
obj=UIAccess()
app.run(threaded=True)
when I run the program and load the page using my browser I am able to view the output of 'id_no' but if I run the same program using twisted with the command,
twistd web --wsgi server.app
I get an internal server error, I am wondering whether this is the correct way to do this?
You only create obj if __name__ == '__main__', which it does not when you run with something besides python server.py. But the id_no view depends on obj being defined, so it fails. Move obj = UIAccess() out of the guard block.
Consider the following minimal working flask app:
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
return "I am /"
#app.route("/api")
def api():
return "I am /api"
if __name__ == "__main__":
app.run()
This happily works. But when I try to make a GET request with the "requests" module from the hello route to the api route - I never get a response in the browser when trying to access http://127.0.0.1:5000/
from flask import Flask
import requests
app = Flask(__name__)
#app.route("/")
def hello():
r = requests.get("http://127.0.0.1:5000/api")
return "I am /" # This never happens :(
#app.route("/api")
def api():
return "I am /api"
if __name__ == "__main__":
app.run()
So my questions are: Why does this happen and how can I fix this?
You are running your WSGI app with the Flask test server, which by default uses a single thread to handle requests. So when your one request thread tries to call back into the same server, it is still busy trying to handle that one request.
You'll need to enable threading:
if __name__ == "__main__":
app.run(threaded=True)
or use a more advanced WSGI server; see Deployment Options.