Server:
Ubuntu 18.04.3 LTS
python3.6
I want show device's message on web, my structure like:
Device--(MQTT)-->Server--(socket.io)-->Web
First, I publish message by mqtt to Server, and Server successed subscribed to get message .
Second, I use socket.io to send message to Web, but problem is here, if Server send message by socket.io after mqtt.on_message, the web's socket.io will be stuck, until refresh web.
My Flask Server a part of code:
from flask import Flask, request, render_template, flash, redirect, url_for
from flask_mqtt import Mqtt
from flask_socketio import SocketIO
from flask_bootstrap import Bootstrap
from flask_login import UserMixin, LoginManager, login_required, current_user, login_user, login_required, logout_user
from flask_sqlalchemy import SQLAlchemy
from flask_bcrypt import Bcrypt
from forms import RegisterForm, LoginForm, AddForm, ChangePasswordForm
import pymysql
from config import Config
from sqlalchemy.ext.declarative import declarative_base
import json
import plotly as py
from datetime import datetime
import eventlet
eventlet.monkey_patch()
# MQTT
app = Flask(__name__)
bcrypt = Bcrypt(app)
app.config['SECRET'] = ''
app.config['TEMPLATES_AUTO_RELOAD'] = True
app.config['MQTT_BROKER_URL'] = ''
app.config['MQTT_BROKER_PORT'] =
app.config['MQTT_USERNAME'] = ''
app.config['MQTT_PASSWORD'] = ''
app.config['MQTT_KEEPALIVE'] = 5
app.config['MQTT_TLS_ENABLED'] = False
app.config['SQLALCHEMY_DATABASE_URI'] = ''
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
mqtt = Mqtt(app)
socketio = SocketIO(app, cors_allowed_origins='*')
if __name__ == '__main__':
socketio.run(app, keyfile='privkey.pem', certfile='fullchain.pem', host='0.0.0.0', port=5002, use_reloader=True, debug=True)
Flask Server MQTT and socket.io part of code:
#mqtt.on_connect()
def handle_connect(client, userdata, flags, rc):
mqtt.subscribe('test')
#mqtt.on_message()
def handle_mqtt_message(client, userdata, message):
topic = message.topic,
raw_payload = message.payload.decode()
if(topic[0] == "test"):
socketio.emit('web', "received")
Web Client javascript code:
socket.on('web', function(data) {
console.log(data);
})
If just use socket.io to send message to web didn't use mqtt it's work fine. I don't know why they can't interact together. Then I find something on flask-mqtt offical file about "MQTT Interact with SocketIO", example code the difference from me is "import eventlet". So i guess is abotu threading problem? But when i also import it will show error, the error is:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/eventlet/hubs/hub.py", line 461, in fire_timers
timer()
File "/usr/local/lib/python3.6/dist-packages/eventlet/hubs/timer.py", line 59, in __call__
cb(*args, **kw)
File "/usr/local/lib/python3.6/dist-packages/eventlet/greenthread.py", line 221, in main
result = function(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/eventlet/wsgi.py", line 818, in process_request
proto.__init__(conn_state, self)
File "/usr/local/lib/python3.6/dist-packages/eventlet/wsgi.py", line 357, in __init__
self.handle()
File "/usr/local/lib/python3.6/dist-packages/eventlet/wsgi.py", line 390, in handle
self.handle_one_request()
File "/usr/local/lib/python3.6/dist-packages/eventlet/wsgi.py", line 419, in handle_one_request
self.raw_requestline = self._read_request_line()
File "/usr/local/lib/python3.6/dist-packages/eventlet/wsgi.py", line 402, in _read_request_line
return self.rfile.readline(self.server.url_length_limit)
File "/usr/lib/python3.6/socket.py", line 586, in readinto
return self._sock.recv_into(b)
File "/usr/local/lib/python3.6/dist-packages/eventlet/green/ssl.py", line 241, in recv_into
return self._base_recv(nbytes, flags, into=True, buffer_=buffer)
File "/usr/local/lib/python3.6/dist-packages/eventlet/green/ssl.py", line 256, in _base_recv
read = self.read(nbytes, buffer_)
File "/usr/local/lib/python3.6/dist-packages/eventlet/green/ssl.py", line 176, in read
super(GreenSSLSocket, self).read, *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/eventlet/green/ssl.py", line 146, in _call_trampolining
return func(*a, **kw)
File "/usr/lib/python3.6/ssl.py", line 874, in read
return self._sslobj.read(len, buffer)
File "/usr/lib/python3.6/ssl.py", line 631, in read
v = self._sslobj.read(len, buffer)
ssl.SSLWantReadError: The operation did not complete (read) (_ssl.c:2309)
I searched about this error, some result say this is eventlet's Bug on python3, Is any solution for this error?
Related
So for our last coding projects we've set up a web API that runs on local host. But now he has set up a virtual server for us to use along with usernames and passwords to use as well as which ports we are alllowed to use. I have my current code here. On the last line it used to be localhost and the port was 8080 but I changed it to reflect the new server we were given. However, it doesn't work and I couldn't seem to find a solution online. I also have the IP address and it didn't work either. I wasn't sure how to add my username and password to the mix as well as I am sure it is needed to access the server.
from flask_cors import CORS
import os
from flask import Flask, jsonify, make_response, Blueprint
from flask_swagger_ui import get_swaggerui_blueprint
from routes import request_api
import ssl
context = ssl.SSLContext()
context.load_cert_chain('certificate.pem', 'key.pem')
APP = Flask(__name__)
SWAGGER_URL = '/swagger'
API_URL = '/static/swagger.json'
SWAGGERUI_BLUEPRINT = get_swaggerui_blueprint(
SWAGGER_URL,
API_URL,
config={
'app_name': "Kales Flask Project"
}
)
APP.register_blueprint(SWAGGERUI_BLUEPRINT, url_prefix=SWAGGER_URL)
APP.register_blueprint(request_api.get_blueprint())
#APP.errorhandler(400)
def handle_400_error(_error):
return make_response(jsonify({'error': 'Misunderstood'}), 400)
#APP.errorhandler(404)
def handle_404_error(_error):
return make_response(jsonify({'error': 'Not found'}), 404)
#APP.errorhandler(401)
def handle_401_error(_error):
return make_response(jsonify({'error': 'Invalid Key Provided'}), 401)
if __name__ == '__main__':
CORS = CORS(APP)
APP.run(host='easel4.cs.utsarr.net', port=int(os.environ.get('PORT', 12145)), ssl_context=context)
Here is the output when I attempt to run this code
C:\Users\kingk\PycharmProjects\AdvanceSoft>python webapi.py
* Serving Flask app "webapi" (lazy loading)
* Environment: production
WARNING: Do not use the development server in a production environment.
Use a production WSGI server instead.
* Debug mode: off
Traceback (most recent call last):
File "webapi.py", line 44, in <module>
APP.run(host='10.100.201.3', port=12145, ssl_context=context)
File "C:\Users\kingk\PycharmProjects\AdvanceSoft\yes\lib\site-packages\flask\app.py", line 943, in run
run_simple(host, port, self, **options)
File "C:\Users\kingk\PycharmProjects\AdvanceSoft\yes\lib\site-packages\werkzeug\serving.py", line 1009, in run_simple
inner()
File "C:\Users\kingk\PycharmProjects\AdvanceSoft\yes\lib\site-packages\werkzeug\serving.py", line 962, in inner
fd=fd,
File "C:\Users\kingk\PycharmProjects\AdvanceSoft\yes\lib\site-packages\werkzeug\serving.py", line 805, in make_server
host, port, app, request_handler, passthrough_errors, ssl_context, fd=fd
File "C:\Users\kingk\PycharmProjects\AdvanceSoft\yes\lib\site-packages\werkzeug\serving.py", line 698, in __init__
HTTPServer.__init__(self, server_address, handler)
File "C:\Users\kingk\AppData\Local\Programs\Python\Python37\lib\socketserver.py", line 452, in __init__
self.server_bind()
File "C:\Users\kingk\AppData\Local\Programs\Python\Python37\lib\http\server.py", line 137, in server_bind
socketserver.TCPServer.server_bind(self)
File "C:\Users\kingk\AppData\Local\Programs\Python\Python37\lib\socketserver.py", line 466, in server_bind
self.socket.bind(self.server_address)
OSError: [WinError 10049] The requested address is not valid in its context
I assume the host='easel4.cs.utsarr.net' is the problem. The Flask documentation of the run method of the Application object recommends:
host – the hostname to listen on. Set this to '0.0.0.0' to have the server available externally as well. Defaults to '127.0.0.1' or the host in the SERVER_NAME config variable if present.
Hi Pythonanywhere fellows,
I'd like to subscribe to a TheThingsNetwork topic using the MQTT API: https://www.thethingsindustries.com/docs/integrations/mqtt/
The Api works fine for me using the paho-mqtt client: https://pypi.org/project/paho-mqtt/ My code looks as follows:
import context
import paho.mqtt.subscribe as subscribe
import time
import re
import json
while True:
m = subscribe.simple(topics=['#'], hostname="eu1.cloud.thethings.network", port=1883, auth={'username':"mycrazyusername#ttn",'password':"mycrazypw"}, msg_count=2)
for a in m:
payload = json.loads(a.payload)
print(payload)
I am receiving the following error that is related to the socks module:
File "/home/Markovicz/.local/lib/python3.8/site-packages/paho/mqtt/subscribe.py", line 262, in simple
callback(_on_message_simple, topics, qos, userdata, hostname, port,
File "/home/Markovicz/.local/lib/python3.8/site-packages/paho/mqtt/subscribe.py", line 174, in callback
client.connect(hostname, port, keepalive)
File "/home/Markovicz/.local/lib/python3.8/site-packages/paho/mqtt/client.py", line 941, in connect
return self.reconnect()
File "/home/Markovicz/.local/lib/python3.8/site-packages/paho/mqtt/client.py", line 1075, in reconnect
sock = self._create_socket_connection()
File "/home/Markovicz/.local/lib/python3.8/site-packages/paho/mqtt/client.py", line 3533, in _create_socket_connection
proxy = self._get_proxy()
File "/home/Markovicz/.local/lib/python3.8/site-packages/paho/mqtt/client.py", line 3494, in _get_proxy
if self._proxy_is_valid(self._proxy):
File "/home/Markovicz/.local/lib/python3.8/site-packages/paho/mqtt/client.py", line 3483, in _proxy_is_valid
return check(p.get("proxy_type"), p.get("proxy_addr"))
File "/home/Markovicz/.local/lib/python3.8/site-packages/paho/mqtt/client.py", line 3480, in check
t in set([socks.HTTP, socks.SOCKS4, socks.SOCKS5]) and a)
AttributeError: module 'socks' has no attribute 'HTTP'
Do you have any ideas how to fix that?
This error has previously been reported here: https://www.pythonanywhere.com/forums/topic/27492/
I am trying to send data from client where I am using python-socketio as mentioend below
#Client.py
import time
import socketio
sio = socketio.Client(engineio_logger=True)
start_timer = None
# if __name__ == '__main__':
sio.connect('http://127.0.0.1:3000')
sio.wait()
sio.emit('connect', {"Data": "Device_id"})
and trying to on server which is using flask-socketio as mentioned below in code
#Server.py
from flask import Flask, render_template, request, jsonify
from flask_socketio import SocketIO
app = Flask(__name__)
# app.config['SECRET_KEY'] = "Social Distance Secret"
socket_app = SocketIO(app)
#socket_app.on('connect')
def handle_id(data):
print(data)
print(request.sid)
if __name__ == '__main__':
socket_app.run(app, debug=True, host='127.0.0.1', port=3000)
I am able to receive the sid but I am not able to fetch the parameters I have given in client.py while emitting to WebSocket
# Error Server.py
Traceback (most recent call last):
File "C:\Users\varul.jain\AppData\Local\Programs\Python\Python36\lib\site-packages\engineio\server.py", line 545, in _trigger_event
return self.handlers[event](*args)
File "C:\Users\varul.jain\AppData\Local\Programs\Python\Python36\lib\site-packages\socketio\server.py", line 721, in _handle_eio_connect
return self._handle_connect(sid, '/')
File "C:\Users\varul.jain\AppData\Local\Programs\Python\Python36\lib\site-packages\socketio\server.py", line 626, in _handle_connect
self.environ[sid])
File "C:\Users\varul.jain\AppData\Local\Programs\Python\Python36\lib\site-packages\socketio\server.py", line 708, in _trigger_event
return self.handlers[namespace][event](*args)
File "C:\Users\varul.jain\AppData\Local\Programs\Python\Python36\lib\site-packages\flask_socketio-4.3.1.dev0-py3.6.egg\flask_socketio\__init__.py", line 283, in _handler
*args)
File "C:\Users\varul.jain\AppData\Local\Programs\Python\Python36\lib\site-packages\flask_socketio-4.3.1.dev0-py3.6.egg\flask_socketio\__init__.py", line 711, in _handle_event
ret = handler()
TypeError: handle_id() missing 1 required positional argument: 'data'
# Errors client.py:
Attempting polling connection to http://127.0.0.1:3000/socket.io/?transport=polling&EIO=3
Traceback (most recent call last):
File "C:/Users/varul.jain/Desktop/people_counter/listenn.py", line 12, in <module>
sio.connect('http://127.0.0.1:3000')
File "C:\Users\varul.jain\AppData\Local\Programs\Python\Python36\lib\site-packages\socketio\client.py", line 279, in connect
six.raise_from(exceptions.ConnectionError(exc.args[0]), None)
File "<string>", line 3, in raise_from
socketio.exceptions.ConnectionError: Unexpected status code 401 in server response
any suggestion will be really helpful
First problem is that you are using a connect event, but this event is reserved. Change the event name to something else:
#socket_app.on('connected')
def handle_id(data):
print(data)
print(request.sid)
The second problem is that in your client you are calling wait(), which blocks until the connection ends, so your emit() call will never get to run. Send the emit before the wait instead:
sio.connect('http://127.0.0.1:3000')
sio.emit('connected', {"Data": "Device_id"})
sio.wait()
I am trying to connect to my mongodb in mlab.com using flask but when I run my flask script I am getting Authentication failed error, please help.
my code:
from flask import Flask, jsonify, request
from flask_pymongo import PyMongo
app = Flask(__name__)
app.config['MONGO_DBNAME'] = 'mydb'
app.config['MONGO_URI'] = 'mongodb://user:pwd#ds157799.mlab.com:57799/mydb'
mongo = PyMongo(app)
#app.route('/framework', methods=['GET'])
def get_all_frameworks():
framework = mongo.db.framework
output = []
for q in framework.find():
output.append({'name' : q['name'], 'language' : q['language']})
return jsonify({'result' : output})
Error:
File "mongo.py", line 12, in <module>
mongo = PyMongo(app)
File "C:\Users\ELCOT\AppData\Local\Programs\Python\Python35\lib\site-packages\flask_pymongo\__init__.py", line 97, in __init__
self.init_app(app, config_prefix)
File "C:\Users\ELCOT\AppData\Local\Programs\Python\Python35\lib\site-packages\flask_pymongo\__init__.py", line 283, in init_app
mechanism=auth_mechanism)
File "C:\Users\ELCOT\AppData\Local\Programs\Python\Python35\lib\site-packages\pymongo\database.py", line 1167, in authenticate
connect=True)
File "C:\Users\ELCOT\AppData\Local\Programs\Python\Python35\lib\site-packages\pymongo\mongo_client.py", line 588, in _cache_credentials
sock_info.authenticate(credentials)
File "C:\Users\ELCOT\AppData\Local\Programs\Python\Python35\lib\site-packages\pymongo\pool.py", line 620, in authenticate
auth.authenticate(credentials, self)
File "C:\Users\ELCOT\AppData\Local\Programs\Python\Python35\lib\site-packages\pymongo\auth.py", line 486, in authenticate
auth_func(credentials, sock_info)
File "C:\Users\ELCOT\AppData\Local\Programs\Python\Python35\lib\site-packages\pymongo\auth.py", line 237, in _authenticate_scram_sha1
res = sock_info.command(source, cmd)
File "C:\Users\ELCOT\AppData\Local\Programs\Python\Python35\lib\site-packages\pymongo\pool.py", line 517, in command
collation=collation)
File "C:\Users\ELCOT\AppData\Local\Programs\Python\Python35\lib\site-packages\pymongo\network.py", line 125, in command
parse_write_concern_error=parse_write_concern_error)
File "C:\Users\ELCOT\AppData\Local\Programs\Python\Python35\lib\site-packages\pymongo\helpers.py", line 145, in _check_command_response
raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: Authentication failed.
Please help to resolve this.
You can use this as an alternative to sign in to your database. This is tested and works :
connection = pymongo.MongoClient(HOST, PORT)
db = connection[databasename]
db.authenticate(database_user, database_pass)
I created a new user with password for my db, and worked fine.
https://www.reddit.com/r/flask/comments/5ftqvm/how_to_use_pymongo_with_hosted_mongodb_mlab/
This Works fine. For me connecting via URI string from mLab database doesn't work as well.
I am using Miguel Grinberg's article to set up Celery with the app factory pattern in order to send email with Flask-Mail. I've been calling various scripts that use Celery without any issues. However I keep getting Runtime Error: working outside of application context with the following task even though I am running the worker inside an app context. Why am I getting this error? How do I get Flask-Mail to work in Celery?
email.py:
from flask import current_app, render_template
from flask.ext.mail import Message
from . import celery, mail
#celery.task
def send_async_email(msg):
mail.send(msg)
def send_email(to, subject, template, **kwargs):
with current_app.test_request_context(): # used app_context() as well.
msg = Message(current_app.config['PORTAL_MAIL_SUBJECT_PREFIX'] + ' ' + subject,
sender=current_app.config['PORTAL_MAIL_SENDER'], recipients=[to])
msg.body = render_template(template + '.txt', **kwargs)
msg.html = render_template(template + '.html', **kwargs)
send_async_email.delay(msg)
__init__.py:
from flask import Flask
from celery import Celery
from flask.ext.mail import Mail
from configuration import config
mail = Mail()
celery = Celery(__name__, broker=config['default'].CELERY_BROKER_URL)
def create_app(config_name):
app = Flask(__name__)
app.config.from_object(config[config_name])
config[config_name].init_app(app)
mail.init_app(app)
celery.conf.update(app.config)
app.register_blueprint(main_blueprint)
return app
celery_worker.py:
import os
from app import celery, create_app
app = create_app(os.getenv('FLASK_CONFIG') or 'default')
app.app_context().push()
Error:
C:\Python27\Scripts\celery.exe worker -A celery_worker.celery --loglevel=info
[2015-09-30 12:07:34,408: INFO/MainProcess] Received task: app.email.send_async_email[3ec772ff-4767-49cb-90ba-445629da30da]
[2015-09-30 12:07:34,417: ERROR/MainProcess] Task app.email.send_async_email[3ec772ff-4767-49cb-90ba-445629da30da] raised unexpected: RuntimeError('working outside of application context',)
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\celery\app\trace.py", line 240, in trace_task
R = retval = fun(*args, **kwargs)
File "C:\Python27\lib\site-packages\celery\app\trace.py", line 438, in __protected_call__
return self.run(*args, **kwargs)
File "<flask_project_path>\app\email.py", line 10, in send_async_email
mail.send(msg)
File "C:\Python27\lib\site-packages\flask_mail.py", line 491, in send
with self.connect() as connection:
File "C:\Python27\lib\site-packages\flask_mail.py", line 508, in connect
return Connection(app.extensions['mail'])
File "C:\Python27\lib\site-packages\werkzeug\local.py", line 338, in __getattr__
return getattr(self._get_current_object(), name)
File "C:\Python27\lib\site-packages\werkzeug\local.py", line 297, in _get_current_object
return self.__local()
File "C:\Python27\lib\site-packages\flask\globals.py", line 34, in _find_app
raise RuntimeError('working outside of application context')
RuntimeError: working outside of application context
I have tried:
Trying to pass the application context to the send_email method.
Moving the send_async_email method to a tasks.py module where the rest of my celery tasks reside.
Rendering the templates outside of the email methods and passing them as arguments.
I was able to fix the issue by creating an instance of the flask application locally:
email.py:
from flask import render_template, current_app
from flask.ext.mail import Message
from . import celery, mail, create_app
#celery.task
def send_async_email(msg):
app = create_app('default' or 'development') # -> fixed
with app.app_context():
mail.send(msg)
def send_email(to, subject, template, **kwargs):
app = current_app._get_current_object()
msg = Message(current_app.config['PORTAL_MAIL_SUBJECT_PREFIX'] + ' ' + subject,
sender=current_app.config['MAIL_USERNAME'], recipients=[to])
msg.body = render_template(template + '.txt', **kwargs)
msg.html = render_template(template + '.html', **kwargs)
send_async_email.delay(msg)