I've my Mosquitto MQTT broker and I've created a simple Django APP that subscribes to the topic $SYS/broker/uptime like below
from django.apps import AppConfig
from threading import Thread
import paho.mqtt.client as mqtt
class MqttClient(Thread):
def __init__(self, broker, port, timeout, topics):
super(MqttClient, self).__init__()
self.client = mqtt.Client()
self.broker = broker
self.port = port
self.timeout = timeout
self.topics = topics
self.total_messages = 0
# run method override from Thread class
def run(self):
self.connect_to_broker()
def connect_to_broker(self):
self.client.on_connect = self.on_connect
self.client.on_message = self.on_message
self.client.connect(self.broker, self.port, self.timeout)
self.client.loop_forever()
# The callback for when a PUBLISH message is received from the server.
def on_message(self, client, userdata, msg):
self.total_messages = self.total_messages + 1
print(str(msg.payload) + "Total: {}".format(self.total_messages))
# The callback for when the client receives a CONNACK response from the server.
def on_connect(self, client, userdata, flags, rc):
# Subscribe to a list of topics using a lock to guarantee that a topic is only subscribed once
for topic in self.topics:
client.subscribe(topic)
class AppMqtteConfig(AppConfig):
default_auto_field = 'django.db.models.BigAutoField'
name = 'app_mqtt'
def ready(self):
MqttClient("localhost", 1883, 60, ["$SYS/broker/uptime"]).start()
For some reason, the print statement on the on_message callback got executed two times, at least from what I'm seeing from the console. See screenshot. I can't understand why
It turned out that in some instances, particularly in tests, the ready method could be called multiple times as suggested in the Documentation.
As explained on this answer the issue is resolved with an if os.environ.get('RUN_MAIN'): before the ready method
Related
I am working on a websocket application that utilizes WebSocketApp from websocket-client. As my websocket is running, and listening for messages I would like to be able to call a method to send a message to the server as needed.
WebSocket
import websocket
class WebSocketTest():
wsapp = None
print("starting websocket app")
ws_is_open = False
def on_message(self, wsapp, message):
print("ON_MESSAGE")
print(message)
def on_open(self, wsapp):
print("SOCKET OPENING")
self.ws_is_open = True
def on_close(self):
print("SOCKET CLOSING")
async def start(self):
self.wsapp = websocket.WebSocketApp("ws://localhost:5001",
on_message=self.on_message,
on_close=self.on_close,
on_open=self.on_open)
self.wsapp.run_forever()
def send(self, msg):
if self.ws_is_open:
print(f"SENDING: {msg}")
self.wsapp.send(msg)
Test Setup File
from websocketapp import WebSocketTest
import asyncio
ws = WebSocketTest()
asyncio.run(ws.start())
ws.send("Hello!")
Output
some_user python % python test.py
starting websocket app
SOCKET OPENING
I have created a setup file that initializes the class, starts the server and then attempts to send a message using the send method. It looks like it never gets to the line that calls send. The connection to the server works, and if I broadcast a message from my websocket server to the client on_message is triggered and I see the message printed out.
I am having an issue with the code below.
The code works perfectly at the beginning. First it says Connected to MQTT Broker! and receives data from it. But after a long time (like 6 hours, or 10 hours etc.) it says again Connected to MQTT Broker! and after that id does not receive any other data.
I am trying to make this program work forever, but i don't know what i have done wrong.
Any ideas?
# python3.6
import random
import mysql.connector
from paho.mqtt import client as mqtt_client
import json
# Code for MQTT Connection
broker = 'YOUR_BROKER'
port = 1883
topic = "YOUR_TOPIC"
# generate client ID with pub prefix randomly
client_id = f'python-mqtt-{random.randint(0, 100)}'
username = "THE_USERNAME"
password = "THE_PASSWORD"
# Function to connect on mqtt
def connect_mqtt() -> mqtt_client:
def on_connect(client, userdata, flags, rc):
if rc == 0:
print("Connected to MQTT Broker!")
else:
print("Failed to connect, return code %d\n", rc)
client = mqtt_client.Client(client_id)
client.username_pw_set(username, password)
client.on_connect = on_connect
client.connect(broker, port)
return client
# function to subscribe from mqtt
def subscribeFunc(client: mqtt_client):
def on_messageFunc(client, userdata, msg):
print(f"Received `{msg.payload.decode()}` from `{msg.topic}` topic")
client.subscribe(topic)
client.on_message = on_messageFunc
def run():
client = connect_mqtt()
subscribeFunc(client)
client.loop_forever()
if __name__ == '__main__':
run()
I tried to find the problem but it seems that nothing changed significantly.
I am expecting this program to receive data without stopping.
Network connections may not be 100% reliable (and servers etc are restarted from time to time) so expecting the connection to remain up forever is unrealistic. The issue with your code is that it connects then subscribes; if the connection is dropped it does not resubscribe. As the connection is clean_session=true and subscription qos=0 (the defaults) the broker will forget about any active subscriptions when the connection drops (so the client will reconnect but not receive any more messages).
The Simple solution is to use the approach shown in the docs and subscribe in the on_connect callback (that way the subscription will be renewed after reconnection):
def on_connect(client, userdata, flags, rc):
# Subscribing in on_connect() means that if we lose the connection and
# reconnect then subscriptions will be renewed.
client.subscribe("$SYS/#")
client = mqtt.Client()
client.on_connect = on_connect
client.connect("mqtt.eclipseprojects.io", 1883, 60)
You may also want to consider the advice in this answer (as per the comment from #mnikley) because that way the broker will queue up messages for you while the connection is down (otherwise these will be lost).
I have a python script, which based on flask and mqtt. The use case is to receive a request via rest-api then to create a new thread which publishes some messages on mosquitto mqtt and expects a response (see subscribe). My problem is that I don't receive any messages. I think it has something to do with the thread, because without the thread it's working very fine..
Do you know what can be the problem?
Thank you in anticipation!
here the code:
from flask import Flask, Response
import paho.mqtt.client as mqtt
from threading import Thread
import threading
app = Flask(__name__)
lock = threading.Lock()
def on_connect(client, userdata, flags, rc): # The callback for when the client connects to the broker
print("Connected with result code {0}".format(str(rc))) # Print result of connection attempt
client.subscribe("/mytopic")
def on_message(client, userdata, msg): # The callback for when a PUBLISH message is received from the server.
print(msg.topic)
client = mqtt.Client(client_id=client_name, clean_session=True)
client.on_connect = on_connect # Define callback function for successful connection
client.on_message = on_message # Define callback function for receipt of a message
client.username_pw_set(mqtt_user, mqtt_password)
client.loop_start()
client.connect(mqtt_host)
def test(param1, param2):
lock.acquire()
try:
ret = client.publish("/mytopic", "")
while True:
check the response from mqtt => but i don't get the response anymore
....
break
finally:
lock.release()
return result
#app.route('/test/check', methods=['POST'])
def check():
global sessionId
sessionId = sessionId + 1
t = Thread(target=test, args=(sessionId,None))
t.start()
return {'id': sessionId, 'eta': 0}
if __name__ == '__main__':
app.run(debug=True)
There are a couple of problems with this.
Both the client.connect() and the client.subscribe() calls need iterations of the client network loop to run in order to complete properly.
The network loop needs to run at least once every keep alive period the time after the connection has been made in order to stop the broker disconnecting the client as dead. This means if there is a delay between starting the code and the first REST request then the client will be disconnected.
Better to use the client.start_loop() function to run MQTT client network loop continuously in the background on it's own.
You should also remove the call to client.subscribe() that is outside the on_connect() callback.
EDIT:
As hashed out in the comments/chat the following works. It looks like running the flask app in debug mode does some strange things and creates multiple MQTT clients over and over again with the same client id. This leads to the broker constantly kicking the old ones off so messages never get delivered.
from flask import Flask, Response
import paho.mqtt.client as mqtt
import time
from threading import Thread
import threading
app = Flask(__name__)
lock = threading.Lock()
sessionId=0
cont=True
def on_connect(client, userdata, flags, rc): # The callback for when the client connects to the broker
print("Connected with result code {0}".format(str(rc))) # Print result of connection attempt
client.subscribe("mytopic")
def on_message(client, userdata, msg): # The callback for when a PUBLISH message is received from the server.
global cont
print(msg.topic)
cont=False
client = mqtt.Client(client_id="foo", clean_session=True)
client.on_connect = on_connect # Define callback function for successful connection
client.on_message = on_message # Define callback function for receipt of a message
#client.username_pw_set(mqtt_user, mqtt_password)
client.connect("localhost", port=1884)
client.loop_start()
def test(param1, param2):
lock.acquire()
try:
ret = client.publish("mytopic", "foo")
while cont:
time.sleep(5)
print("loop")
finally:
lock.release()
result = "foo"
return result
#app.route('/test/check', methods=['POST'])
def check():
global sessionId
sessionId = sessionId + 1
t = Thread(target=test, args=(sessionId,None))
t.start()
return {'id': sessionId, 'eta': 0}
if __name__ == '__main__':
print("started")
app.run()
I have tested the example program on paho-mqtt, and I know that the function loop_forever() can handles reconnecting. But my question is that, although loop_forever() can reconnect, it cannot re-subscribe. It should be a problem when the server suddenly crashes, in this case, the client is still listening, but when the server is restarted, the client can reconnect, but cannot subscribe message any more. I think maybe I should re-write the loop_forever() function, but I am not sure if I were right, and how to do it.
import sys
try:
import paho.mqtt.client as mqtt
except ImportError:
# This part is only required to run the example from within the examples
# directory when the module itself is not installed.
#
# If you have the module installed, just use "import paho.mqtt.client"
import os
import inspect
cmd_subfolder = os.path.realpath(os.path.abspath(os.path.join(os.path.split(inspect.getfile( inspect.currentframe() ))[0],"../src")))
if cmd_subfolder not in sys.path:
sys.path.insert(0, cmd_subfolder)
import paho.mqtt.client as mqtt
def on_connect(mqttc, obj, flags, rc):
print("rc: "+str(rc))
def on_message(mqttc, obj, msg):
print(msg.topic+" "+str(msg.qos)+" "+str(msg.payload))
def on_publish(mqttc, obj, mid):
print("mid: "+str(mid))
def on_subscribe(mqttc, obj, mid, granted_qos):
print("Subscribed: "+str(mid)+" "+str(granted_qos))
def on_log(mqttc, obj, level, string):
print(string)
# If you want to use a specific client id, use
# mqttc = mqtt.Client("client-id")
# but note that the client id must be unique on the broker. Leaving the client
# id parameter empty will generate a random id for you.
mqttc = mqtt.Client()
mqttc.on_message = on_message
mqttc.on_connect = on_connect
mqttc.on_publish = on_publish
mqttc.on_subscribe = on_subscribe
# Uncomment to enable debug messages
#mqttc.on_log = on_log
mqttc.connect("m2m.eclipse.org", 1883, 60)
mqttc.subscribe("$SYS/#", 0)
mqttc.loop_forever()
The easy way to deal with this is to do your subscribing in the on_connect callback, then when you reconnect all the subscriptions will be restored as well.
While instantiating your mqtt client you can set the "clean session" flag to false.
mqttc = mqtt.Client(clean_session=False)
Citation from the mosquitto manual:
Clean session / Durable connections
On connection, a client sets the "clean session" flag, which is sometimes also known as the "clean start" flag. If clean session is set to false, then the connection is treated as durable. This means that when the client disconnects, any subscriptions it has will remain and any subsequent QoS 1 or 2 messages will be stored until it connects again in the future. If clean session is true, then all subscriptions will be removed for the client when it disconnects.
Through TCPListner of python class ThreadedTCPServer i am not able to get mqtt connection more than 250-255 when connecting to RabbitMQ broker following is my code :
class service(SocketServer.BaseRequestHandler):
def handle(self):
def on_connect(client, userdata, flags, rc):
client.subscribe("subscribe")
def on_message(client, userdata, msg):
print "Received on Topic : '"+str(msg.topic)+"' and Message is : "+str(msg.payload)
strConnName="MQTT-Pub-Sub-"+str(self.client_address)
mqttc = mqtt.Client(strConnName,True,threadcounter)
mqttc.on_connect = on_connect
mqttc.on_message = on_message
mqttc.on_disconnect = on_disconnect
mqttc.connect("BROKERIP", 1883, 1000)
mqttc.loop_start()
while True:
data = self.request.recv(8192)
if data:
mqttc.publish("routing_key",str(data))
class ThreadedTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
pass
SocketServer.TCPServer.allow_reuse_address = True
t = ThreadedTCPServer((socket.gethostname(),PORT), service)
t.serve_forever()
So, when connecting to broker I am creating distinct connection for each new TCP request, But it is not going above 255.
Is there any limit for python ThreadedTCPServer Class or any thing missing or need to be configured. Although my machine ulimit -n is 8012
Thanks in advance.