I try to setup a mqtt client in python3. This is not the first time im doing this, however i came across a rather odd behaviour.
When trying to call a function, which contains a bug, from one of the callback functions (on_connect or on_message), python does not throw an exception (at least it is not printed), it just stops there. I tied together a short example, that reproduces that behaviour.
Does someone has an idea?
import paho.mqtt.client as mqtt
import re
import os.path
import json
from termcolor import colored
client = mqtt.Client()
def func():
test = 1 + "1"
print("Should never reach that")
def on_connect(client, userdata, flags, rc):
"""Establishes connection to broker
"""
print("Connected to broker with result code " + str(rc))
client.subscribe("test")
def on_message(client,userdata,msg):
print("Recieved message on " + msg.topic)
params = {}
if msg.topic == "test":
print("Invoke func")
func()
if __name__ == "__main__":
client.on_connect = on_connect
client.on_message = on_message
client.connect("localhost",1883,60)
client.loop_forever()
This is the output when sending a message to the topic "test":
Connected to broker with result code 0
Recieved message on test
Invoke func
When calling func() from main, i get the correct TypeError thrown. So something catches this exception in paho. I had a look at an olderproject (python2 though) and tried to recreate the behaviour. There the exception gets thrown correctly. What do i miss?
EDIT
I can catch the exception by putting the func() call in a try block. How ever, it does not stop the execution of the program when not catched. I dont get why
For anybody who comes across this and wonders why all exceptions inside of a mqtt callback are not thrown or at least not visible: In contrast to the python2 version of paho, the clients already catches ALL exceptions that occur when calling on of the user set callback functions. The output of this catch is then outputted to the on_log callback function. If this is not implemented by the user, there will be no visible output. So just add
def on_log(client, userdata, level, buff):
print(buff)
mqttc.on_log = on_log
to your code, to print out the exception.
This will be due to the fact that the on_message function is called by the network thread and it will be wrapping that call in a try block to stop errors in on_message from stopping that thread.
If you want to an error to stop the app then you should use your own try block in on_message and behave appropriately.
You can catch the errors using try + expect and then manually print the error message and pointer to the source of error using the traceback. This will give you mode details than using the on_log function.
import traceback
def on_message(client, userdata, msg):
try:
do_something(msg)
except:
traceback.print_exc()
quit(0)
Related
I have a little python script which connects, subscribes, publish and disconnects to a mqtt broker. Everything is fine, but I only can use the received data in the on_message function.
first the actual code:
import time
import sys
sys.path.append('C:\\Users\\user\\Python\\pyproj\\project1\\Lib\\site-packages\\paho_mqtt-1.6.1-py3.10.egg')
import paho.mqtt.client as mqtt
userdata = "Leer"
client_name = "Pythonscript"
broker_address="000.000.000.000"
port = 1234
topic = "topic to publish"
subtopic = "topic to recieve"
mqtt1 = ''
def on_log(client, userdata, level, buf):
print(" client: ",client)
print(" userdata: ",userdata)
print(" level: ",level)
print(" buf: ",buf)
def on_connect(client, userdata, flags, rc):
if rc==0:
#print("connected OK ")
#print("Subscribing to topic ",subtopic)
client.subscribe(subtopic)
else:
print("Bad Connection Returned code=",rc)
def on_message(client,userdata,msg):
topic=msg.topic
m_decode=str(msg.payload.decode("utf-8","ignore"))
global mqtt1
mqtt1 = m_decode
print("message recieved:",m_decode)
print("message in mqtt1:",mqtt1)
return m_decode
def on_disconnect(client, userdata, flags, rc=0):
print("Disconnected result code "+str(rc))
print("creating new instance ",client_name)
client = mqtt.Client(client_name)
client.on_connect=on_connect
client.on_disconnect=on_disconnect
#client.on_log=on_log
client.on_message=on_message
print("connecting to broker ",broker_address+" :"+str(port))
client.connect(broker_address,port,60)
client.loop_start()
print("Publishing message to topic ",topic)
client.publish(topic, "python mqqt message")
result = mqtt1 + "changes"
print("recieved Message in Variable:",result)
time.sleep(4)
client.loop_stop()
client.disconnect()
i would like to get the whole message object and work with that data in my script.
I see my msg.payload in the console with print in on_message.
but i cant use it, for instance, in the variable result.
I tried with a global variable, with a return from the on_message function. but nothing works.
I think I don't understand which parameters I should give the on_message function to get my returned value (later the whole msg object)
perhaps somebody can help me to understand.
here is my console output:
=========== RESTART: C:\Users\user\Python\pyproj\project1\mqtt.py ==========
creating new instance Pythonscript
connecting to broker ip :1883
Publishing message to topic my topic
recieved Message in Variable: changes
message recieved: 73
message in mqtt1: 73
message recieved: 91
message in mqtt1: 91
message recieved: 57
message in mqtt1: 57
message recieved: 50
message in mqtt1: 50
message recieved: 50
message in mqtt1: 50
Disconnected result code 0
But I cant get the value in an variable out of the on_message function...
You need to remember that the on_message() function is not called by any of your code, it is called by the MQTT client's network loop as it processes incoming packets from the network.
The return at the end of on_message() will not do anything useful as there is nowhere to return anything to.
Your next problem is that you have to remember that MQTT is asynchronous, so trying to read the value of mqtt1 immediate after the call to client.publish() will just not work. Your code will try to read the value immediately, but you have no way of knowing how long it will take for a message to be delivered as a possible response from any client that is subscribed.
Remember that you should not think of MQTT as something like HTTP, it is not a request response synchronous model. While MQTTv5 added the concept of response topics in the headers, it did not change the fact that a response message is totally separate the request message and may arrive, arrive late, never arrive or arrive multiple times from multiple other clients.
You need to spend some time learning about asynchronous systems and look at how you maintain state under those circumstances (e.g. a state machine model)
I implemented connection to broker like:
app.py
import paho.mqtt.client as mqtt
client = mqtt.Client(client_id='my_client', clean_session=False)
my_client = MyClient(client)
try:
my_client.start()
while True:
try:
client.loop()
except Exception as e:
my_client.start()
except Exception as e:
client.loop_stop()
exit(1)
MyClient.py
class MyClient:
def __init__(self, mqtt=None):
self.mqtt = mqtt
def start(self):
self.mqtt.subscribe('some/topic')
I have part of code where I want to pause topics listening:
self.mqtt.unsubscribe('some/topic')
And later I want to subscribe back to it I want to call start() again like: self.start()
But it never subscribe again. Any idea why?
Calling start() after the exception is thrown won't work as the client is most likely not connected at that point.
You should move your subscriptions to the on_connect callback then it will always re-subscribe after the client has (re)connected
As for your original question, probably better to just set a boolean flag and use it to gate processing the message rather than unsubscribing/subscribing when you want to ignore messages.
I want to implement a Paho MQTT Python Service which is always running, receiving and sending messages. If an error occurs in any instance it should restart.
I implemented two classes which each start a threaded network loop with paho's loop_start(). These classes then have some callback functions which call other classes and so on.
For now i have a simple Python script which calls the classes and loops:
from one import one
from two import two
import time
one()
two()
while True:
if one.is_alive():
print("one is still alive")
else:
print("one died - do something!")
time.sleep(1)
And here my class "one":
import paho.mqtt.client as mqtt
import json
class one():
def __init__(self):
self.__client = mqtt.Client(client_id = "one")
self.__client.connect("localhost", 1883)
self.__client.subscribe("one")
self.__client.on_connect = self.__on_connect
self.__client.on_message = self.__on_message
self.__client.on_disconnect = self.__on_disconnect
self.__client.loop_start()
def __on_connect(self, client, userdata, flags, rc):
print("one: on_connect")
def __on_disconnect(self, client, userdata, flags, rc):
print("one: on_disconnect")
def __on_message(self, client, userdata, message):
str_message = message.payload.decode('utf-8')
message = json.loads(str_message)
print("one: on_message: " + str(message))
def is_alive(self):
return True
However - if I send a package which produces an error (a pickled message instead of json for example) my "is_alive"-function is still returning True but the paho-implementation is not responsive anymore. So no further messages are sent to on_message. So only a part of the class is still responsive!?
Class "two" is still responsive and the script is running in the "while True" still.
How do i properly check the functionality of such a class?
I think you have to build a checker method like class1.isAlive() which tells you if the class is waitng for requests. Also I think you have to build this in the while True loop and react than of failures.
Additionally, you could write your own event with a wait function. Wait is more CPU hungry but it is more responsive. See here for example. But it depends on your python version.
I'm somewhat experienced when it comes to MQTT and in Python and this has baffled me for the last hour or so.
This is the script I'm working with:
#!/usr/bin/python
import json
import socket
import paho.mqtt.client as mqtt
client = mqtt.Client()
try:
client.connect('localhost', 4444)
except:
print "ERROR: Could not connect to MQTT."
mode_msg = {
'mode': '2'
}
client.publish("set", payload=json.dumps(mode_msg), qos=2, retain=False)
This code won't run. I have no idea why. Most baffeling is, when I add " client.loop_forever()" at the bottom, it will run...
I've tried adding "client.disconnect()" at the bottom as well to get it to disconnect properly, but it all won't help. Is there something I'm missing right now?
It looks like you are trying to publish a single message, the paho client has a specific message to do just that.
#!/usr/bin/python
import paho.mqtt.publish as publish
mode_msg = {
'mode': '2'
}
publish.single("paho/test/single", payload=json.dumps(mode_msg), qos=2, hostname="localhost", port=4444)
The problem with your original code is that you are you need to run the network loop to handle the publish (and because you are publishing with qos=2 which needs to reply to the broker acknowledgement of the publish), you can do it as follows:
#!/usr/bin/python
import json
import paho.mqtt.client as mqtt
run = True
def on_publish(client, userdata, mid):
run = False;
client = mqtt.Client()
client.on_publish = on_publish
try:
client.connect('localhost', 4444)
except:
print "ERROR: Could not connect to MQTT."
mode_msg = {
'mode': '2'
}
client.publish("set", payload=json.dumps(mode_msg), qos=2, retain=False)
while run:
client.loop()
client.disconnect()
client.loop_forever() won't work because it does exactly what the name suggests, it loops forever, so would never reach your client.disconnect(). This uses the on_publish callback to break out of the loop calling client.loop() and then disconnect.
The paho.mqtt client library is built around an event loop that must run to properly process and maintain the MQTT protocol.
Thus to make things happen you need to call some of the loop() functions, as mentioned in the documentation
I've installed Mosquitto and used in the terminal without problem, also installed paho for c++, both mosquitto and paho work well.
Then installed paho for python via pip. Now i'm trying to test it with this code but the on_message and on_connect functions never get called. I'm doing
mosquitto_pub -h localhost -t "test" -m "This is a message"
while the python script runs but it doesn't work. It never gets to the line that prints "on_message!"
import paho.mqtt.client as mqtt
def on_connect(self, mqtt_client, obj, flags, rc):
mqtt_client.subscribe("test")
def on_message(self, mqtt_client, obj, msg):
print "on_message()"
print "Initializing subscriber"
mqtt_client = mqtt.Client()
mqtt_client.on_connect = on_connect
mqtt_client.on_message = on_message
mqtt_client.connect("localhost",1883)
print "Listening"
while True:
mqtt_client.loop()
What am i doing wrong?
The on_message callback gets called only if You are subscribed to a topic.
To do that, You need to call the subscribe method. For example:
print "Initializing subscriber"
mqtt_client = mqtt.Client()
mqtt_client.on_connect = on_connect
mqtt_client.on_message = on_message
mqtt_client.connect("localhost", 1883) # "localhost" or IP or
mqtt_client.subscribe("test") # topic name, same as the "-t" argument in mosquitto_pub
print "Listening"
mqtt_client.loop_forever()
As for the on_connect callback, what caused the issue for me was that I called the connack_string method as it was Client's method, but it is not. Looking at the source code, we see that it is defined directly in the module, not inside a class:
def connack_string(connack_code):
So, I replaced this:
def connect_callback(client, userdata, flags, rc):
print("Connection attempt returned: " + client.connack_string(rc)) # client !!
with this:
def connect_callback(client, userdata, flags, rc):
print("Connection attempt returned: " + mqtt.connack_string(rc)) # mqtt !!
What You can also try is to omit the -h localhost part from the mosquitto_pub command, like this:
mosquitto_pub -t "test" -m "This is a message"
Also, watch for the number of parameters that the on_connect callback receives (4). This is advice to everybody else reading this post. :)
The following is working fine for me.
I have removed the self from the callbacks and swapped the loop for mqtt_client.loop_forever()
import paho.mqtt.client as mqtt
def on_connect(mqtt_client, obj, flags, rc):
mqtt_client.subscribe("test")
def on_message(mqtt_client, obj, msg):
print "on_message()"
print "Initializing subscriber"
mqtt_client = mqtt.Client()
mqtt_client.on_connect = on_connect
mqtt_client.on_message = on_message
mqtt_client.connect("localhost",1883)
print "Listening"
mqtt_client.loop_forever()
This is an old thread but I just hit a similar problem. I had:
def on_message (client, userdata,message):
print("received message", str(message.payload.decode("utf-8")
client.loop_start()
client.subscribe(topic)
client.on_message=on_message
time.sleep(30)
client.loop_stop()
If I changed the time from 30 to 300 then on_message was never executed.
Go figure?