I'm somewhat experienced when it comes to MQTT and in Python and this has baffled me for the last hour or so.
This is the script I'm working with:
#!/usr/bin/python
import json
import socket
import paho.mqtt.client as mqtt
client = mqtt.Client()
try:
client.connect('localhost', 4444)
except:
print "ERROR: Could not connect to MQTT."
mode_msg = {
'mode': '2'
}
client.publish("set", payload=json.dumps(mode_msg), qos=2, retain=False)
This code won't run. I have no idea why. Most baffeling is, when I add " client.loop_forever()" at the bottom, it will run...
I've tried adding "client.disconnect()" at the bottom as well to get it to disconnect properly, but it all won't help. Is there something I'm missing right now?
It looks like you are trying to publish a single message, the paho client has a specific message to do just that.
#!/usr/bin/python
import paho.mqtt.publish as publish
mode_msg = {
'mode': '2'
}
publish.single("paho/test/single", payload=json.dumps(mode_msg), qos=2, hostname="localhost", port=4444)
The problem with your original code is that you are you need to run the network loop to handle the publish (and because you are publishing with qos=2 which needs to reply to the broker acknowledgement of the publish), you can do it as follows:
#!/usr/bin/python
import json
import paho.mqtt.client as mqtt
run = True
def on_publish(client, userdata, mid):
run = False;
client = mqtt.Client()
client.on_publish = on_publish
try:
client.connect('localhost', 4444)
except:
print "ERROR: Could not connect to MQTT."
mode_msg = {
'mode': '2'
}
client.publish("set", payload=json.dumps(mode_msg), qos=2, retain=False)
while run:
client.loop()
client.disconnect()
client.loop_forever() won't work because it does exactly what the name suggests, it loops forever, so would never reach your client.disconnect(). This uses the on_publish callback to break out of the loop calling client.loop() and then disconnect.
The paho.mqtt client library is built around an event loop that must run to properly process and maintain the MQTT protocol.
Thus to make things happen you need to call some of the loop() functions, as mentioned in the documentation
Related
My script reads data from MQTT server and writes it to postgres table.
I'm using loop_forever().
The program is supposed to run nonstop.
When the first connection is received everything works fine, but after some time (from minutes to days) on_connect() is called again. The program works (in the meaning that there is no error in connection) but no meassages are received any more.
In order to debug I tried following:
induce disconnect by switching off and on the network connection
shutting off and on the server
calling client.disconnect()
To my surprise first and second thing did nothing - there was no logs about new connection and the running program just kept running after the connection revived.
The third attempt was unsuccesfull, I couldn't make it work.
Other remarks:
I tried using loop_start() instead of loop_forever but was not succesfull with that at all
So basically the questions are:
how to counter-act ?
how to disconnect manually to replicate the problem of calling on_connect (and loosing incoming data)
My code:
import json
import sys
from paho.mqtt import client as mqtt_client
import psycopg2
import logging as log
from datetime import datetime
import certifi
from collections import defaultdict
def connect_mqtt(userdict) -> mqtt_client:
def on_connect(client, userdata, flags, rc):
log.info(f"{datetime.now()}: Trying connect")
if rc == 0:
log.info(f"{datetime.now()}: Connection returned result: " + mqtt_client.connack_string(rc))
else:
log.info("Failed to connect, return code %d\n", rc)
client = mqtt_client.Client(client_id=conf['client_id'], protocol=mqtt_client.MQTTv31, userdata=userdict)
client.tls_set(certifi.where())
client.tls_insecure_set(True)
client.username_pw_set(conf['username'], conf['password'])
client.on_connect = on_connect
client.connect(conf['broker'], conf['port'])
return client
def on_message(client, userdata, msg):
now_ts_in_s = round(datetime.timestamp(datetime.now()))
now_dt_in_s = datetime.fromtimestamp(now_ts_in_s)
try:
value = float(msg.payload.decode())
data = [now_dt_in_s, value]
insert_to_psql(userdata['conn'], data)
except ValueError:
pass
def insert_to_psql(conn, data):
cursor = conn.cursor()
insert_query = "INSERT INTO data (time, value) VALUES (%s, %s) ON CONFLICT " \
"DO " \
"NOTHING;"
cursor.execute(insert_query, data)
conn.commit()
def run():
psql_conn = "postgres://postgres:blablabla"
conn = psycopg2.connect(psql_conn)
userdict = {'collected_data': defaultdict(list), 'conn': conn, 'first_conn': True}
client = connect_mqtt(userdict)
client.subscribe(conf['topic'])
client.on_message = on_message
try:
client.loop_forever()
finally:
client.disconnect()
conn.close()
if __name__ == '__main__':
with open(sys.argv[1]) as f:
conf = json.load(f)
run()
If connect is called, then disconnect is probably called before that. Could be some temporary network issue. You should configure the corresponding callback.
Note, that because of this, it's most important that you subscribe in the on_connect callback and not one time outside that. When paho disconnects and connects again, it won't resubscribe automatically. That's why subscriptions should be made in the on_connect callback.
To you question how to test this. You can run a local MQTT broker, and just shut it down after your app has connected the first time and then start it again.
Apart from that, messages won't be lost if you configure your broker accordingly. MQTT has various QOS settings for that specific purpose.
If you think your app is the problem, and its not some networking issue, you could gain a somewhat more solid setup by deploying your app multiple times and let the replicas subscribe via shared subscription. https://www.hivemq.com/blog/mqtt5-essentials-part7-shared-subscriptions/
After importing the AWSIoTMQTTClient module with
from AWSIoTPythonSDK.MQTTLib import AWSIoTMQTTClient
I went ahead and configured the MQTT client connection
myMQTTClient = AWSIoTMQTTClient("my-clientid")
myMQTTClient.configureEndpoint("123abc-ats.iot.us-east-1.amazonaws.com", 8883)
myMQTTClient.configureCredentials(ROOT_KEY, PRIVATE_KEY, CERT)
myMQTTClient.connect()
I defined helloworld function that I want to use as a callback to catch the messages from the topic as:
def helloworld(client, params, packet):
print('...topic:', packet.topic)
print('...payload:', packet.payload)
myMQTTClient.publish(topic="home/fromserver", QoS=1, payload="{'message':'hello from server'}" )
Please note that the last line in the helloworld function I publish the message back to MQTT to the "home/from-server" topic.
I added two more lines to the script and run it
myMQTTClient.subscribe("home/to-server", 1, helloworld)
while True:
time.sleep(1)
I can fetch the messages from the to-server topic. But publishing the message to from-server topic crashes with AWSIoTExceptions.publishTimeoutException
How can I publish a message back to MQTT without raising the publishTimeoutException?
I am attempting to write a python script (for Raspbian) that sends mqtt message on button pushes, and changes LED's on/off when mqtt messages are received.
I can send no worries, and my script structure on 4 RPi's is the same;
import
set variables
while true:
do stuff endlessly
I however can not get a basic script running for paho to receive in this structure.
I tried to follow the paho guide but can not adapt it to an endless while loop application.
Why can't I receive mqtt messages? I cross check with 2 terminals; mosquito sub & pub, they are definitely been sent/received elsewhere on network.
Edit;
New on_connect callback displays "Connected" every 2 seconds, but the script still doesn't receive/print mqtt messages. It still does send them when I push the button.
Attempted to fire up a different MQTT broker in Docker, made no difference.
Edit3; WORKS! :D Last I test I must not have been awake and had the subscribe line commented out in the on_connect callback.
I also removed the "P1" from the mqtt.Client thing which I copied from a tutorial.
Can happierly confirm, it sends and receives mqtt messages. Now I can integrate into my larger script. Thanks hardlib
New code is this;
import sys
sys.path.append('/usr/local/lib/python2.7/dist-packages/paho/mqtt')
import os
import time
import paho.mqtt.client as mqtt
from gpiozero import Button
from time import sleep
def on_message(client, userdata, message):
print("message received " ,str(message.payload.decode("utf-8")))
print("message topic=",message.topic)
print("message qos=",message.qos)
print("message retain flag=",message.retain)
def on_connect(client, userdata, flags, rc):
print("Connected with result code "+str(rc))
client.subscribe("button")
button = Button(25)
broker_address="192.168.1.10"
client = mqtt.Client()
client.on_message=on_message
client.on_connect=on_connect
client.connect(broker_address, 1883, 60)
#client.subscribe("button")
client.loop_start()
while 1:
if button.is_pressed:
print("button pressed")
client.publish("button","ON",1)
sleep(0.1)
sleep(0.1)
print("*")
I have tested the example program on paho-mqtt, and I know that the function loop_forever() can handles reconnecting. But my question is that, although loop_forever() can reconnect, it cannot re-subscribe. It should be a problem when the server suddenly crashes, in this case, the client is still listening, but when the server is restarted, the client can reconnect, but cannot subscribe message any more. I think maybe I should re-write the loop_forever() function, but I am not sure if I were right, and how to do it.
import sys
try:
import paho.mqtt.client as mqtt
except ImportError:
# This part is only required to run the example from within the examples
# directory when the module itself is not installed.
#
# If you have the module installed, just use "import paho.mqtt.client"
import os
import inspect
cmd_subfolder = os.path.realpath(os.path.abspath(os.path.join(os.path.split(inspect.getfile( inspect.currentframe() ))[0],"../src")))
if cmd_subfolder not in sys.path:
sys.path.insert(0, cmd_subfolder)
import paho.mqtt.client as mqtt
def on_connect(mqttc, obj, flags, rc):
print("rc: "+str(rc))
def on_message(mqttc, obj, msg):
print(msg.topic+" "+str(msg.qos)+" "+str(msg.payload))
def on_publish(mqttc, obj, mid):
print("mid: "+str(mid))
def on_subscribe(mqttc, obj, mid, granted_qos):
print("Subscribed: "+str(mid)+" "+str(granted_qos))
def on_log(mqttc, obj, level, string):
print(string)
# If you want to use a specific client id, use
# mqttc = mqtt.Client("client-id")
# but note that the client id must be unique on the broker. Leaving the client
# id parameter empty will generate a random id for you.
mqttc = mqtt.Client()
mqttc.on_message = on_message
mqttc.on_connect = on_connect
mqttc.on_publish = on_publish
mqttc.on_subscribe = on_subscribe
# Uncomment to enable debug messages
#mqttc.on_log = on_log
mqttc.connect("m2m.eclipse.org", 1883, 60)
mqttc.subscribe("$SYS/#", 0)
mqttc.loop_forever()
The easy way to deal with this is to do your subscribing in the on_connect callback, then when you reconnect all the subscriptions will be restored as well.
While instantiating your mqtt client you can set the "clean session" flag to false.
mqttc = mqtt.Client(clean_session=False)
Citation from the mosquitto manual:
Clean session / Durable connections
On connection, a client sets the "clean session" flag, which is sometimes also known as the "clean start" flag. If clean session is set to false, then the connection is treated as durable. This means that when the client disconnects, any subscriptions it has will remain and any subsequent QoS 1 or 2 messages will be stored until it connects again in the future. If clean session is true, then all subscriptions will be removed for the client when it disconnects.
This code that was suggested to handle publishing a message every 10 seconds. But how to handle reconnects if needed?
import paho.mqtt as mqtt
import time
mqttc=mqtt.Client("ioana")
mqttc.connect("127.0.0.1" 1883, 60, True)
#mqttc.subscribe("test/", 2) # <- pointless unless you include a subscribe callback
mqttc.loop_start()
while True:
mqttc.publish("test","Hello")
time.sleep(10)# sleep for 10 seconds before next call
The script is the absolute bare bones of what is needed send a MQTT message repeatedly but it will automatically reconnect if disconnected as it stands.
You can have it print a message when it is disconnected and reconnected to track this by modifying it as follows:
import paho.mqtt.client as mqtt
import time
def onDisconnect(client, userdata, rc):
print("disonnected")
def onConnect(client, userdata, rc):
print("connected")
mqttc=mqtt.Client("ioana")
mqttc.on_connect = onConnect
mqttc.on_disconnect = onDisconnect
mqttc.connect("127.0.0.1", port=1883, keepalive=60)
mqttc.loop_start()
while True:
mqttc.publish("test","Hello")
time.sleep(10)# sleep for 10 seconds before next call
EDIT:
To test. If you are using mosquitto as your broker then you will probably have the mosquitto_pub command installed, you can use this to force the python to disconnect by using the same client id.
mosquitto_pub -t test -i 'ioana' -m foo