Using paho-mqtt and trying to have it receive queued messages. The broker I'm using is emqx 4.2.2 and this is my script:
from paho.mqtt.client import Client, MQTTv5
def on_connect(mqttc, obj, flags, rc, other):
print(" Session present: " + str(flags['session present']))
print(" Connection result: " + str(rc))
mqttc.subscribe([
('/message/1', 1)
])
def on_message(*args, **kwargs):
print("received a message")
client = Client(
client_id='test-client-id',
protocol=MQTTv5,
)
client.username_pw_set(
username="test-user-2",
password="test"
)
client.on_connect = on_connect
client.on_message = on_message
client.connect(
host='localhost',
port=1883,
keepalive=60,
clean_start=False,
)
client.loop_forever()
I now go and publish a message to the broker:
mosquitto_pub -u test-user-2 -P test -t '/message/1' -m 'this is a message' -q 1 -V mqttv5
While the client is connected to the broker, It does receive the messages but given that I'm
subscribing with QoS 1 and that messages are published with QoS 1 I am expecting that if I
disconnect my client from the broker, then publish some more QoS 1 messages to that topic and then reconnect my client to the broker using the same fixed client_id, then my client will receive the messages that have been queued while my client was away. Well that's not happening and simulating the same functionality with mosquitto_sub with the -c flag everything works as expected, which leads me to ask myself ... is there a problem with paho-mqtt? Am I doing something wrong?
In MQTT v5, clean start means only whether the session is deleted at the start or not. To control how long the session lasts after you disconnect, you need to set the session expiry interval:
import paho.mqtt.properties as properties
...
connect_properties = properties.Properties(properties.PacketTypes.CONNECT)
connect_properties.SessionExpiryInterval = 3600
client.connect("localhost", 1883, 60, properties=connect_properties)
QOS only works if the client is SUBSCRIBEd...if you want to receive messages before you are Subscribed, you need to PUBLISH messages with the Retain flag set...and then you only get the last message that was sent. If you really need to get multiple messages, then you need to use something like AMQ, and not MQTT.
It should be clean_session not clean_start and it needs to be passed to the Client constructor not the connect() function:
...
client = Client(
client_id='test-client-id',
protocol=MQTTv5,
clean_session=False
)
client.username_pw_set(
username="test-user-2",
password="test"
)
client.on_connect = on_connect
client.on_message = on_message
client.connect(
host='localhost',
port=1883,
keepalive=60
)
...
Related
I am having an issue with the code below.
The code works perfectly at the beginning. First it says Connected to MQTT Broker! and receives data from it. But after a long time (like 6 hours, or 10 hours etc.) it says again Connected to MQTT Broker! and after that id does not receive any other data.
I am trying to make this program work forever, but i don't know what i have done wrong.
Any ideas?
# python3.6
import random
import mysql.connector
from paho.mqtt import client as mqtt_client
import json
# Code for MQTT Connection
broker = 'YOUR_BROKER'
port = 1883
topic = "YOUR_TOPIC"
# generate client ID with pub prefix randomly
client_id = f'python-mqtt-{random.randint(0, 100)}'
username = "THE_USERNAME"
password = "THE_PASSWORD"
# Function to connect on mqtt
def connect_mqtt() -> mqtt_client:
def on_connect(client, userdata, flags, rc):
if rc == 0:
print("Connected to MQTT Broker!")
else:
print("Failed to connect, return code %d\n", rc)
client = mqtt_client.Client(client_id)
client.username_pw_set(username, password)
client.on_connect = on_connect
client.connect(broker, port)
return client
# function to subscribe from mqtt
def subscribeFunc(client: mqtt_client):
def on_messageFunc(client, userdata, msg):
print(f"Received `{msg.payload.decode()}` from `{msg.topic}` topic")
client.subscribe(topic)
client.on_message = on_messageFunc
def run():
client = connect_mqtt()
subscribeFunc(client)
client.loop_forever()
if __name__ == '__main__':
run()
I tried to find the problem but it seems that nothing changed significantly.
I am expecting this program to receive data without stopping.
Network connections may not be 100% reliable (and servers etc are restarted from time to time) so expecting the connection to remain up forever is unrealistic. The issue with your code is that it connects then subscribes; if the connection is dropped it does not resubscribe. As the connection is clean_session=true and subscription qos=0 (the defaults) the broker will forget about any active subscriptions when the connection drops (so the client will reconnect but not receive any more messages).
The Simple solution is to use the approach shown in the docs and subscribe in the on_connect callback (that way the subscription will be renewed after reconnection):
def on_connect(client, userdata, flags, rc):
# Subscribing in on_connect() means that if we lose the connection and
# reconnect then subscriptions will be renewed.
client.subscribe("$SYS/#")
client = mqtt.Client()
client.on_connect = on_connect
client.connect("mqtt.eclipseprojects.io", 1883, 60)
You may also want to consider the advice in this answer (as per the comment from #mnikley) because that way the broker will queue up messages for you while the connection is down (otherwise these will be lost).
I am working with a device that publishes to the topic test/123, where 123 is the name of the device. I need to subscribe to that topic (and processes received messages); in addition I also need to send a word to the same topic (test/123). The device only looks at this topic.
How can I distinguish between incoming and outgoing by content? More precisely, how to send correctly. In the on_message method you need to do this or you need to create another method, but then how to receive incoming messages there. From the incoming messages I need to get the name of the device and then work with it.
import paho.mqtt.client as mqtt
import paho.mqtt.publish as publish
# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, flags, rc):
print("Connected with result code "+str(rc))
# Subscribing in on_connect() means that if we lose the connection and
# reconnect then subscriptions will be renewed.
client.subscribe("/test/#")
# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
print(msg.topic+" "+str(msg.payload))
imei = msg.topic.split('test/')[1]
data = msg.payload.decode()
print(imei)
print(data)
publish(imei)
def publish(imei):
client = mqtt.Client()
user = 'test'
passw = '1111'
client.username_pw_set(user,passw)
client.connect("localhost",1883)
topic = '/test/'+ imei
client.publish(topic,'hello')
print('SEND')
client.disconnect()
client = mqtt.Client()
user = 'test'
passw = '1111'
client.username_pw_set(user,passw)
client.on_connect = on_connect
client.on_message = on_message
client.connect("localhost", 1883, 60)
client.loop_forever()
MQTT does not differentiate between clients in any way, that means if a client subscribes to a given topic it will receive ALL messages on that topic, including the ones it publishes it's self. So with your current design you will always get the message you publish in response to the first message back and this will trigger re-sending that message.
MQTT messages do NOT contain any information about who published the message unless you choose to add it to the payload, so you have no way to identify the incoming message as being the one you just published and this will cause a message loop storm.
The CORRECT solution is to not use the same topic for the 2 messages.
MQTT v5 has a flag that can be passed as part of establishing the connection
which prevents messages being returned to the client that published them. At this time it does not appear that the Paho Python library has a way to set this flag.
If you are using MQTT v3.1.1 and the mosquitto or RSMB MQTT broker then there is an undocumented option (this is not part of the MQTT spec) that can be set which will also prevent messages being returned. The following code will ONLY work with the 2 brokers I have mentioned.
import paho.mqtt.client as mqtt
# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, flags, rc):
print("Connected with result code "+str(rc))
# Subscribing in on_connect() means that if we lose the connection and
# reconnect then subscriptions will be renewed.
client.subscribe("test/#")
# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
print(msg.topic+" "+str(msg.payload))
imei = msg.topic.split('test/')[1]
data = msg.payload.decode()
print(imei)
print(data)
publish(client, imei)
def publish(client,imei):
topic = 'test/'+ imei
client.publish(topic,'hello')
print('SEND')
client = mqtt.Client()
user = 'test'
passw = '1111'
client.username_pw_set(user,passw)
client.on_connect = on_connect
client.on_message = on_message
client.enable_bridge_mode()
client.connect("localhost", 1883, 60)
client.loop_forever()
p.s. do not start topics with a leading / while legal according to the spec, it will break things like shared subscriptions and adds an extra null to the start of the topic tree.
I'm trying to connect from python script(paho.mqtt.python) to mosquitto broker.
I can connect from terminal using this command:
mosquitto_sub -h localhost -p 8883 -v -t 'owntracks/#' -u owntracks -P 12qwaszx
But when I'm trying to connect via python script I'm getting the error:
Socket error on client <unknown>, disconnecting.
The script I'm using is the example:
(from here: https://owntracks.org/booklet/tech/program/)
import paho.mqtt.client as mqtt
import json
# The callback for when the client successfully connects to the broker
def on_connect(client, userdata, rc):
''' We subscribe on_connect() so that if we lose the connection
and reconnect, subscriptions will be renewed. '''
client.subscribe("owntracks/+/+")
#tried also: client.subscribe("owntracks/#")
# The callback for when a PUBLISH message is received from the broker
def on_message(client, userdata, msg):
topic = msg.topic
try:
data = json.loads(str(msg.payload))
print "TID = {0} is currently at {1}, {2}".format(data['tid'], data['lat'], data['lon'])
except:
print "Cannot decode data on topic {0}".format(topic)
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect("localhost", 8883, 60)
# Blocking call which processes all network traffic and dispatches
# callbacks (see on_*() above). It also handles reconnecting.
client.loop_forever()
Here is the content of my config file (I changed the "localhost" from my real IP - tried both of them):
# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example
pid_file /var/run/mosquitto.pid
log_dest file /var/log/mosquitto/mosquitto.log
include_dir /etc/mosquitto/conf.d
listener 8883 "localhost"
persistence true
persistence_location /var/lib/mosquitto/
persistence_file mosquitto.db
log_dest syslog
log_dest stdout
log_dest topic
log_type error
log_type warning
log_type notice
log_type information
connection_messages true
log_timestamp true
allow_anonymous false
password_file /etc/mosquitto/pwfile
Any help would be highly appreciated.
Your python script is attempting to connect to what looks like a TLS secured setup without preparing the client connect method to apply these details to the transaction. Try the following:
def ssl_prep():
ssl_context = ssl.create_default_context()
ssl_context.load_verify_locations(cafile=ca)
ssl_context.load_cert_chain(certfile=mycert, keyfile=priv)
return ssl_context
ca = "PATH_TO_YOUR_CA_FILE"
priv = "PATH_TO_YOUR_PEM_FILE"
mycert = "PATH_TO_YOUR_CERT_FILE"
topics = "YOUR_TOPICS"
broker = "BROKER_URL"
client = mqtt.Client()
ssl_context= ssl_prep()
client.tls_set_context(context=ssl_context)
client.username_pw_set(username="UNAME",password="PASS")
client.connect(broker, port=8883)
By providing the ssl context to the connection attempt before the attempt is made, this should connect assuming you have all the detail specific to your own setup in place.
Try to reduce the KeepAlive time from 1min to 30sec or lower:
client.connect("localhost", 8883, 30)
//Default: connect(host, port=1883, keepalive=60, bind_address=””)
I have configured MQTT Broker, receiving published messages from another piece of code (not written or accessible by me). I added another topic to the broker configuration and am now trying to publish data to this new topic from a piece of python code. I get the feedback that the message is published by the callback function, but no actual data is received.
Is there anything I am missing?
I am using the following code:
import paho.mqtt.client as mqtt
import time
#=========================================================================
def on_connect(client, userdata, flags, rc) :
print "on_connect()"
#=========================================================================
def on_publish(client, userdata, mid) :
print "on_publish()"
#=========================================================================
def send() :
mqttc = mqtt.Client()
mqttc.on_connect = on_connect
mqttc.on_publish = on_publish
#host = "localhost"
host = "127.0.0.1"
port = 1883
keepalive = 60
print "\nConnect to '%s', port '%s', keepalive '%s'" % (host, port, keepalive)
mqttc.connect(host=host, port=port, keepalive=keepalive)
time.sleep(3)
mqttc.loop_start()
time.sleep(3)
topic = "data/MY/TOPIC"
msg = "MY_MESSAGE"
print "Publish to '%s' msg '%s'" % (topic, msg)
mqttc.publish(topic, msg, qos=2)
time.sleep(3)
mqttc.loop_stop()
# end send()
#=========================================================================
if __name__ == "__main__":
send()
# end if
Getting the stdout
Connect to '127.0.0.1', port '1883', keepalive '60'
on_connect()
Publish to 'data/MY/TOPIC' msg 'MY MESSAGE'
on_publish()
I am not sure if the loop() functions are necessary, but if I do not embed the publishing in the loop_start() and loop_stop(), I do not get a on_connect callback.
The loop functions are necessary as these are where all the network traffic is processed.
Manually setting up a connection to the broker to just send a single message like this is not a good idea, it would be better to start the client, leave it running (by calling loop_start() and not calling loop_stop()) in the background and then just call the publish method on the mqttc client object.
If you don't want to keep a instance of the client running then you should use the single message publish helper method provided by the paho python library (https://pypi.python.org/pypi/paho-mqtt/1.1#single):
import paho.mqtt.publish as publish
publish.single("paho/test/single", "payload", hostname="iot.eclipse.org")
I have tested the example program on paho-mqtt, and I know that the function loop_forever() can handles reconnecting. But my question is that, although loop_forever() can reconnect, it cannot re-subscribe. It should be a problem when the server suddenly crashes, in this case, the client is still listening, but when the server is restarted, the client can reconnect, but cannot subscribe message any more. I think maybe I should re-write the loop_forever() function, but I am not sure if I were right, and how to do it.
import sys
try:
import paho.mqtt.client as mqtt
except ImportError:
# This part is only required to run the example from within the examples
# directory when the module itself is not installed.
#
# If you have the module installed, just use "import paho.mqtt.client"
import os
import inspect
cmd_subfolder = os.path.realpath(os.path.abspath(os.path.join(os.path.split(inspect.getfile( inspect.currentframe() ))[0],"../src")))
if cmd_subfolder not in sys.path:
sys.path.insert(0, cmd_subfolder)
import paho.mqtt.client as mqtt
def on_connect(mqttc, obj, flags, rc):
print("rc: "+str(rc))
def on_message(mqttc, obj, msg):
print(msg.topic+" "+str(msg.qos)+" "+str(msg.payload))
def on_publish(mqttc, obj, mid):
print("mid: "+str(mid))
def on_subscribe(mqttc, obj, mid, granted_qos):
print("Subscribed: "+str(mid)+" "+str(granted_qos))
def on_log(mqttc, obj, level, string):
print(string)
# If you want to use a specific client id, use
# mqttc = mqtt.Client("client-id")
# but note that the client id must be unique on the broker. Leaving the client
# id parameter empty will generate a random id for you.
mqttc = mqtt.Client()
mqttc.on_message = on_message
mqttc.on_connect = on_connect
mqttc.on_publish = on_publish
mqttc.on_subscribe = on_subscribe
# Uncomment to enable debug messages
#mqttc.on_log = on_log
mqttc.connect("m2m.eclipse.org", 1883, 60)
mqttc.subscribe("$SYS/#", 0)
mqttc.loop_forever()
The easy way to deal with this is to do your subscribing in the on_connect callback, then when you reconnect all the subscriptions will be restored as well.
While instantiating your mqtt client you can set the "clean session" flag to false.
mqttc = mqtt.Client(clean_session=False)
Citation from the mosquitto manual:
Clean session / Durable connections
On connection, a client sets the "clean session" flag, which is sometimes also known as the "clean start" flag. If clean session is set to false, then the connection is treated as durable. This means that when the client disconnects, any subscriptions it has will remain and any subsequent QoS 1 or 2 messages will be stored until it connects again in the future. If clean session is true, then all subscriptions will be removed for the client when it disconnects.