We Keep Coding

Mqtt Subscriber, loop_forever() does not work

I am having an issue with the code below.
The code works perfectly at the beginning. First it says Connected to MQTT Broker! and receives data from it. But after a long time (like 6 hours, or 10 hours etc.) it says again Connected to MQTT Broker! and after that id does not receive any other data.
I am trying to make this program work forever, but i don't know what i have done wrong.
Any ideas?
# python3.6
import random
import mysql.connector
from paho.mqtt import client as mqtt_client
import json
# Code for MQTT Connection
broker = 'YOUR_BROKER'
port = 1883
topic = "YOUR_TOPIC"
# generate client ID with pub prefix randomly
client_id = f'python-mqtt-{random.randint(0, 100)}'
username = "THE_USERNAME"
password = "THE_PASSWORD"
# Function to connect on mqtt
def connect_mqtt() -> mqtt_client:
def on_connect(client, userdata, flags, rc):
if rc == 0:
print("Connected to MQTT Broker!")
else:
print("Failed to connect, return code %d\n", rc)
client = mqtt_client.Client(client_id)
client.username_pw_set(username, password)
client.on_connect = on_connect
client.connect(broker, port)
return client
# function to subscribe from mqtt
def subscribeFunc(client: mqtt_client):
def on_messageFunc(client, userdata, msg):
print(f"Received `{msg.payload.decode()}` from `{msg.topic}` topic")
client.subscribe(topic)
client.on_message = on_messageFunc
def run():
client = connect_mqtt()
subscribeFunc(client)
client.loop_forever()
if __name__ == '__main__':
run()
I tried to find the problem but it seems that nothing changed significantly.
I am expecting this program to receive data without stopping.

Network connections may not be 100% reliable (and servers etc are restarted from time to time) so expecting the connection to remain up forever is unrealistic. The issue with your code is that it connects then subscribes; if the connection is dropped it does not resubscribe. As the connection is clean_session=true and subscription qos=0 (the defaults) the broker will forget about any active subscriptions when the connection drops (so the client will reconnect but not receive any more messages).
The Simple solution is to use the approach shown in the docs and subscribe in the on_connect callback (that way the subscription will be renewed after reconnection):
def on_connect(client, userdata, flags, rc):
# Subscribing in on_connect() means that if we lose the connection and
# reconnect then subscriptions will be renewed.
client.subscribe("$SYS/#")
client = mqtt.Client()
client.on_connect = on_connect
client.connect("mqtt.eclipseprojects.io", 1883, 60)
You may also want to consider the advice in this answer (as per the comment from #mnikley) because that way the broker will queue up messages for you while the connection is down (otherwise these will be lost).

MQTT Subscriber to ThingsBoard broker in python

Situation:
I have a python virtual sensor (a python program) that submits data via MQTT protocol to my device in ThingsBoard.
I can visualize data on the dashboard, so I'm sure that the data are received.
Problem:
When I try to connect a python subscriber to the thingsboard broker (demo.thingsboard.io) using paho I obtain that the connection code is 0, so the connection is OK, however I see that the dashboard stops visualising the data from the virtual sensor but the subscriber does not receive anything.
The virtual sensor is publishing at v1/devices/me/telemetry and the subscriber is subscribed at the same topic v1/devices/me/telemetry.
How to show the data published by the virtual sensor on my subscriber client?
VIRTUAL SENSOR CODE:
import paho.mqtt.client as paho #mqtt library
import os
import json
import time
from datetime import datetime
ACCESS_TOKEN="vgFztmvT6bps7JCeOEZq" #Token of your device
broker="demo.thingsboard.io" #host name
port=1883 #data listening port
def on_publish(client,userdata,result): #create function for callback
print("data published to thingsboard \n")
pass
client1= paho.Client("control1") #create client object
client1.on_publish = on_publish #assign function to callback
client1.username_pw_set(ACCESS_TOKEN) #access token from thingsboard device
client1.connect(broker,port,keepalive=60) #establish connection
while True:
payload="{"
payload+="\"Humidity\":60,";
payload+="\"Temperature\":25";
payload+="}"
ret= client1.publish("v1/devices/me/telemetry",payload) #topic-v1/devices/me/telemetry
print("Please check LATEST TELEMETRY field of your device")
print(payload);
time.sleep(5)
CLIENT SUBSCRIBER CODE:
import paho.mqtt.client as mqtt
import time
token = "vgFztmvT6bps7JCeOEZq"
broker="demo.thingsboard.io" # host name
port=1883
topic = "v1/devices/me/telemetry"
# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, flags, rc) :
if (rc==0) :
print("connected OK Returned code = ", rc)
else :
print("Bad connection Returned code = ", rc)
def on_message(client, userdata, msg) :
print (msg.topic + " " + str(msg.payload))
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.username_pw_set(token)
client.connect(broker , port, 60)
client.subscribe(topic)
client.loop_forever()

In you publisher client , you are using a topic named:"v1/devices/me/telemetry".
However, you are not subscribing to the same topic using the subscriber client.
Change the following line in your Subscriber client program:
client.subscribe(token)
to
client.subscribe(topic)
This should solve the problem.

I believe TB will only allow for one connection per client so you should use different access_keys for your publisher and subscriber.

Try to use tcp://51.159.155.114 but not demo.thingsboard.io

How to receive messages from multiple clients over mqtt?

I am sending a number of messages simultaneously from multiple clients from one python script and trying to receive them on another script. The problem I am getting is that the message is received but only from the first client that gets connected and it keeps on looping over it.
What I need to have is that I get messages from each client in the manner they are published.
import paho.mqtt.client as mqtt
import time
def on_connect(client, userdata, flags, rc):
if rc == 0:
print("Connected to broker")
global Connected
Connected = True
else:
print("Connection failed")
def on_message(client, userdata, message):
print ("Message received: " + str(message.payload) + " from " + str(client))
Connected = False
client = mqtt.Client()
client.on_connect= on_connect
client.on_message= on_message
client.connect(host)
client.loop_start()
while Connected != True:
time.sleep(0.1)
client.subscribe("test")
print("subscribed")
client.loop_stop()

You are misunderstanding what the client argument in the on_message callback is.
This value is a link to the local instance of the MQTT client that has subscribed to the topic.
MQTT messages do not carry any information about the client that published them, unless you explicitly encode it into the payload. Part of the point of a Pub/Sub protocol like MQTT is to totally decouple the information creator (publisher) from the information consumer (subscriber).
Also you should move the call to client.subscribe("test") to inside the on_connect() callback because as you have it you are trying to resubscribe to the same topic 10 times a second which will achieve absolutely nothing, except to generate unneeded load on the broker.

socket error when connecting paho.mqtt.python to mosquitto broker

I'm trying to connect from python script(paho.mqtt.python) to mosquitto broker.
I can connect from terminal using this command:
mosquitto_sub -h localhost -p 8883 -v -t 'owntracks/#' -u owntracks -P 12qwaszx
But when I'm trying to connect via python script I'm getting the error:
Socket error on client <unknown>, disconnecting.
The script I'm using is the example:
(from here: https://owntracks.org/booklet/tech/program/)
import paho.mqtt.client as mqtt
import json
# The callback for when the client successfully connects to the broker
def on_connect(client, userdata, rc):
''' We subscribe on_connect() so that if we lose the connection
and reconnect, subscriptions will be renewed. '''
client.subscribe("owntracks/+/+")
#tried also: client.subscribe("owntracks/#")
# The callback for when a PUBLISH message is received from the broker
def on_message(client, userdata, msg):
topic = msg.topic
try:
data = json.loads(str(msg.payload))
print "TID = {0} is currently at {1}, {2}".format(data['tid'], data['lat'], data['lon'])
except:
print "Cannot decode data on topic {0}".format(topic)
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect("localhost", 8883, 60)
# Blocking call which processes all network traffic and dispatches
# callbacks (see on_*() above). It also handles reconnecting.
client.loop_forever()
Here is the content of my config file (I changed the "localhost" from my real IP - tried both of them):
# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example
pid_file /var/run/mosquitto.pid
log_dest file /var/log/mosquitto/mosquitto.log
include_dir /etc/mosquitto/conf.d
listener 8883 "localhost"
persistence true
persistence_location /var/lib/mosquitto/
persistence_file mosquitto.db
log_dest syslog
log_dest stdout
log_dest topic
log_type error
log_type warning
log_type notice
log_type information
connection_messages true
log_timestamp true
allow_anonymous false
password_file /etc/mosquitto/pwfile
Any help would be highly appreciated.

Your python script is attempting to connect to what looks like a TLS secured setup without preparing the client connect method to apply these details to the transaction. Try the following:
def ssl_prep():
ssl_context = ssl.create_default_context()
ssl_context.load_verify_locations(cafile=ca)
ssl_context.load_cert_chain(certfile=mycert, keyfile=priv)
return ssl_context
ca = "PATH_TO_YOUR_CA_FILE"
priv = "PATH_TO_YOUR_PEM_FILE"
mycert = "PATH_TO_YOUR_CERT_FILE"
topics = "YOUR_TOPICS"
broker = "BROKER_URL"
client = mqtt.Client()
ssl_context= ssl_prep()
client.tls_set_context(context=ssl_context)
client.username_pw_set(username="UNAME",password="PASS")
client.connect(broker, port=8883)
By providing the ssl context to the connection attempt before the attempt is made, this should connect assuming you have all the detail specific to your own setup in place.

Try to reduce the KeepAlive time from 1min to 30sec or lower:
client.connect("localhost", 8883, 30)
//Default: connect(host, port=1883, keepalive=60, bind_address=””)

Send python input through mosquitto

Here is the code I'm running for my client. It works pretty well, but it doesn't allow python inputs to be made. I considered making another .py-file for typing and sending messages, but I'm not sure how to import the established connection.
Is it somehow possible to enable a python input chat using mqtt?
import paho.mqtt.client as mqtt
# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, flags, rc):
print("Connected with result code "+str(rc))
# Subscribing in on_connect() means that if we lose the connection and
# reconnect then subscriptions will be renewed.
client.subscribe("hello/world")
# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
print(msg.topic+"| "+ userdate + " said: "+str(msg.payload))
id = raw_input('username: ')
client = mqtt.Client(id)
client.on_connect = on_connect
client.on_message = on_message
client.connect_async("192.168.0.24", 1883, 60)
# Blocking call that processes network traffic, dispatches callbacks and
# handles reconnecting.
# Other loop*() functions are available that give a threaded interface and a
# manual interface.
client.loop_forever()