We Keep Coding

MQTT Subscriber Failed to Subscribe to the Same Topic With MQTT Publisher

I'm trying to make a simple "transit schedule changes" program using python MQTT where the publisher can input flight number (will be used as topic) and new flight time, while the transit location will be picked random from given list.
The subscriber will have to input flight number too which will be used as topic. But in my codes, it looks like the subscriber failed to get the message published to the same topic because it keeps on printing Connected Successfully (I'm using client.loop_forever()). Can someone please help me to figure out what's wrong with my code?
This is my first time asking question, so please ask me if something isn't clear from my explanation. Thank you so much :)
Publisher:
import paho.mqtt.client as mqtt
import time
from datetime import datetime, date
import random
def on_connect(client, userdata, flags, rc):
if (rc==0):
global connected
connected = True
#print("Successfully Connected.")
client.on_publish = on_publish
else:
print("Failed to connect.")
def on_publish(client, userdata, mid):
print("Published successfully. MID: "+str(mid))
listTransit = ["Singapura", "Qatar", "Korea Selatan", "Turki", "Republik Tiongkok",
"Amerika Serikat", "Jepang", "Uni Emirat Arab", "Oman", "Islandia"]
broker_address="broker.emqx.io"
client = mqtt.Client("Publisher")
client.on_connect = on_connect
client.connect(broker_address, port=1883)
client.loop_start()
topic = input("Masukkan nomor penerbangan: ")
negaraTujuan = input("negara tujuan: ")
print("Masukkan waktu penerbangan baru (Format: [jam::menit::detik])")
str_time = input()
Date = date.today()
Time = datetime.strptime(str_time, '%H::%M::%S').time()
Location = random.randrange(0,len(listTransit))
if (listTransit[Location] != negaraTujuan):
message = Date.strftime("%Y/%m/%d")+"\nTujuan: "+negaraTujuan+"\nLokasi Transit : "+listTransit[Location]+"\nJam terbang : "+Time.strftime("%H:%M:%S")
client.publish(topic, message)
print("Topic: ",topic)
print(message)
else:
while listTransit[Location] == negaraTujuan:
Location = random.randrange(0,len(listTransit))
message = Date.strftime("%Y/%m/%d")+"\nTujuan: "+negaraTujuan+"\nLokasi Transit : "+listTransit[Location]+"\nJam terbang : "+Time.strftime("%H:%M:%S")
client.publish(topic, message)
print(message)
client.loop_stop()
Subscriber:
import paho.mqtt.client as mqtt
import time
from datetime import datetime, datetime
import re
def on_connect(client, userdata, flags, rc):
if (rc == 0):
print("Connected successfully.")
#global topic
#topic = input("Masukkan nomor penerbangan anda: ")
#client.subscribe(topic)
else:
print("Connection failed.")
def on_message(client, userdata, msg):
print("on_message callback function activated.")
sched = str(msg.payload.decode("utf-8"))
print(sched)
def on_subscribe(client, userdata, mid, granted_qos):
print("Subscribed to "+topic+" successfully")
broker_address="broker.emqx.io"
topic = input("Masukkan nomor penerbangan anda: ")
negaraTujuan = input("negara tujuan: ")
client = mqtt.Client("Subscriber")
client.subscribe(topic)
client.on_connect = on_connect
client.on_message = on_message
client.on_subscribe = on_subscribe
client.connect(broker_address, port=1883)
client.loop_forever()
what I got from running both codes are
Masukkan nomor penerbangan: YT05TA
negara tujuan: Australia
Masukkan waktu penerbangan baru (Format: [jam::menit::detik])
12::50::00
Topic: YT05TA
2023/01/03
Tujuan: Australia
Lokasi Transit : Amerika Serikat
Jam terbang : 12:50:00
Published successfully. MID: 1
from Publisher
and
Masukkan nomor penerbangan anda: YT05TA
negara tujuan: Australia
Connected successfully.
Connected successfully.
Connected successfully.
Connected successfully.
from subscriber. It doesn't even print the print("on_message callback function activated.")

You are using a public broker, so that means it is likely to have LOTS of over clients.
Every client MUST have a UNIQUE Client ID, so using Publisher and Subscriber is very likely to clash with other clients.
The MQTT specification says the broker must disconnect the currently connected client when a new client connects with the same Client ID. As most client libraries will try and reconnect when they are disconnects this leads to a battle between the two clients to stay connected.
Change them both to random values

Live Location Tracking using Python

I am using an app called cedalo connect to get my phone's GPS data using an online mqtt server. I constantly get latitude and longitude information using this app.
I want to show it on a map using python. I wrote such a code but it kept giving me errors like (AttributeError: module 'folium' has no attribute 'MarkerCluster')
I honestly don't know what else to use.
Could you please tell me what I can do about it?
what I tried:
import json
import paho.mqtt.client as mqtt
import folium
import time
from folium.plugins import MarkerCluster
# Variable to hold the current latitude and longitude
latitude = 50.780036278929614
longitude = 6.10363592985153
# Callback function when a message is received
def on_message(client, userdata, message):
global latitude, longitude
# Convert the message payload from bytes to string
data = message.payload.decode()
# parse the json string
data = json.loads(data)
# Extract the latitude and longitude
if 'GPS' in data:
gps_data = data["GPS"]
latitude = gps_data["Latitude"]
longitude = gps_data["Longitude"]
# Create MQTT client
client = mqtt.Client()
# Attach callback function to the message event
client.on_message = on_message
# Connect to the MQTT broker
client.connect("broker.mqttdashboard.com", 1883)
# Subscribe to the desired topic
client.subscribe("gpsdata")
# Start the MQTT loop
client.loop_start()
# Initialize the map
# Initialize the map
m = folium.Map(location=[latitude, longitude], zoom_start=15)
mc = folium.MarkerCluster()
m.add_child(mc)
marker = None
while True:
# Check if the latitude and longitude have been updated
if latitude and longitude:
# Remove the previous marker
if marker:
mc.remove_child(marker)
# Update the marker on the map to the current location
marker = folium.Marker(location=[latitude, longitude])
mc.add_child(marker)
# Save the map
m.save("current_location.html")
# Wait for one second
time.sleep(1)
The result is an attribute error.

The MarkerCluster class is located in folium.plugins. So, instead of
mc = folium.MarkerCluster()
try
mc = MarkerCluster()
or
mc = folium.plugins.MarkerCluster()

How to run multile device under same paho-mqtt script

I am writing a script which will log more than one device with different credentials using paho-mqtt. All the client is running in the same address and with the same port. If I change the username and pass then I get the different feeds depending upon the credentials. It's working fine if I write for different devices different script. But I want to log all the devices in on_connet. I have written the script but it's working only for one device. Here is the script:
import paho.mqtt.client as mqtt
import time, json, threading, logging,ssl
clients=[
{"ursername":"username1","password": 'password1'},
{"ursername":"username2","password":'password2'},
{"ursername":"username3","password":'password3'}
]
nclients=len(clients)
run = True
def Create_connections():
for i in range(nclients):
t=int(time.time())
client_id = "client" + str(t)
client = mqtt.Client(client_id)
username = credentials[i]["ursername"]
password = credentials[i]["password"]
client.on_log=on_log
client.on_connect = on_connect
client.on_subscribe=on_subscribe
client.on_message = on_message
client.on_disconnect = on_disconnect
print("connecting to broker")
client.tls_set("CXXXXX.crt", tls_version=ssl.PROTOCOL_TLSv1_2)
client.tls_insecure_set(True)
client.username_pw_set(username, password)
client.loop_start()
client.connect("XXXXX", XXXX, XX)
print("Loop pass ")
def on_log(client, userdata, level, buf):
print("message:" + str(buf))
print("userdata:" + str(userdata))
def on_message(client, userdata, message):
msg="message received",str(message.payload.decode("utf-8"))
print(msg)
def on_connect(client, userdata, flags, rc):
print("Connected with result code:"+str(rc))
client.subscribe('v3/+/devices/+/up')
def on_disconnect(client, userdata, rc):
pass
def on_publish(client, userdata, mid):
print("mid: " + str(mid) + '\n')
def on_subscribe(mosq, obj, mid, granted_qos):
print("Subscribed: " + str(mid) + " " + str(granted_qos))
mqtt.Client.connected_flag=False
no_threads=threading.active_count()
print("current threads =",no_threads)
print("Creating Connections ",nclients," clients")
Create_connections()
Response
current threads = 1
Creating Connections 2 clients
____________________________________________________________________________
client01597648398
<paho.mqtt.client.Client object at 0x7f73c0a5b1d0>
username1
connecting to broker
message:Sending CONNECT (u1, p1, wr0, wq0, wf0, c1, k60) client_id=b'client01597648398'
userdata:None
message:Received CONNACK (0, 0)
userdata:None
Connected with result code:0
message:Sending SUBSCRIBE (d0, m1) [(b'XX/+/devices/+/XX', 0)]
userdata:None
message:Received SUBACK
userdata:None
Subscribed: 1 (0,)
message:Sending PINGREQ
userdata:None
message:Received PINGRESP
userdata:None
Any help will be highly appreciated. Thanks in advance

Your problem lies here:
while run:
client.loop_forever()
loop_forever() is a blocking call which will only return when the associated client is disconnected:
This is a blocking form of the network loop and will not return until the client calls disconnect(). It automatically handles reconnecting. paho-mqtt
So your other clients never get initialized/connected. You might want to use loop_start instead - this will use a separate thread to handle communication for every client:
def Create_connections():
for i in range(nclients):
# ...
client.loop_start()
client.connect("XXXXXXXXXX", XXXX, XX)
# ...
Create_connections()
while run:
pass

Why does the paho python client publish short messages but not long ones

This works :
while True:
print('')
command_input = input()
if command_input == 'q':
break
mcp = Mqtt_command_publisher
mcp.publish_command(device_ids, command_input)
But this does not:
class Mqtt_command_bl:
def update_minutes_to_run_at(json):
if not json['device_ids']:
return 'Request must contain device ids'
device_ids = json['device_ids']
minutes_to_run_at = json['minutes_to_run_at']
minutes_to_run_at_command_section = ''
for i in minutes_to_run_at:
m = '"{}",'.format(i)
if i == minutes_to_run_at[len(minutes_to_run_at) - 1]:
m = '"{}"'.format(i)
minutes_to_run_at_command_section += m
#command_input = 'jq \'.+{{minutes_to_run_at:[{}]}}\' /home/pi/hallmonitor_lite/config.json > /home/pi/hallmonitor_lite/tmp.json && mv /home/pi/hallmonitor_lite/tmp.json /home/pi/hallmonitor_lite/new_config.json'.format(minutes_to_run_at_command_section)
command_input = 'mkdir /home/pi/hallmonitor_lite/hello_world'
mcp = Mqtt_command_publisher
mcp.publish_command(device_ids, command_input)
return 'Success'
The class they both call:
class Mqtt_command_publisher:
def publish_command(device_ids, command_input):
mqtt_msg = json.dumps({'device_ids':device_ids,'command':command_input})
print('\n{}'.format(mqtt_msg))
client = mqtt.Client()
client.connect('********', ****, 30)
client.publish('topic/commands', mqtt_msg)
client.disconnect()
Looking at the print statements output from the Mqtt_command_publisher, the output can be the exact same, however, only one of them will execute, and I don't see why one works and the other does not.
I tried this command for testing: mkdir /home/pi/hallmonitor_lite/hello_world
This is the receiving part:
device_id = 0
with open('/home/pi/hallmonitor_lite/config.json') as json_data_file:
data = json.load(json_data_file)
device_id = data['device_id']
def on_connect(client, userdata, flags, rc):
print("Connected with result code: " + str(rc))
client.subscribe("topic/commands")
def on_message(client, userdata, msg):
mqtt_message = msg.payload.decode()
print(mqtt_message)
ids_and_command = json.loads(mqtt_message)
if str(device_id) in ids_and_command['device_ids'] or not ids_and_command['device_ids']:
print(('Executing: {}').format(ids_and_command['command']))
os.system(ids_and_command['command'])
client = mqtt.Client()
client.connect("********", ****, 30)
client.on_connect = on_connect
client.on_message = on_message
client.loop_forever()
Any ideas?

The problem is most likely because the second set of code is creating a message bigger than will fit in a single TCP packet.
This is a problem because you are not running the client network loop so the client.publish command can only send a single packet, the rest of the message would have been sent by the network loop, but even if it was running you are calling disconnect immediately after the publish call.
The client is not meant to be spun up for a single message like that, it is meant to be started and then left running with you just calling the publish method when you want to send a message. If you don't want to do that or can't for some reason there is a specific helper class in the paho python package that will do all the heavy lifting of starting the client, sending the message and then tearing everything down nicely. The docs for the single publish are here.
import paho.mqtt.publish as publish
publish.single("paho/test/single", "payload", hostname="mqtt.eclipse.org")

How to run 2 differents loops in 2 differents threads?

I'm doing a telemetry application using Azure IoT Hub, Azure IoT SDK in Python and a raspberry pi with temperature and humidity sensors.
Humidity + Temperature sensors => Rasperry Pi => Azure IoT Hub
In my first implementation thanks azure examples, I used one loop that collect data from the temperature sensor and the humidity sensor, and send them to Azure IoT Hub in the same time every 60 second.
>>> 1 Loop every 60s = Collect data & send data of temperature and humidity
Now I would like to send them with different frequencies, I mean :
One loop will collect the data of the temperature sensor and send it to Azure IoT Hub every 60 seconds;
Whereas a second loop will collect the data of the humidity sensor and send it to Azure IoT Hub every 600 seconds.
>>> 1 Loop every 60s= Collect data & send data of temperature
>>> 2 Loop every 600s= Collect data & send data of humidity
I think the tool I need is multi-threading, but I don't understand which library or structure I have to implement in my case.
Here is the code provided by Azure, including one loop that handles temperature and humidity at the same time. Reading the data and sending to Azure every 60 seconds.
import random
import time
import sys
# Using the Python Device SDK for IoT Hub:
from iothub_client import IoTHubClient, IoTHubClientError,
IoTHubTransportProvider, IoTHubClientResult
from iothub_client import IoTHubMessage, IoTHubMessageDispositionResult,
IoTHubError, DeviceMethodReturnValue
# The device connection string to authenticate the device with your IoT hub.
CONNECTION_STRING = "{Your IoT hub device connection string}"
# Using the MQTT protocol.
PROTOCOL = IoTHubTransportProvider.MQTT
MESSAGE_TIMEOUT = 10000
# Define the JSON message to send to IoT Hub.
TEMPERATURE = 20.0
HUMIDITY = 60
MSG_TXT = "{\"temperature\": %.2f,\"humidity\": %.2f}"
def send_confirmation_callback(message, result, user_context):
print ( "IoT Hub responded to message with status: %s" % (result) )
def iothub_client_init():
# Create an IoT Hub client
client = IoTHubClient(CONNECTION_STRING, PROTOCOL)
return client
def iothub_client_telemetry_sample_run():
try:
client = iothub_client_init()
print ( "IoT Hub device sending periodic messages, press Ctrl-C to exit" )
#******************LOOP*******************************
while True:
# Build the message with simulated telemetry values.
temperature = TEMPERATURE + (random.random() * 15)
humidity = HUMIDITY + (random.random() * 20)
msg_txt_formatted = MSG_TXT % (temperature, humidity)
message = IoTHubMessage(msg_txt_formatted)
# Send the message.
print( "Sending message: %s" % message.get_string() )
client.send_event_async(message, send_confirmation_callback, None)
time.sleep(60)
except IoTHubError as iothub_error:
print ( "Unexpected error %s from IoTHub" % iothub_error )
return
except KeyboardInterrupt:
print ( "IoTHubClient sample stopped" )
if __name__ == '__main__':
print ( "IoT Hub Quickstart #1 - Simulated device" )
print ( "Press Ctrl-C to exit" )
iothub_client_telemetry_sample_run()
I would like to use the same structure of functions, including two loops that handles temperature and humidity, one every 60s and one every 600s.
while True:
# Build the message with simulated telemetry values.
temperature = TEMPERATURE + (random.random() * 15)
msg_txt_formatted1 = MSG_TXT1 % (temperature)
message1 = IoTHubMessage(msg_txt_formatted1)
# Send the message.
print( "Sending message: %s" % message1.get_string() )
client.send_event_async(message1, send_confirmation_callback, None)
time.sleep(60)
while True:
# Build the message with simulated telemetry values.
humidity = HUMIDITY + (random.random() * 20)
msg_txt_formatted2 = MSG_TXT2 % (humidity)
message2 = IoTHubMessage(msg_txt_formatted2)
# Send the message.
print( "Sending message: %s" % message2.get_string() )
client.send_event_async(message2, send_confirmation_callback, None)
time.sleep(600)
How can I do that? How to call those loops with multi-threading or another method?

It may be simpler to do something like
while True:
loop_b()
for _ in range(10):
loop_a()
time.sleep(60)
or even
while True:
time.sleep(1)
now = time.time()
if now % 60 == 0:
loop_a()
if now % 600 == 0:
loop_b()
But if you really want to use threads, then:
import threading
class LoopAThread(threading.Thread):
def run(self):
loop_a()
class LoopBThread(threading.Thread):
def run(self):
loop_b()
...
thread_a = LoopAThread()
thread_b = LoopBThread()
thread_a.start()
thread_b.start()
thread_a.join()
thread_b.join()

Here are two competing approaches to consider
Don't bother with threads at all. Just have one loop that sleeps every 60 seconds like you have now. Keep track of the last time you sent humidity data. If 600 seconds has passed, then send it. Otherwise, skip it and go to sleep for 60 seconds. Something like this:
from datetime import datetime, timedelta
def iothub_client_telemetry_sample_run():
last_humidity_run = None
humidity_period = timedelta(seconds=600)
client = iothub_client_init()
while True:
now = datetime.now()
send_temperature_data(client)
if not last_humidity_run or now - last_humidity_run >= humidity_period:
send_humidity_data(client)
last_humidity_run = now
time.sleep(60)
Rename iothub_client_telemetry_sample_run to temperature_thread_func or something like it. Create a separate function that looks just like it for humidity. Spawn two threads from the main function of your program. Set them to daemon mode so they shutdown when the user exits
from threading import Thread
def temperature_thread_func():
client = iothub_client_init()
while True:
send_temperature_data(client)
time.sleep(60)
def humidity_thread_func():
client = iothub_client_init()
while True:
send_humidity_data(client)
time.sleep(600)
if __name__ == '__main__':
temp_thread = Thread(target=temperature_thread_func)
temp_thread.daemon = True
humidity_thread = Thread(target=humidity_thread_func)
humidity_thread.daemon = True
input('Polling for data. Press a key to exit')
Notes:
If you decide to use threads, consider using an
event
to terminate them cleanly.
time.sleep is not a precise way to keep
time. You might need a different timing mechanism if the samples need
to be taken at precise moments.