Suppose that there is one redis-client which is subscribing to channel c1 and another redis-client publishes "data" to channel c1.
At this point, does the data published is pending in a redis-server by when the client subscribing to "c1" get the data(by calling pubsub.listen() or pubsub.get_message()) or go directly to the client subscribing to the channel c1 through a redis-server?
In other words, When a redis-client call pubsub.getMessage() or pubsub.listen(), does redis-client send a request to redis-server to get data or just get data from the local socket buffer?
When I read some documents it is saying that pubsub.get_message() use select module for a socket internally.
It seems to mean that the data subscribed already is in the client local buffer, not server.
Could you give me any suggestion?
When a client does PUBLISH, this command immediately pushes the message into all sockets that are interested in it. The message is not stored in redis. If a client wasn't subscribed at the time of publishing, he will never see the message.
I'm not familiar with python driver for redis, but judging from the names, I'd guess that listen sends a SUBSCRIBE command and get_message takes a message from the local buffer.
Redis pubsub is fire and forget.
In other words, when a publish command is sent, the message is received by online subscribers, and who wasn't subscribed/listening for messages when the publish command was sent, won't never get these messages.
Related
I am trying to implement a way to know when a specific device has stopped publishing messages to an mqtt broker and in that case send an email to myself. I want to do this to be able to know if there is a problem with the device that is publishing the messages so I can check it and turn it back on. To try to accomplish this I created a mqtt client that subscribes to the topic that the device publishes e.g. test/device_1 and then set as last will and testament for that device status/device_1 where I put as payload="Offline". Ideally, I want to be able to do this for more than 1 device, but let's assume I just want it for the simple case of one device.
I created another script that implements another client that is subscribed to the topic status/device_1 and then on the on_message function it checks if it gets the payload="Offline" and if it does get it then I send an email to myself.
This approach however doesn't work as when I turn off my device, the mqtt client that is subscribed to the topic test/device_1 keeps listening but gets no messages. In other words, it doesn't send its last will even when the topic is empty. However, it seems to work when I stop the script that is subscribed to the topic test/device_1.
Am I missing something or is it not possible to accomplish what I am trying to do? Any advice is appreciated!
First LWT messages are only published by the broker if the client goes offline unexpectedly, which means if it shuts down cleanly then it will not get published, you have to manually publish a message as part of shut down process.
You can set the retained flag on the LWT so that any client that subscribes after the device has gone offline it will still see the state message.
You should also clear that message on when the device starts up, either by publishing something like Online or a null payload to clear the retained message.
You can use last timestamp when device published the messages as heartbeat and in your another script, if (current_time - last_timestamp) > 30 then you can email yourself that your device is offline. this way you can have threshold of time when you want to get email of device offline.
script 1:
mqttCient.on_message(msg):
last_timestamp = time.time()
script 2:
cur_time = time.time()
if ((cur_time - last_timestamp) > 30):
# email yourself
use Json file or database as buffer for storing and accessing last_timestamp
I am using Python based paho mqtt client to publish data to mosquitto mqtt broker.
Let's assume a scenario, when client wanted to publish message and broker got disconnected.
So python based client object buffers that message in _out_message (Ordered Dictionary), and keep retrying to send messages.
I wanted to know,
For how long mqtt client will buffer such message?
Is there any time limit or retry limit after which client will drop the message.
I wanted to dump/log such messages.
According to Eclipse Paho Python documentation, you can set the maximum number of outgoing messages with Quality of Service greater than 0 (QoS > 0) that can be pending in the outgoing message queue with the method:
max_queued_messages_set(self, queue_size)
It seems that using the default value (0) all the messages are kept until the MQTT client is able to send them. So, in the end, I suppose that messages are kept until the Python process reach the memory limit imposed by the operating system.
You can force the MQTT client to discard messages using the method reinitialise.
reinitialise(client_id="", clean_session=True, userdata=None)
In a web application with MQTT in python (using paho-mqtt lib) I would like to know if there is a way to get the broker status in real time, because the only way that i find is to store the variable "rc" into the method on_connect but it's more like a client/connection state.
EDIT 1 : after reading mosquitto broker documentation, i found that you can subscribe to '$SYS/broker/connection/#' which is supposed to give you back 1 if the connection is up and 0 if it goes down. However when i do :
subscribe.callback(self.message_callback, '$SYS/broker/connection/#', port = port, hostname=broker, auth=authentication, protocol=client.MQTTv31, tls=TLS)
impossible to get payload and topic this message although i'm doing exactly the same command to get messages from my sensors except that the topic is '#' and it's working perfectly.
Does anyone knows why ?
There is no way to poll the state of the connection to the broker from the client.
The on_disconnect callback should be called when the connection to the broker is dropped.
This should be kicked off when the keep alive times out, but also as the result of a failure to publish (if you try to publish data before the timeout expires).
Also the rc from a call to the publish command will indicate if the connection has dropped.
I am trying to send a message using publish.single and then receive it and act upon the data received. Hence, I can't proceed unless I receive something from the server, so is there a way to write a statement that will wait for a message from MQTT before proceeding?
Example code:
publish.single("$topic", data, ip_address)
#can't do anything here
receive(data_from_broker) #or anythin that looks like it!
#continue with the program here
The short answer is you don't.
At least not in the way you describe. MQTT is an asynchronous protocol, there is no sense of sending a message and waiting for a response, a publishing client has no way to know if the is 0, 1 or n subscribing clients listening on the topic the message is published.
You will need to build something called a state machine to keep track of where in the program flow you are when messages are received.
e.g.
Application published message and sets flag in the state machine to say the message was sent
Remote client receives message and publishes a response
New message is received by the first client, it checks the state machine to determine that the message should be treated as a response to the original message.
To subscribe to a topic you will have to move on from using publish.single to the full MQTT client so you can setup the onMessage callback to handle the incoming messages.
In the tornado websocket chat example, participants are stored in the set (link) , it is convenient in the case of a single server. But if run multiple instances of the application and nginx as a load balancer, how in this case, better to store the participants?
You may consider using the pubsub feature of Redis.
(link)
Edit :
When your clients log on to your chatroom, they can subscribe to a channel, say chatroom. It does not matter which tornado instances they are using.
Using this module you can keep asynchronously listening to the channel.
If another client sends a message to your chatroom (that is publish a message to the channel chatroom), all tornado instances will automatically sends the messages to those who subscribed to the channel. Then you can send the message via websocket.
You can take a look at this demo for the example.