Currently developing something like "smart home" and I have few different devices in my home. All of them connected to OpenHab via MQTT. I'm using Paho MQTT library (Python) for my purposes.
Generally, MQTT has "keepalive" property. This property describes how much time my client will be connected (AFAIK it sends the ping to the server) to MQTT server when there are no updates on the subscribed topic.
But here I have a huge problem. Needed topic could be updated once per hour or even once per few days/months. Let's say that this is indoor alarm.
How can I avoid that keepalive timeout or ignore that field? Could it be unlimited?
You have miss understood what the keepalive value represents.
MQTT clients can stay connected indefinitely even if they do not publish or receive any messages. But the broker needs to keep track of which clients are still connected so it knows when to send the Last Will and Testament (LWT) message for the client. In order to do this it uses the keepalive time.
Every time a message is sent or received by the client, the broker resets a timer, if this timer exceeds 1.5 times the value of the keepalive time then the broker marks the client as disconnected and processes the LWT. To prevent clients with very low messages rates from being disconnected, such a client can send a PINGREQ packet at any time (most likely on timeout of the keepalive value) to the server/broker. The server receives the PINGREQ, answers with a PINGRESP packet and it will reset the keepalive timer to zero and leave the client in the connected state.
See Keep Alive section of the MQTT standard: (http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html#_Toc385349238)
The Client can send PINGREQ at any time, irrespective of the Keep Alive value, and use the PINGRESP to determine that the network and the Server are working. If the Keep Alive value is non-zero and the Server does not receive a Control Packet from the Client within one and a half times the Keep Alive time period, it MUST disconnect the Network Connection to the Client as if the network had failed
When sending the initial MQTT CONNECT message from a client, you can supply an optional "keep-alive" value. This value is a time interval, measured in seconds, during which the broker expects a client to send a message, such as a PUBLISH message. If no message is sent from the client to the broker during the interval, the broker automatically closes the connection. Note that the keep-alive value you specify is multiplied by 1.5, so setting a 10-minute keep-alive actually results in a 15 minute interval.
Have a look at the Keep Alive section of the MQTT specification:
A Keep Alive value of 0 has the effect of turning off the Keep Alive mechanism. If Keep Alive is 0 the Client is not obliged to send MQTT Control Packets on any particular schedule. v5 spec source
Therefore, set the keep alive to 0, and then the client doesn't have to send a keep alive signal as often. The server should respect that this connection with a client (e.g. from last year) should still be connected, but it won't be guaranteed (The client might be disconnected when the server is shut down).
Related
I created a session using the following script:
import dolphindb as ddb
s = ddb.session()
s.connect("localhost", 8848)
The session often got disconnected when interacting with the DolphinDB server. How can I keep the session alive for a specified period of time?
In usual circumstances, sessions created with ddb.session will not get disconnected.
DolphinDB Python API provides the following parameters to keep sessions alive in case of network error:
(1) keepAliveTime and reconnect of the connect function
keepAliveTime - specifies the duration between two keepalive transmissions to detect the TCP connection status. Set the parameter to release half-open TCP connections timely when the network is unstable. The default value is 30 (seconds). For example, when you set keepAliveTime to 50, the session would not got disconnected if the network was down for 30 seconds.
reconnect - When set to true, Python API tries to reconnect the session to the server when a connection error is detected.
(2) ddb.session.setTimeout()
Only available on Linux
The static method setTimeout of the Session object specifies the TCP connection timeout period (in seconds) before the peer acknowledges (ACK) receiving the data packets. After the timeout period expires, the TCP connection is closed. It is equivalent to the TCP_USER_TIMEOUT option. The default value is 30. If the value is specified as 0, TCP will use the system default.
Use ddb.session.setTimeout() in the following data transmission scenarios:
Your network bandwidth is full and the amount of data you want to transmit is large
One-time transmission of a large amount of data. Transmitting a large amount of data takes a lot of time and, as a result, ACK may not be received in time. After the timeout period expires, the system closes the TCP connection (the session).
Let's say we have an established websocket connection between our server and a remote server and the remote server from time to time sends some data to us.
So does our websocket connection spend outbound data traffic when receiving data?
My guess is it does not, because the receiving data gets accumulated in the memory. So when you do a .recv(), the websocket just pulls out the data from the memory locally and sends nothing to the other server. Is this correct?
At the level of the web socket protocol it is likely that recipient server sends a ping message to the originating client every couple of minutes, which responds with a pong.
So, the few-tens-of-bytes pong message flows back every once in a while.
At the TCP / IP level, the receiver server responds to every second incoming data packet with an ACK packet, comprising 30 bytes. Incoming data packets can carry up to 1460 bytes of payload data plus the 30 bytes.
So, there's a minimal amount of reverse data transmission, but it's not zero.
If you must have zero reverse transmission you need to use a datagram protocol. But datagram protocols are lossy: there's no way to recover a lost datagram.
I am using Python based paho mqtt client to publish data to mosquitto mqtt broker.
Let's assume a scenario, when client wanted to publish message and broker got disconnected.
So python based client object buffers that message in _out_message (Ordered Dictionary), and keep retrying to send messages.
I wanted to know,
For how long mqtt client will buffer such message?
Is there any time limit or retry limit after which client will drop the message.
I wanted to dump/log such messages.
According to Eclipse Paho Python documentation, you can set the maximum number of outgoing messages with Quality of Service greater than 0 (QoS > 0) that can be pending in the outgoing message queue with the method:
max_queued_messages_set(self, queue_size)
It seems that using the default value (0) all the messages are kept until the MQTT client is able to send them. So, in the end, I suppose that messages are kept until the Python process reach the memory limit imposed by the operating system.
You can force the MQTT client to discard messages using the method reinitialise.
reinitialise(client_id="", clean_session=True, userdata=None)
I have a basic implementation of a TCP client using python sockets, all the client does is connect to a server and send heartbeats every X seconds. The problem is that I don't want to send the server a heartbeat if the connection is closed, but I'm not sure how to detect this situation without actually sending a heartbeat and catch an exception. When I turn off the server, in the traffic capture I see FIN/ACK arriving and the client sends an ACK back, this is when I want my code to do something (or at least change some internal state of the connection). Currently, what happens is after the server went down and X seconds passed since last heartbeat the client will try to send another heartbeat, only then I see RST packet in the traffic capture and get an exception of broken pipe (errno 32). Clearly python socket handles the transport layer and the heartbeats are part of application layer, the problem I want to solve is not to send the redundant heartbeat after FIN/ACK arrived from server, any simple way to know the connection state with python socket?
I have 2 rabbitmq in cluster load balanced by an Azure Internal Load Balancer. The client connect to the LB with BlockingConnection.
When there a are message exchanged by the clients, everything works fine. But when there is no activity, it seems my clients are disconnected and and not able to receive messages anymore.
I am wondering if there is a solution to this problem ? I assume the load balancer or rabbitmq is closing the connection due to inactivity. I wanted to let pika trigger some heartbeat to rabbitmq (so that the load balancer keep the connection opened) but didn't find any good solution. Could you advice ?
EDIT 1
It seems pika BlockingConnections does not support heartbeat. Heart beat disables blocking connection
Thank you.
According to the Pika doc http://pika.readthedocs.org/en/0.10.0/modules/parameters.html, it seems Pika BlockingConnentions with URLParameters specified heart_interval(such as amqps://www-data:rabbit_pwd#rabbit1/web_messages?heartbeat_interval=30) that can keep connection opened, but the value of heart_interval can not larger than the value of rabbit server suggested.
Heartbeat Timeout Interval
The heartbeat timeout value defines after what period of time the peer TCP connection should be considered dead by RabbitMQ and client libraries. This value is negotiated between the client and RabbitMQ server at the time of connection. The client must be configured to request heartbeats. In RabbitMQ versions 3.0 and higher, the broker will attempt to negotiate heartbeats by default (although the client can still veto them). The timeout is in seconds, and default value is 60 (580 prior to release 3.5.5).
Heartbeat frames are sent about every timeout / 2 seconds. After two missed heartbeats, the peer is considered to be unreachable. Different clients manifest this differently but the TCP connection will be closed. When a client detects that RabbitMQ node is unreachable due to a heartbeat, it needs to re-connect.
Heartbeats can be disabled by setting the timeout interval to 0.
Sample code from Pika doc:
import pika
parameters = pika.URLParameters('amqps://www-data:rabbit_pwd#rabbit1/web_messages?heartbeat_interval=30')
connection = pika.BlockingConnection(parameters)