We Keep Coding

Python websockets - message send with success after 'Broken pipe' error

I have server and client written in python. My server is implemented using asyncio and library called 'websockets'. This is asynchronous architecture. Client from the other hand is implemented in library called 'websocket-client'. They are 2 different code bases and repositories.
In server repository i am calling serve method to start websocket server that is accepting connection from clients and allows them to send messages to sever. It looks like this:
async with serve(
self.messages_loop, host, port, create_protocol=CentralRouterServerProtocol
) as ws_server:
...
Client is using websocket-client library and it is connecting to websocket by calling 'create_connection' method. Later it is calling 'send' method to send message to server. Code:
client = create_connection(f'ws://{central_router.public_ip}', timeout=24*60*60, header=cls.HEADERS)
cls.get_client().send(json.dumps(message_dict)) // Sends message later loop. After user will type something from input.
Main requirement is that client can only send messages. It cant read it.After that server is sending ping every X seconds to confirm that connection is alive. Server waits another Y seconds for client to reply to him. Client cant reply to server, because it is running on synchronous block of code. The server is closing the connection but client doesnt know about it. Client is not reading from websocket (so he cant get information about closed websocket -> is that true?). After that sobody is typing something into input, and client is sending message to server. AND NOW -> the websocket-client send method is not raising any exception (that connection is closed), but messages will never get to the server. If user will type message one more time, it will get finnaly exception
[Errno 32] Broken pipe
but the first message after connection close will never raise and error/exception.
Why is that? What is going on? My first solution was to set ping_timeout to None on server side. It will make server not to wait this Y seconds for response, and it will never close a connection. However this is wrong solution, because it can cause zombie connections on server side.
Do anyone know, why client can sand one more message with success, after the pipe was broken?

Paho MQTT (Python) - when to call client.loop_stop() during disconnection?

I'm writing a simple Python client using Paho MQTT to communicate with mosquitto. I'm running into some issues with client.disconnect() and more specifically client.loop_stop().
According to the Paho docs, loop_start() is used to start up a threaded network loop. I've found that the most reliable way to call this is immediately after calling client.connect(). Apparently calling it just prior can have some unexpected effects. Anyway, for me this works fine.
The issue is around when I call client.loop_stop() around the time I wish to disconnect. Some online tutorials show that the best place to call this is in the on_disconnect handler, to be sure that the disconnection has fully completed, and so that any pending subscription or unsubscription attempts have been fully handled by the broker. This appears to work, but if I then attempt to reconnect (by a call to client.connect()) then the connection attempt does not work correctly - the client state gets stuck midway through, and the mosquitto broker only reports the following:
1642374165: New connection from 127.0.0.1:39719 on port 1883.
And nothing else. The connection has not worked. I'm not sure if the Broker is waiting for something from the Client (most likely?), or the Client has sent bad data to the Broker and triggered some kind of issue, but whatever the case, the connection is not valid.
If I move the loop_stop() call to just prior to the call to client.disconnect(), I get much more reliable behaviour, and the broker shows a proper subsequent connection attempt:
1642375893: New connection from 127.0.0.1:38735 on port 1883.
1642375893: New client connected from 127.0.0.1:38735 as client0 (p2, c0, k60).
1642375893: No will message specified.
1642375893: Sending CONNACK to client0 (1, 0)
However I understand that this may cause other issues - in particular, the disconnection may occur before any pending subscription or unsubscription requests have been fully processed, since the network loop is terminated before the disconnection is performed.
What I'd like to know is what's the official word on how to do a clean disconnection and terminate the network loop thread properly, without resorting to arbitrary time.sleep() delays to give things enough time to work themselves out.

Mqtt check if subscribed topic has stopped publishing messages

I am trying to implement a way to know when a specific device has stopped publishing messages to an mqtt broker and in that case send an email to myself. I want to do this to be able to know if there is a problem with the device that is publishing the messages so I can check it and turn it back on. To try to accomplish this I created a mqtt client that subscribes to the topic that the device publishes e.g. test/device_1 and then set as last will and testament for that device status/device_1 where I put as payload="Offline". Ideally, I want to be able to do this for more than 1 device, but let's assume I just want it for the simple case of one device.
I created another script that implements another client that is subscribed to the topic status/device_1 and then on the on_message function it checks if it gets the payload="Offline" and if it does get it then I send an email to myself.
This approach however doesn't work as when I turn off my device, the mqtt client that is subscribed to the topic test/device_1 keeps listening but gets no messages. In other words, it doesn't send its last will even when the topic is empty. However, it seems to work when I stop the script that is subscribed to the topic test/device_1.
Am I missing something or is it not possible to accomplish what I am trying to do? Any advice is appreciated!

First LWT messages are only published by the broker if the client goes offline unexpectedly, which means if it shuts down cleanly then it will not get published, you have to manually publish a message as part of shut down process.
You can set the retained flag on the LWT so that any client that subscribes after the device has gone offline it will still see the state message.
You should also clear that message on when the device starts up, either by publishing something like Online or a null payload to clear the retained message.

You can use last timestamp when device published the messages as heartbeat and in your another script, if (current_time - last_timestamp) > 30 then you can email yourself that your device is offline. this way you can have threshold of time when you want to get email of device offline.
script 1:
mqttCient.on_message(msg):
last_timestamp = time.time()
script 2:
cur_time = time.time()
if ((cur_time - last_timestamp) > 30):
# email yourself
use Json file or database as buffer for storing and accessing last_timestamp

How to write a function that awaits for a message from MQTT before continuing the program?

I am trying to send a message using publish.single and then receive it and act upon the data received. Hence, I can't proceed unless I receive something from the server, so is there a way to write a statement that will wait for a message from MQTT before proceeding?
Example code:
publish.single("$topic", data, ip_address)
#can't do anything here
receive(data_from_broker) #or anythin that looks like it!
#continue with the program here

The short answer is you don't.
At least not in the way you describe. MQTT is an asynchronous protocol, there is no sense of sending a message and waiting for a response, a publishing client has no way to know if the is 0, 1 or n subscribing clients listening on the topic the message is published.
You will need to build something called a state machine to keep track of where in the program flow you are when messages are received.
e.g.
Application published message and sets flag in the state machine to say the message was sent
Remote client receives message and publishes a response
New message is received by the first client, it checks the state machine to determine that the message should be treated as a response to the original message.
To subscribe to a topic you will have to move on from using publish.single to the full MQTT client so you can setup the onMessage callback to handle the incoming messages.

How does redis publish work?

Suppose that there is one redis-client which is subscribing to channel c1 and another redis-client publishes "data" to channel c1.
At this point, does the data published is pending in a redis-server by when the client subscribing to "c1" get the data(by calling pubsub.listen() or pubsub.get_message()) or go directly to the client subscribing to the channel c1 through a redis-server?
In other words, When a redis-client call pubsub.getMessage() or pubsub.listen(), does redis-client send a request to redis-server to get data or just get data from the local socket buffer?
When I read some documents it is saying that pubsub.get_message() use select module for a socket internally.
It seems to mean that the data subscribed already is in the client local buffer, not server.
Could you give me any suggestion?

When a client does PUBLISH, this command immediately pushes the message into all sockets that are interested in it. The message is not stored in redis. If a client wasn't subscribed at the time of publishing, he will never see the message.
I'm not familiar with python driver for redis, but judging from the names, I'd guess that listen sends a SUBSCRIBE command and get_message takes a message from the local buffer.

Redis pubsub is fire and forget.
In other words, when a publish command is sent, the message is received by online subscribers, and who wasn't subscribed/listening for messages when the publish command was sent, won't never get these messages.