We Keep Coding

How can I let my code send messages simultaneously?

I'm trying to do UDP socket programming in Python, and I want both the client and the server to be able to send messages without the need to wait for the other party to send a message first.
Here is my server code:
import socket
sock=socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
sock.bind(('127.0.0.1',12345))
while True:
data,addr=sock.recvfrom(4096) #byte size
print("Client says: ")
print(str(data))
message = bytes(input("Enter message here: ").encode('utf_8'))
sock.sendto(message,addr)
and here is my client code:
import socket
client_socket=socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
while True:
msg=input("Enter your message here: ")
client_socket.sendto(msg.encode('utf_8'),('127.0.0.1',12345))
data,addr=client_socket.recvfrom(4096) #byte size
print("Server says")
print(data)
What should I edit in my code to make this work?

You need to create two functions to handle listening and sending.
Then you start them as threads so they run in parallel.
Each function has its own loop. The receive thread waits for messages and the send thread waits for user input
def client_receive():
while True:
data,addr=client_socket.recvfrom(4096) #byte size
print("Server says")
print(data)
def send_chat_message():
while True:
msg=input("Enter your message here: ")
client_socket.sendto(msg.encode('utf_8'),('127.0.0.1',12345))
receive_thread = threading.Thread(target=client_receive)
receive_thread.start()
send_thread = threading.Thread(target=send_chat_message)
send_thread.start()
The above code will not really work at this point. But there are good tutorials out there. Search for python socket chat threaded tutorial
Often they are not based on a server handling inputs, but multiple clients connecting to a server. If you understand the concept, you'll have no problem adjusting the server to allow inputs there aswell.

BLE device does not make new /dev/input/eventX when it reconnects, using Python Gatt Library

I am new to python gatt module, and i am having a problem with reconnections.
Basically what I am trying to do is establish a connection with a Bluetooth Low Energy (BLE) device with the python gatt module( https://github.com/getsenic/gatt-python ) and then read the input from the /dev/input/eventX path with the evdev module. I also want to automate the reconnection process, so when the device gets out of range and comes back, it will reconnect and continue working normally.
The problem is that when the device disconnects, and eventually reconnects (via simple routine like this: catch disconnect message -> try to reconnect) if the reconnection has taken more than 2-3 minutes, the connection process does not make a new /dev/input/eventX path. This is not happening when the reconnection is successful in between the first 1-2 minutes.
The error I am getting when the 2-3 minutes have passed is:
File "/usr/lib/python3.7/site-packages/dbus/proxies.py", line 145, in
call
File "/usr/lib/python3.7/site-packages/dbus/connection.py", line 651, in call_blocking
dbus.exceptions.DBusException:
org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible
causes include: the remote application did not send a reply, the
message bus security policy blocked the reply, the reply timeout
expired, or the network connection was broken.
The core of the script is the following:
def reconnect(mac_address):
try:
devices[mac_address].connect()
except:
print(f"thread from {mac_address} crashed")
class AnyDevice(gatt.Device):
shut_down_flag = False
def connect_succeeded(self):
super().connect_succeeded()
print(f"{self.mac_address} Connected")
def connect_failed(self, error):
super().connect_failed(error)
print(f"{self.mac_address} Connection failed.")
reconnect_thread = threading.Thread(target=reconnect, name=f'reconnect {self.mac_address}',args=(self.mac_address,))
reconnect_thread.start()
def disconnect_succeeded(self):
super().disconnect_succeeded()
print(f"{self.mac_address} Disconnected")
if not self.shut_down_flag:
reconnect_thread = threading.Thread(target=reconnect, name=f'reconnect {self.mac_address}',args=(self.mac_address,))
reconnect_thread.start()
def gatt_connect_device(mac_address):
global devices
devices.update({f'{mac_address}': AnyDevice(mac_address=f'{mac_address}', manager=manager)})
devices[f'{mac_address}'].connect()
#==== OPEN bd_addresses.txt JSON FILE ====#
if path.exists("bd_addresses.txt"):
with open("bd_addresses.txt", "r") as mac_addresses_json:
mac_addresses = json.load(mac_addresses_json)
else:
print("bd_addresses.txt file NOT FOUND\nPlace it in the same directory as the multiple_scanners.py")
#========================================#
devices={}
manager = gatt.DeviceManager(adapter_name='hci0')
for scanner_number in mac_addresses:
device_instance_thread=threading.Thread(target=gatt_connect_device, name=f'device instance for {mac_addresses[scanner_number]}', args=(mac_addresses[scanner_number],))
device_instance_thread.start()
time.sleep(3)
manager.run()

python3.5: asyncio, How to wait for "transport.write(data)" to finish or to return an error?

I'm writing a tcp client in python3.5 using asyncio
After reading How to detect write failure in asyncio? that talk about the high-level streaming api, I've tried to implement using the low level protocol api.
class _ClientProtocol(asyncio.Protocol):
def connection_made(self, transport):
self.transport = transport
class Client:
def __init__(self, loop=None):
self.protocol = _ClientProtocol()
if loop is None:
loop = asyncio.get_event_loop()
self.loop = loop
loop.run_until_complete(self._connect())
async def _connect(self):
await self.loop.create_connection(
lambda: self.protocol,
'127.0.0.1',
8080,
)
# based on https://vorpus.org/blog/some-thoughts-on-asynchronous-api-design-in-a-post-asyncawait-world/#bug-3-closing-time
self.protocol.transport.set_write_buffer_limits(0)
def write(self, data):
self.protocol.transport.write(data)
def wait_all_data_have_been_written_or_throw():
pass
client = Client()
client.write(b"some bytes")
client.wait_all_data_have_been_written_or_throw()
As per the python documentation, I know write is non-blocking, and I would like the wait_all_data_have_been_written_or_throw to tell me if all data have been written or if something bad happened in the middle (like a connection lost, but I assume there's way more things that can go bad, and that the underlying socket already return exception about it?)
Does the standard library provide a way to do so ?

The question is mainly related to TCP sockets functionality, not asyncio implementation itself.
Let's look on the following code:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, port))
s.send(b'data')
Successful send() call means the data was transferred into kernel space buffer for the socket, nothing more.
Data was not sent via wire, not received by peer and, obviously, not processed by received.
Actual sending is performed asynchronously by Operation System Kernel, user code has no control over it.
What's why wait_all_data_have_been_written_or_throw() make not much sense: writing without an error doesn't assume receiving these data by peer but only successful moving from user-space buffer to kernel-space one.

Closing clients sockets after server has exited

I am writing a simple client/server socket program where clients connect with server and communicate and then they send exit msg to server and then server closes the connection. The code looks like below.
server.py
import socket
import sys
from threading import Thread
try:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# This is to prevent the socket going into TIME_WAIT status and OSError
# "Address already in use"
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
except socket.error as e:
print('Error occured while creating the socket {}'.format(e))
server_address = ('localhost', 50000)
sock.bind(server_address)
print('**** Server started on {}:{} ****'.format(*server_address))
sock.listen(5)
def client_thread(conn_sock, client_add):
while True:
client_msg = conn_sock.recv(1024).decode()
if client_msg.lower() != 'exit':
print('[{0}:{1}] {2}'.format(*client_add, client_msg))
serv_reply = 'Okay ' + client_msg.upper()
conn_sock.send(bytes(serv_reply, 'utf-8'))
else:
conn_sock.close()
print('{} exitted !!'.format(client_add[0]))
sys.exit()
try:
# Keep the server until there are incominmg connections
while True:
# Wait for the connctions to accept
conn_sock, client_add = sock.accept()
print('Recieved connection from {}:{}'.format(*client_add))
conn_sock.send(
bytes('***** Welcome to {} *****'.format(server_address[0]), 'utf-8'))
Thread(target=client_thread, args=(
conn_sock, client_add), daemon=True).start()
except Exception as e:
print('Some error occured \n {}'.format(e))
except KeyboardInterrupt as e:
print('Program execution cancelled by user')
conn_sock.send(b'exit')
sys.exit(0)
finally:
sock.close()
client.py
import socket
import sys
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ('localhost', 50000)
print('Connecting to {} on {}'.format(*server_address))
sock.connect(server_address)
def exiting(host=''):
print('{} exitted !!'.format(host))
sys.exit()
while True:
serv_msg = sock.recv(1024).decode()
if serv_msg.lower() != 'exit':
print('{1}: {0}'.format(serv_msg, server_address[0]))
client_reply = input('You: ')
sock.send(bytes(client_reply, 'utf-8'))
if client_reply.lower() == 'exit':
exiting()
else:
exiting('Server')
What I want is in case server exits either through ctrl-c or any other way I want all client sockets to be closed and send msg to clients upon which they should close their socket as well.
I am doing below in except section but for some reason the msg sent by server is not being received by the client.
except KeyboardInterrupt as e:
print('Program execution cancelled by user')
conn_sock.send(b'exit')
sys.exit(0)
Surprisingly if I send the 'exit' msg from client_thread as srvr_reply, the client accepts the msg and exit the client socket at its end just fine. So I am not sure as to why the server is not able to send the same message in except section of the code as mentioned above.

I'm sorry to say that abnormal termination of TCP/IP connections is undetectable unless you try to send data through the connection.
This is known as a "Half Open" socket and it's also mention in the Python documentation.
Usually, when a server process crashes, the OS will close TCP/IP sockets, signaling the client about the closure.
When a client receives the signal, the server's termination can be detected while polling. The polling mechanism (i.e. poll / epoll / kqueue) will test for the HUP (hung up) event.
This is why "Half Open" sockets don't happen in development unless the issue is forced. When both the client and the server run on the same machine, the OS will send the signal about the closure.
But if the server computer crashes, or connectivity is lost (i.e. mobile devices), no such signal is sent and the client never knows.
The only way to detect an abnormal termination is a failed write attempt read will not detect the issue (it will act as if no data was received).
This is why they invented the ping concept and this is why HTTP/1.1 servers and clients (that don't support pings) use timeouts to assume termination.
There's a good blog post about Half Open sockets here.
EDIT (clarifications due to comments)
How to handle the situation:
I would recommend the following:
Add an explicit Ping message (or an Empty/NULL message) to your protocol (the messages understood by both the clients and the server).
Monitor the socket for inactivity by recording each send or recv operation.
Add timeout monitoring to your code. This means that you will need to implement polling, such as select (or poll or the OS specific epoll/kqueue), instead of blocking on recv.
When connection timeout is reached, send the Ping / empty message.
For an easy solution, reset the timeout after sending the Ping.
The next time you poll the socket, the polling mechanism should alert you about the failed connection. Alternatively, the second time you try to ping the server/client you will get an error message.
Note that the first send operation might succeed even though the connection was lost.
This is because the TCP/IP layer sends the message but the send function doesn't wait for the TCP/IP's ACK confirmation.
However, by the second time you get to the ping, the TCP/IP layer would have probably realized that no ACK is coming and registered the error in the socket (this takes time).
Why the send failed before exiting the server
The comment I left about this issue is wrong (in part).
The main reason the conn_sock.send(b'exit') failed is because conn_sock is a local variable in the client thread and isn't accessible from the global state where the SIGINT (CTRL+C) is raised.
This makes sense, as what would happen if the server has more than a single client?
However, it is true that socket.send only schedules the data to be sent, so the assumption that the data was actually sent is incorrect.
Also note that socket.send might not send the whole message if there isn't enough room in the kernel's buffer.

Python client keeps websocket open and reacts to received messages

I'm trying to implement a REST client in python that reacts to messages received from the server received through an opened websocket with the concerned server.
Here is the scenario:
client opens a websocket with the server
from time to time, the server sends a message to the client
when the client receives the messages, it gets some information from
the server
The current client I have is able to open the websocket and to receive the message from the server. However, as soon as it receives the messages, it gets the information from the server then terminates while I'd like to keep it listening for other messages that will make it get a new content from the server.
Here is the piece of code I have:
def openWs(serverIp, serverPort):
##ws url setting
wsUrl = "ws://"+serverIp+":"+serverPort+"/websocket"
##open ws
ws = create_connection(wsUrl)
##send user id
print "Sending User ID..."
ws.send("user_1")
print "Sent"
##receiving data on ws
print "Receiving..."
result = ws.recv()
##getting new content
getUrl = "http://"+serverIp+":"+serverPort+"/"+result+"/entries"
getRest(getUrl)
I don't know if using threads is appropriate or not, I'm not expert in that.
If someone could help, it'll be great.
Thanks in advance.

I finished with this code, doing what I'm expecting. Git it from here
import websocket
import thread
import time
def on_message(ws, message):
print message
def on_error(ws, error):
print error
def on_close(ws):
print "### closed ###"
def on_open(ws):
def run(*args):
for i in range(3):
time.sleep(1)
ws.send("Hello %d" % i)
time.sleep(1)
ws.close()
print "thread terminating..."
thread.start_new_thread(run, ())
if __name__ == "__main__":
websocket.enableTrace(True)
ws = websocket.WebSocketApp("ws://localhost:5000/chat",
on_message = on_message,
on_error = on_error,
on_close = on_close)
ws.on_open = on_open
ws.run_forever()