I'm trying to understand how send and receive are working.
I was trying to send continuously data to a server and i noticed that the server would receive mixed bytes because i was sending to much data at a time. See my code:
Server:
import socket, struct
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(("",1996))
server.listen(0)
c,d = server.accept()
while True:
data = c.recv(1024)
print( struct.unpack("i", data)[0] )
Client:
import socket, struct
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.connect(("192.168.1.4",1996))
while True:
data = 1
server.send( struct.pack("i", data) )
Then i change the while loops to this:
Server:
data = c.recv(1024)
print( struct.unpack("i", data)[0] )
c.send( str.encode("Server received your message. You now can continue
sending more data") )
Client:
data = 1
server.send( struct.pack("i", data) )
#Wait to secure the send.
server.recv(1024)
This is working. I'm making sure that the client won't send data before the
server already receive the previous send.
But what if i want to do the same for the server too? How can i make sure that the server will send bytes to the client in a safe way?
I already tried this and i notice that i created an infinity loop because(I used multi-threading in order to send and receive at the same time on the server):
client was sending some data and then waiting to get a signal from the server
that he can send again.
the server was getting some data then sending the signal and after that waiting for a signal from the user that he can send again.
But because the client was actually sending data again, the whole thing was going on again and this caused me an infinity talk-reply loop.
So what can i do to make a continuously conversation between two sockets without mixing the bytes together?
Your problem is caused by Nagle algorithm which works by combining a number of small outgoing messages, and sending them all at once as TCP is a stream protocol. You can enable TCP_NODELAY socket option by calling sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1) to sent data as soon as possible, even if there is only a small amount of data. And on the receiver side, it isn't going to get one packet at a time either, you must implement message boundaries itself if you want "continuous conversation between two sockets without mixing the bytes together".
Related
i need to send a data each every X seconds from a client to a server with SOCKET, so i try this code:
for i in range(100):
localip = '127.0.0.1'
port = 5010
bufferSize = 1024
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR,1)
client.connect((localip, port))
pacote = '2B4F08FE0F3B'
client.send(bytes(pacote, "utf-8"))
client.close()
time.sleep(10)
But in the serv i recive a package with nothing (' '). I need send 100x the same message to the serv, each 10 seconds. Somebody knows where is the mistake?
I have other Client.py where i put manually the data (with the input() ) and he works perfect, but now i need to do this automatically.
The client need to open a new connection to send each data.
But in the serv i recive a package with nothing (' ').
This is because the socket gets closed after sending the message, i.e. client.close().
I need send 100x the same message to the serv, each 10 seconds. Somebody knows where is the mistake?
If the server expects the client to create a single connection and then send many data over this connection before closing it, then the client should do exactly this. Instead your client opens a new connection for each new message and then closes the connection immediately after the message.
This means either you must fix the server to match the clients behavior or fix the client to match the servers behavior.
I have a udp client function written in python running on Windows 10. I'm aware this is not production quality code but I'm only trying to grasp the fundamentals here
client = socket(AF_INET, SOCK_DGRAM)
client.bind(('192.168.0.107', CLIENT_PORT))
client.setblocking(False)
while True:
try:
data = client.recv( 1024 )
except:
continue
if data is not None:
print(data.decode('utf-8'))
I have a server running on an embedded device sending out small udp packets periodically (udp payload size of 22). This client gets about 10 of these packets give or take a few then the script stops receiving the udp packets. The only exception raised in the try/catch block is that there is no data to receive. If I change to blocking the behaviour is the same.
[WinError 10035] A non-blocking socket operation could not be completed immediately
The server is still sending the packets, I can see them in Wireshark with the expected IP addresses, ports, and verified checksums.
The thing is if I add a send after the receive stops the problem goes away entirely and I can continue receiving the udp payloads
client = socket(AF_INET, SOCK_DGRAM)
client.bind(('192.168.0.107', CLIENT_PORT))
client.setblocking(False)
while True:
try:
data = client.recv( 1024 )
except:
continue
if data is not None:
print(data.decode('utf-8'))
client.sendto("a_udp_payload".encode('utf-8'), ('192.168.0.108' , SERVER_PORT))
Is there something fundamental that I am missing here? Why does the stop working on the first snippet but not the second? Is there some buffer flushing issue?
Many thanks
Try to set timeout for the socket client.settimeout(2)
I`m trying to write very simple http server in python. Working version is like this:
def run(self, host='localhost',port=8000):
with socket.socket(socket.AF_INET,socket.SOCK_STREAM) as s:
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((host,port))
s.listen(1)
while True:
connection, adress = s.accept()
with connection:
data = b''
while True:
recived = connection.recv(1024)
data += recived
if len(recived) < 1024:
break
if data != b'':
handle_request(data,connection)
It works , but i have some misunderstanding whats going on.
As i understand, socket "s" accept connection from the client -> and return new socket object "connection" from which i can read what client sends to me and send response. I read data from connection until client send empty line b''. After this point TCP part ends and I pass recived bytes to handler which parse recived data as HTTP.
Qestions: At this point i read all the data which client send to me, but if i want to limit max size of HTTP request, should i just do something like this:
..................................
with connection:
data = b''
request_size_limit=1024*100 # some desired http request max size
while True:
recived = connection.recv(1024)
data += recived
if len(recived) < 1024 or len(data) > request_size_limit:
break
if data != b'':
handle_request(data,connection)
If i do something like this how can I inform client, that for example i have at most 1024*1024 free bytes of RAM and I can`t handle requests larger than this?
If clients want to send more that this limit, he must send several separated requests which will contain 1 part of necessary data?
Or for example for big POST request i must parse each recv(1024) while i found \r\n\r\n sequence , check content length and recv() content length by parts 1024b into some file and proceed after?
A1) If you can't handle the request because it is too large consider just closing the connection. Alternatively you can read (and discard) everything they send and then respond with a 413 Request Took Large.
A2) You'll need to work out a protocol for sending just parts of a request at a time. HTTP doesn't do this natively.
A3) If you can read the whole request in chunks and save it to a file, then it sounds like you have a solution to the 1024*1024 RAM limit, doesn't it?
But fix the issues with reading chunked data off the socket.
I have this sort of a catch-the-flag assignment where I need to connect to a server and a port, receive data, extract the number that was given in this data and resend it. I'm guessing I need to continue doing that until some flag will arrive.
I've tried doing that with python sockets (which I barely understand), and came up with this code:
import socket
import re
#AF_INET for IPv4, SOCK_STREAM for TCP
clientsocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Tell the socket what IP and port number to connect to
clientsocket.connect(('35.157.111.68', 10172))
while 1:
# Recieve 1024 bytes of data.
data = clientsocket.recv(1024)
if not data: break
# Get only numbers from string
data = re.sub('\D',"", data)
# Send our result to the server.
clientsocket.send(str(data))
It's establishing a connection and receiving the data, but when it sends the number back it doesn't accept it properly and it does it for only one round (doesn't loop).
Referencing this example (and the docs): https://pymotw.com/2/socket/tcp.html I am trying to achieve bidirectional communication with blocking sockets between a client and a server using TCP.
I can get one-way communication to work from client->server or server->client, but the socket remains blocked or "hangs" when trying to receive messages on both the server and client. I am using a simple algorithm(recvall), which uses recv, to consolidate the packets into the full message.
I understand the sockets remain blocked by design until all the data is sent or read(right?), but isn't that what sendall and recvall take care of? How come disabling recv on either the client or server "unblocks" it and causes it to work? And ultimately what am I doing wrong that is causing the socket to stay blocked?
Here is my code, the only fundamental difference really being the messages that are sent:
recvall(socket)(shared between client and server):
def recvall(socket):
data = ''
while True:
packet = socket.recv(16)
if not packet: break
data += packet
return data
server.py (run first):
import socket
host = 'localhost'
port = 8080
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host, port))
s.listen(5)
while True:
(client, address) = s.accept()
print 'client connected'
try:
print recvall(client)
client.sendall('hello client')
finally:
client.close()
client.py:
import socket
s = socket.create_connection((args.ip, args.port))
try:
s.sendall('hello server')
print recvall(s)
finally:
s.close()
From my understanding (epiphany here), the main problem is that recv inside recvall is only concerned with retrieving the stream (in the same way send is only concerned with sending the stream), it has no concept of a "message" and therefore cannot know when to finish reading. It read all the bytes and did not return any additional bytes, but that is NOT a signal that the message is finished sending, there could be more bytes waiting to be sent and it would not be safe to assume otherwise.
This requires us to have an explicit indicator for when to stop reading. recv and send are only concerned with managing the stream and therefore have no concept of a message (our "unit"). This article has some great solutions to this problem. Since I am sending fixed-length messages, I opted to check that the length is as expected before finishing recv. Here is the updated version of recvall, note MSG_LENGTH must be defined and enforced in order for recvall to not block the socket.
def recvall(socket):
data = ''
while len(data) < MSG_LENGTH:
packet = socket.recv(BUFFER_SIZE)
if not packet: break
data += packet
return data
Bidirectional communication now works, the only catch being the client and server must know the length of the message they will receive, again this is not an issue in my case. This is all new to me so someone please correct me on terminology and concepts.