Good evening,
I am working on a project which is using eventlet http://eventlet.net/ ontop of wsgi in Python to create a websocket server. So far everything is working well. We did some deep level packet analysis today however and noticed something odd. We are trying to minimize the outbound data from the server. At the moment it only seems to send ACK's which is taking up much of the data outbound at the moment (around 1M per hour).
No matter what we set the receive window too on the Ubuntu sysctl file an ACK is being sent to the client at the same time (after 5 packets). We want to make this window larger such that maybe 1 ACK is sent every 15 packets these packets are 1440 Bytes in size
Below are our sysctl settings for the TCP send and receive buffers
net.core.wmem_max = 4194304
net.core.rmem_max = 6291456
net.ipv4.tcp_rmem = 4096 2097152 6291456
net.ipv4.tcp_wmem = 4096 1048576 4194304
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_timestamps = 1
net.ipv4.tcp_sack = 1
net.core.netdev_max_backlog = 5000
As you will see we have scaling enabled, with it enabled the server is setting a Win=8159 in its Ack however even with it disabled and a default of 64k being sent in the servers ACK window the ACK is actually being sent back to the client around every 7k.
Is there something fundamental we are missing here? Can a client set a buffer limit at which it will no longer send data unless it receive an ACK? Is there some sort of time limit at which a ACK is sent regardless of the received buffer size?
Thank you for your help.
Is there some sort of time limit at which a ACK is sent regardless of the received buffer size?
ACK is being sent to give a feedback from receiver, if sender will not receive ACK during some time (estimated RTT) it guesses that packet is lost and will retransmit this packet again. In basic tcp version, ACK will be sent in response to every packet, but as optimization many OS implements delayed ACK. Receiver will wait some time before sending ACK, and if receiver will receive another packet during this timeout, it will generate only one ACK (with largest number) for those packets that was received during this time. Because of this ACK will be sent regardless any receiver buffer. On linux you can disable delayed ACK via TCP_QUICKACK socket option, in this case every received packet will generate ACK packet. On some versions of linux you also can change this timeout, but this could cause unnecessary data retransmission.
Can a client set a buffer limit at which it will no longer send data unless it receive an ACK?
It possible via changing send buffer. If send buffer is full next write operations would block (in case of blocking client). Every ACK will free packet from send buffer and will allow next write operations.
Related
Let's say we have an established websocket connection between our server and a remote server and the remote server from time to time sends some data to us.
So does our websocket connection spend outbound data traffic when receiving data?
My guess is it does not, because the receiving data gets accumulated in the memory. So when you do a .recv(), the websocket just pulls out the data from the memory locally and sends nothing to the other server. Is this correct?
At the level of the web socket protocol it is likely that recipient server sends a ping message to the originating client every couple of minutes, which responds with a pong.
So, the few-tens-of-bytes pong message flows back every once in a while.
At the TCP / IP level, the receiver server responds to every second incoming data packet with an ACK packet, comprising 30 bytes. Incoming data packets can carry up to 1460 bytes of payload data plus the 30 bytes.
So, there's a minimal amount of reverse data transmission, but it's not zero.
If you must have zero reverse transmission you need to use a datagram protocol. But datagram protocols are lossy: there's no way to recover a lost datagram.
I am on working a project with Socket TCP/IP ( Server-C# and Client-Python).
Streaming video after sometimes,the data of Recv Socket is splitted.
My data is buff = 22000 bytes,if it is splitted it will become :
buff = 1460
buff = 20600
I don't know why,i have researched some methods with MTU,Fragmentation,Windows Size,....but not have result
Specially,if i setsocketopt the process will appear less.
self.sk.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 1048576)
enter image description here --Image about the data is splitted
This is my Recv Socket.
buff = self.sk.recv(1048576)
print("BUFF RECEIVE ::: ::::: ---->>>>> ",len(buff))
if buff == b'' :
self.sk=None
buff = None
return buff
Suggestions: This just only happened to Chrome Browser(It mean,can't Streaming Video if loss data ).But at Firefox,it is not.It seem to blink a moment if loss data but It can continue stream after.
enter image description here - Chrome and FireFox
That is just the way TCP works. It is a streaming protocol with no built-in message framing, so it is free to place bytes into packets in any amounts it chooses — all it guarantees is that the bytes will still be in the correct order when they are read() by the receiving socket.
Therefore, if your program requires a certain number of bytes before it can proceed, it is up to your program to do the necessary buffering to assemble those bytes together.
As for why TCP might behave the way you observed —it is likely reacting to network conditions (dropped packets, feedback from the receiving host’s TCP stack, etc), and trying to make transmission as efficient as possible given its current environment. It’s 100% up to the TCP stack how it wants to transmit the data, and different TCP stacks may behave differently, which is fine as long as they follow the rules of the TCP specification.
After a long time,I have found the answer for my issue.
Solution for TCP/IP client socket message boundary problem
**1/**When you send a package from Server to Client with Send(Write).At Client side,the Receive will not get full data in sometimes.It not mean,Send/write at Server not send enough data.Just because this is TCP/IP protocol,Receive is not graduatee and the package will be fragmentation at Client Side ( your code ).
**2/**You can solve this issue by add more pattern at send/write Server Side. For example, send(data) --> send ( Pattern + data) and at Client side,you can use patern to check data.
**3/**Limitations of this method,the package after fragmentation,it can "combine together" or sometime it can't not.For example,your send data = 4000 and at Client side,your receive = 1460 + 2540.
This is what I understood with my issue.
I am reading a networking book and from what I have read about the TCP protocol, it makes sure the data will be sent. I want to write some code to do a file transfer. Before getting to that, I also read in the Python documents this passage:
"Applications are responsible for checking that all data has been
sent; if only some of the data was transmitted, the application needs
to attempt delivery of the remaining data"
This seems to contradict what I read in the networking book. The passage above says applications are responsible for the lost data.
I may be misunderstanding so I want to ask some questions:
1-If I have to check that the data is sent, then why use TCP?
2-I read in the networking book that TCP does the math to make sure that the data is there. Then why isn't using TCP a waste of time ?
3- The python docs didn't specify a buffer size. what is the maximum size of buffer to send at a time?
4-I read in the networking book that the server can increase the amount of
data that it can send if it knows the client can receive it. can this change
the size of the buffer more than the maximum number?
Here is my code attempt so far:
Server code:
import socket
s = socket.socket()
host = socket.gethostname()
port = 3000
s.bind((host,port))
s.listen(1)
c,addr = s.accept()
with open("Filetosend","rb") as File:
data= File.read(1024)
while data:
c.send(data)
data = File.read(1024)
s.close()
Client code:
import socket
s= socket.socket()
host = socket.gethostname()
port = 3000
s.connect((host,port))
with open("Filetowrite","wb") as File:
data = s.recv(1024)
while data:
File.write(data)
data = s.recv(1024)
s.close()
TCP tries to guarantee that if the data is delivered, it's correct and in order. It uses checksums to ensure data isn't corrupted, and sequence numbers to ensure that data is delivered in order and with no gaps. And it uses acknowledgements so the sender will know that data has been received.
But suppose there's a network failure in the middle of a transmission. If it happens after the data segment is received, but before the acknowledgement is sent back, the sender will not know that the data was received. The sender will keep trying to resend the data, and will eventually time out and report an error to the application.
Most TCP APIs don't allow the application to find out precisely where in the communication the error happened. If you sent a megabyte, and get an error, it could have happened at the beginning, when hardly anything was sent, or at the end when most of the data was sent. It could even have happened after all the data was sent -- maybe just the last ACK was lost.
Furthermore, the write() system call generally just puts the data in a kernel buffer. It doesn't wait for the data to be sent to the network, and doesn't wait for the receiver to acknowledge it.
Even if you successfully close the connection, you can't be totally sure. When you close the connection, the sender sends a message to the recipient saying they're done sending data. But closing the connection just queues this in the network stack, it doesn't wait for the other system to acknowledge it.
This is why application protocols have their own level of acknowledgements, on top of the basic TCP protocol. For instance, in the SMTP protocol, the client sends the message contents, followed by a line with a . to indicate the end, then waits for the server to send back a response code that indicates that the message was received successfully and is being delivered or queued. TCP's checking ensures that if you receive this response, the message contents were sent intact.
Regarding the general ability of any protocol to guarantee perfect delivery of all messages, you should read about the Two Generals' Problem. No matter what you do, there's no way to verify delivery of all messages in any communication, because the only way to confirm that the last message was delivered is by sending another message in reply, and now that reply is the last message, and needs confirmation.
If you were writing a basic python TCP server and client would you need to add the SYN, SYN ACK and ACK responses yourself or is it taken care of by the socket module?
If you need to write it yourself would it be as simple as something like this?
Client:
#set up clientSocket
data = "SYN"
clientSocket.send(data.encode('utf-8'))
if((clientSocket.recv(1024)).decode('utf-8') == "SYN ACK") {
data = "ACK"
clientSocket.send(data.encode('utf-8'))
}
and the Server responding in a similar way with checking if recv is SYN and then send SYN ACK. If not what is the correct way of implementing it?
Haven't ran the above code, not bothered about the syntax being 100% correct just curious about the logic behind it
The SYN and ACK are handled by the operating system, with no direct intervention by the user. The user program is not required, nor is it able, to send the initial SYN, SYN-ACK, ACK handshake packets.
The initial SYN is sent on behalf of the client as a consequence of socket.socket.connect().
The initial SYN-ACK is sent on behalf of the server as a consequence of socket.socket.listen().
I'm trying to write an IRC bot but I'm not exactly sure how the receiving of data works. What I currently have:
while True:
data = socket.recv(1024)
#process data
Let's say that for whatever reason it takes it more time to process the data, what would happen if something is sent at that time? Will it get skipped or get added to some sort of a queue and processed after the current one is done?
Depending upon the protocol type the behavior will be different.
TCP:
The TCP RFC clearly states:
TCP provides a means for the receiver to govern the amount of data
sent by the sender. This is achieved by returning a "window" with
every ACK indicating a range of acceptable sequence numbers beyond
the last segment successfully received. The window indicates an
allowed number of octets that the sender may transmit before
receiving further permission.
Also from wikipedia the information is similar:
TCP uses an end-to-end flow control protocol to avoid having the
sender send data too fast for the TCP receiver to receive and process
it reliably. For example, if a PC sends data to a smartphone that is
slowly processing received data, the smartphone must regulate the data
flow so as not to be overwhelmed. TCP uses a sliding window flow
control protocol. In each TCP segment, the receiver specifies in the
receive window field the amount of additionally received data (in
bytes) that it is willing to buffer for the connection. The sending
host can send only up to that amount of data before it must wait for
an acknowledgment and window update from the receiving host.
UDP:
UDP doesn't have any flow control mechanism as TCP. However there is an other implementation of UDP such as RUDP that have some of the features of TCP like flow control.
Here is an other interesting link for the differences between TCP & UDP.