We Keep Coding

python socket send data after receiving fin (stuck in LAST_ACK)

I have a python program that is supposed to receive some arbitrary bytes and send them back after receiving a fin. I already was able to implement this as you can see below.
The problem I am having is that the connection is never properly closed. Using ss -tan I can see that the connection keeps being stuck in LAST_ACK state. This is although the connection seems to be closed correctly looking at the Wireshark packet trace. I have attached an Image of the Wireshark packet trace that is the result of first sending "AAAAA" then an out of order "BBB" and then filling the hole with "CCC".
Looking at the Wireshark packets I think that all packets should be correctly acknowledged and the connection shoud terminate normally without being stuck in LAST_ACK state and without doing the retransmissions at the bottom. But I Still would guess that there is a problem with the connection closing packets.
# Echo server program
import socket
HOST = '' # Symbolic name meaning all available interfaces
PORT = 6000 # Arbitrary non-privileged port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(1)
conn, addr = s.accept()
print ('Connected by', addr)
data_acc = b''
while 1:
data = conn.recv(1024)
if not data: break
data_acc += data
print(data)
print("send data back")
conn.sendall(data_acc)
conn.close()

I found the answer to the problem. The sequence number of ACK on line 56 in the screenshot needs to be increased by one. By doing so Wireshark also stops interpreting it as a keep alive packet.

Can socket receive infinite data stream python?

While trying to attempt to go for python based socket, I have 2 questions which I am not able to resolve. Kindly help me. Here is my sample socket code:
import socket
import threading
import chardet
bind_ip = '0.0.0.0'
bind_port = 9999
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((bind_ip, bind_port))
server.listen(1) # max backlog of connections
print (('Listening on {}:{}').format(bind_ip, bind_port))
def handle_client_connection(client_socket):
request = client_socket.recv(4096 )
result = chardet.detect(request)
print(result)
print (request.decode(result['encoding']))
client_socket.send('ACK!'.encode(result['encoding']))
client_socket.close()
while True:
client_sock, address = server.accept()
print (('Accepted connection from {}:{}').format(address[0], address[1]))
client_handler = threading.Thread(
target=handle_client_connection,
args=(client_sock,) # without comma you'd get a... TypeError: handle_client_connection() argument after * must be a sequence, not _socketobject
)
client_handler.start()
The above one is server and the below is client:
import socket
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(('127.0.0.1', 9999))
client.send(str('test data').encode("utf-16"))
response = client.recv(4096)
print(response.decode("utf-16"))
Now the questions:
1) What is the meaning of the number in this statement: client.recv(4096)? What is 4096, is it bit or byte of kilobyte for data receiving?
2) Can I receive infinite data stream through the socket? Meaning, as this statement client.recv(4096), whatever is the menaing of 4096, may be byte, then the socket will receive the 4096 bytes of data only. I do not have a control on the size of data received through the socket, hence, can I generalize it to accept any size of data through socket?
Please help me get the answers to above queries. I tried the python documentation for socket but didn't found much. I guess I miss something. Please help me get through it.

According to documentation
"If a message is too long to fit in the supplied buffer, excess bytes may be discarded depending on the type of socket the message is received from."
found here
"https://linux.die.net/man/2/recv"
which was from python 3.6 docs
"he maximum amount of data to be received at once is specified by bufsize. See the Unix manual page recv(2)"
found here
"https://docs.python.org/3.6/library/socket.html"
so it is a bytes object and it may actually truncate the message depending on the protocol. So message integrity is handled by the next layer up. So presumably you would loose part of a UDP packet, and you would get either a retry or an additional packet for TCP if the message was too large to fit in the buffer.

Python Bidirectional TCP Socket Hanging on socket.recv

Referencing this example (and the docs): https://pymotw.com/2/socket/tcp.html I am trying to achieve bidirectional communication with blocking sockets between a client and a server using TCP.
I can get one-way communication to work from client->server or server->client, but the socket remains blocked or "hangs" when trying to receive messages on both the server and client. I am using a simple algorithm(recvall), which uses recv, to consolidate the packets into the full message.
I understand the sockets remain blocked by design until all the data is sent or read(right?), but isn't that what sendall and recvall take care of? How come disabling recv on either the client or server "unblocks" it and causes it to work? And ultimately what am I doing wrong that is causing the socket to stay blocked?
Here is my code, the only fundamental difference really being the messages that are sent:
recvall(socket)(shared between client and server):
def recvall(socket):
data = ''
while True:
packet = socket.recv(16)
if not packet: break
data += packet
return data
server.py (run first):
import socket
host = 'localhost'
port = 8080
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host, port))
s.listen(5)
while True:
(client, address) = s.accept()
print 'client connected'
try:
print recvall(client)
client.sendall('hello client')
finally:
client.close()
client.py:
import socket
s = socket.create_connection((args.ip, args.port))
try:
s.sendall('hello server')
print recvall(s)
finally:
s.close()

From my understanding (epiphany here), the main problem is that recv inside recvall is only concerned with retrieving the stream (in the same way send is only concerned with sending the stream), it has no concept of a "message" and therefore cannot know when to finish reading. It read all the bytes and did not return any additional bytes, but that is NOT a signal that the message is finished sending, there could be more bytes waiting to be sent and it would not be safe to assume otherwise.
This requires us to have an explicit indicator for when to stop reading. recv and send are only concerned with managing the stream and therefore have no concept of a message (our "unit"). This article has some great solutions to this problem. Since I am sending fixed-length messages, I opted to check that the length is as expected before finishing recv. Here is the updated version of recvall, note MSG_LENGTH must be defined and enforced in order for recvall to not block the socket.
def recvall(socket):
data = ''
while len(data) < MSG_LENGTH:
packet = socket.recv(BUFFER_SIZE)
if not packet: break
data += packet
return data
Bidirectional communication now works, the only catch being the client and server must know the length of the message they will receive, again this is not an issue in my case. This is all new to me so someone please correct me on terminology and concepts.

Receiving 'Request String' in Python

I'm building a game-server in Python. The functionality is pretty well-defined. The server will listen on the port 6000 and a remote client will send a request. Then the server will establish a connection to the client's port 7000. From then on, the client will keep sending 'requests' (basically, strings such as "UP#", "DOWN#", "SHOOT#" etc.) to server's port 6000.
This is the problem. I have made a 'server' who listens on the port 6000. This means I cannot bind a client to the same port. Is there a way that I can get the data string of an incoming request in a server? So far, I only have this.
What am I doing wrong here? Any workarounds for this issue? In short, can I read the incoming request string from a client in the server code?
Thanks in advance.
def receive_data(self):
errorOccured = False
connection = None
try:
listener = socket.socket() # Create a socket object.
host = socket.gethostname()
port = 6000 # The port that the server keeps listening to.
listener.bind(('', port))
# Start listening
listener.listen(5)
statement = ("I:P0:7,6;8,1;0,4;3,8;3,2;1,6:5,4;9,3;8,7;2,6;1,4;2,7;6,1;6,3:2,1;8,3;5,8;9,8;7,2;0,3;9,4;4,8;7,1;6,8#\n","S:P0;0,0;0#","G:P0;0,0;0;0;100;0;0:4,3,0;5,4,0;3,8,0;2,7,0;6,1,0;5,8,0;1,4,0;1,6,0#", "C:0,5:51224:824#","G:P0;0,0;0;0;100;0;0:4,3,0;5,4,0;3,8,0;2,7,0;6,1,0;5,8,0;1,4,0;1,6,0#","G:P0;0,1;2;0;100;0;0:4,3,0;5,4,0;3,8,0;2,7,0;6,1,0;5,8,0;1,4,0;1,6,0#")
# This is just game specific test data
while True:
c, sockadd = listener.accept() # Establish connection with client.
print 'incoming connection, established with ', sockadd
i = 0 # Just a counter.
while i<len(statement):
try:
self.write_data(statement[i], sockadd[0])
time.sleep(1) # The game sends updates every second to the clients
i = i + 1
#print listener.recv(1024) -- this line doesn't work. gives an error
except:
print "Error binding client"
c.close() # Close the connection
return
except:
print "Error Occurred"

I'm going to answer it because I got some help and figured it out.
The most basic thing I can do is to use the client connection which is c for this purpose. In here, instead of the commented line data=listener.recv(1024) I should have used data= c.recv(1024). Now it works.
Another way is to use SocketServers with a StreamingRequestHandler. While this is ideal for usage of typical servers, if a lot of objects are involved it could reduce the flexibility.

Python raw IPv6 socket errors

I am having some problems using raw IPv6 sockets in python. I connect via:
if self._socket != None:
# Close out old socket first
self._socket.close()
self._socket = socket.socket(socket.AF_INET6, socket.SOCK_RAW)
self._socket.bind((self._interface,0))
self._socket.sendall(data)
where self._interface is my local address; specifically "fe80::fa1e:dfff:fed6:221d". When trying this, I get the following error:
File "raw.py", line 164, in connect
self._socket.bind((self._interface,0))
File "<string>", line 1, in bind
socket.error: [Errno 49] Can't assign requested address
If I use my ipv6 localhost address for self._interface ("::1") I can actually bind the address, but can not send anything:
self._socket.sendall(data)
File "<string>", line 1, in sendall
socket.error: [Errno 39] Destination address required
Why would a raw socket need a destination address? Has anyone worked with raw IPv6 sockets in python, and can help me understand why this is happening?

Although this is an old question, i thought of adding an answer that works and helps any one who stumbles upon it latter.
The key problems are:
Raw sockets are not bound and connected to other sockets. Also sendto is the correct api to use.
Moreover, 4 tuple structure for destination address is required for ipv6 packets as opposed to two tuple ones for ipv4.
Lastly, the stack (at least on Linux mint 15) is more strict on ipv6 packets. If you try sending an empty icmpv4 echo request, python allows it and sends a meaning less packet on wire. Where as in case of ipv6, it simply gives error of 'invalid argument' when you try sending an empty packet. Hence a valid request is also required in case of ipv6. Following example does that all for ipv6 and sends a valid ping echo request to loop back address.
import socket
def main(dest_name):
addrs = socket.getaddrinfo(dest_name, 0, socket.AF_INET6, 0, socket.SOL_IP)
print addrs
dest = addrs[0]
# A minimal ICMP6-echo message (thanks to abarnert)
data = '\x80\0\0\0\0\0\0\0'
icmp = socket.getprotobyname('ipv6-icmp')
#print icmp
send_socket = socket.socket(socket.AF_INET6, socket.SOCK_RAW, icmp)
print "sent to " + str(dest[4])
send_socket.sendto(data, dest[4])
send_socket.close()
if __name__ == '__main__':
main('::1')

I don't understand your combination of bind and sendall. In my understanding, bind is for server sockets and sendall requires a connection. Did you mean connect instead of bind?
Anyway, the IPv6 equivalent of INADDR_ANY is, according to the man page, IN6ADDR_ANY_INIT. Python does not define a constant for it, but this is the same as '::' (all zero).
This worked for me (as root):
>>> import socket
>>> s = socket.socket(socket.AF_INET6, socket.SOCK_RAW, socket.IPPROTO_RAW)
>>> s.bind(('::', 0))
EDIT:
Oops, I first did not saw that you actually managed to bind the socket to an address. However your second problem is obvious: You must first connect to some address before you can send data. Or use sendto with an address. This is not different from IPv4.

This code provides a raw socket with L2 access. Unfortunately OSX does not support the socket.PF_PACKET...
soc = socket.socket(socket.PF_PACKET, socket.SOCK_RAW) #create the raw-socket
soc.bind(("lo0", 0))
soc.send(packet)