We Keep Coding

WinError 10060 python connect python socket through router

I tried setting up a TCP socket with Python, and it works totally fine as long as I'm in the same network. But my problem is that I can't get it to work if one of the devices isn't in the same network.
I already tried setting the bind IP to '0.0.0.0', and if I try connecting I get socket error 10060. I think the problem probably has something to do with port-forwarding, but I don't know how to do it in Python.
Here is the code I used to test it, and I don't know how I get the public IP from my device in Python (not the gethostbyname() function, that only returns the private IP).
I hope you can help me.
Server.py
HOST = '0.0.0.0'
PORT = 5000
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind((HOST, PORT))
s.listen()
conn, addr = s.accept()
with conn:
while True:
text = input("Enter text: ").encode("utf-8")
send_msg(conn,text)
echo = recv_msg(conn).decode("utf-8")
print(echo)
client.py
#HOST = '192.168.0.220'
HOST = 'x.x.x.x' #ip address of server
PORT = 5000`
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect((HOST, PORT))
while True:
data = recv_msg(s)
send_msg(s,data)
def send_msg(sock, msg):
msg = struct.pack('>Q', len(msg)) + msg
sock.sendall(msg)
def recv_msg(sock):
raw_msglen = recvall(sock, 8)
if not raw_msglen:
return None
msglen = struct.unpack('>Q', raw_msglen)[0]
return recvall(sock, msglen)
def recvall(sock: socket.socket, n):
data = bytearray()
while len(data) < n:
packet = sock.recv(n - len(data))
if not packet:
return None
data.extend(packet)
return data

The code shown is fine. bind()'ing the server to 0.0.0.0 is typically the correct thing to do, that allows it to listen on all local IPs installed on its machine. Or, you can bind() it to the specific IP of the adapter that is connected to your network. Either way is fine.
On the client side, if the client is on the same network, it can connect() to the server's LAN IP, and all is well.
The problem comes when the client is not on the same network. That means the client simply cannot connect to your server directly. There is no physical route to facilitate that connection. So, to remedy that, you must setup a Port Forwarding rule in your LAN router. You must open a public IP/Port on the router's WAN, and map it to the server's LAN IP/Port. Then the client will be able to connect to the router's WAN IP/Port and let the router forward the packets to the server's LAN IP/Port, and vice versa.
If your router supports uPNP, your server code can setup this Port Forwarding programmably. There are uPNP libraries available, or your OS may even have APIs for that. Otherwise, your router admin must setup the Port Forwarding rule by hand.
To discover your router's WAN IP for the client to connect to, your server code can query an external site like https://whatismyip.com, or even query the router directly (if the router has an API for this purpose). Otherwise, you should subscribe to a 3rd party Dynamic DNS service that assigns you a static hostname that the client can always connect to, and then your server code, or even the router itself, can update the IP for that hostname whenever the router's WAN IP changes.

What is `conn, addr = s.accept()` in python socket?

I searched documentations and tutorials but no one talked about this, for example this is server script
import socket
server = socket.socket()
print("socket created")
server.bind(("localhost", 9999))
server.listen(3)
print("waiting for connection")
while True:
client, addr = server.accept()
print(client)
print(addr)
name = client.recv(1024).decode()
print("connected with", addr, client, name)
client.send(b"welcome bro")
client.close()
When printing client, I get this:
proto=0, laddr=('127.0.0.1', 9999), raddr=('127.0.0.1', 36182)
And addr variable :
('127.0.0.1', 36182)
Why these two variable defined by one and got two different output?
What is the logic behind scene?

From the documentation of the socked module:
socket.accept()
Accept a connection. The socket must be bound to an
address and listening for connections. The return value is a pair
(conn, address) where conn is a new socket object usable to send and
receive data on the connection, and address is the address bound to
the socket on the other end of the connection.

The script does not answer this by itself, however, I assume laddr=('127.0.0.1', 9999) is the listening address of the server-side app. That's where connections are established. the raddr is the connection port the request comes from. When you listen to a port with a server, the client uses any non-reserved port >1024 to connect to the server and this is totally random, as long as it is defined in the client-app.
So you have to different connection points for one established connection. The one port and address as the sender-side (described as raddr) and the one as the receiver side (here described as laddr - for listen)
That's basically the logic behind any TCP-related connection.

accept() function returns a socket descriptor that is connected to your TCP server. In this case, it returns a tuple of objects.
The first parameter, conn, is a socket object that you can use to send data to and receive data from the client that is connected.
The second parameter, addr, contains address information about the client that is connected(e.g., IP address and remote part).

Python, TCP sendall

How can I send to a specific client? TCP Server can send to all clients
I store the sendip and sendport, obtained using this code:
clientssendback = []
clientsocket, address = s.accept()
clientssendback.append(address)
for x in clientssendback:
sendip = str(x[0])
sendport = str(x[1])
clientsocket.sendall(bytes("hello", "utf-8"), (sendip, sendport))
I'll have so I can send it to all client there are loged into the server.
Can some one help me with that? Am I doing the right thing?

i have a few notes:
the accept() function returns a tuple. the first is the connection made between the server and client. it is not the client's socket.
this means that you establish a unique connection between every client and the server when you accept him. the transfering of data happens through this coonection.
first you need to save these coonections in a list, and loop between them, and send the message to each client
import socket
connections = []
server_socket = socket.socket() # by default it's TCP/IP
server_socket.bind((ip_address, port)
server_socket.listen(20)
conn, addr = server_socket.accept()
connections.append(conn)
now you now how to establish a connection and save it in a list. now lets send a message which is "hello" to all clients
for connection in connections:
connection.send(b"hello") # the b changes the string to bytes

How to make 2 clients connect each other directly, after having both connected a meeting-point server?

I'm writing a toy meeting-point/relay server listening on port 5555 for two clients "A" and "B".
It works like this: every byte received by the server from the firstly-connected client A will be sent to the secondly-connected client B, even if A and B don't know their respective IP:
A -----------> server <----------- B # they both connect the server first
A --"hello"--> server # A sends a message to server
server --"hello"--> B # the server sends the message to B
This code is currently working:
# server.py
import socket, time
from threading import Thread
socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket.bind(('', 5555))
socket.listen(5)
buf = ''
i = 0
def handler(client, i):
global buf
print 'Hello!', client, i
if i == 0: # client A, who sends data to server
while True:
req = client.recv(1000)
buf = str(req).strip() # removes end of line
print 'Received from Client A: %s' % buf
elif i == 1: # client B, who receives data sent to server by client A
while True:
if buf != '':
client.send(buf)
buf = ''
time.sleep(0.1)
while True: # very simple concurrency: accept new clients and create a Thread for each one
client, address = socket.accept()
print "{} connected".format(address)
Thread(target=handler, args=(client, i)).start()
i += 1
and you can test it by launching it on a server, and do two netcat connections to it: nc <SERVER_IP> 5555.
How can I then pass the information to the clients A and B that they can talk directly to each other without making the bytes transit via the server?
There are 2 cases:
General case, i.e. even if A and B are not in the same local network
Particular case where these two clients are in the same local network (example: using the same home router), this will be displayed on the server when the 2 clients will connect to the server on port 5555:
('203.0.113.0', 50340) connected # client A, router translated port to 50340
('203.0.113.0', 52750) connected # same public IP, client B, router translated port to 52750
Remark: a previous unsuccesful attempt here: UDP or TCP hole punching to connect two peers (each one behind a router)
and UDP hole punching with a third party

Since the server knows the addresses of both clients, it can send that information to them and so they would know each others adress. There are many ways the server can send this data - pickled, json-encoded, raw bytes. I think the best option is to convert the address to bytes, because the client will know exactly how many bytes to read: 4 for the IP (integer) and 2 for the port (unsigned short). We can convert an address to bytes and back with the functions below.
import socket
import struct
def addr_to_bytes(addr):
return socket.inet_aton(addr[0]) + struct.pack('H', addr[1])
def bytes_to_addr(addr):
return (socket.inet_ntoa(addr[:4]), struct.unpack('H', addr[4:])[0])
When the clients receive and decode the address, they no longer need the server, and they can establish a new connection between them.
Now we have two main otions, as far as I know.
One client acts as a server. This client would close the connection to the server and would start listening on the same port. The problem with this method is that it will only work if both clients are on the same local network, or if that port is open for incoming connections.
Hole punching. Both clients start sending and accepting data from each other simultaneously. The clients must accept data on the same address they used to connect to the rendezvous server, which is knwn to each other. That would punch a hole in the client's nat and the clients would be able to communicate directly even if they are on different networks. This proccess is expleined in detail in this article Peer-to-Peer Communication Across Network Address Translators, section 3.4 Peers Behind Different NATs.
A Python example for UDP Hole Punching:
Server:
import socket
def udp_server(addr):
soc = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
soc.bind(addr)
_, client_a = soc.recvfrom(0)
_, client_b = soc.recvfrom(0)
soc.sendto(addr_to_bytes(client_b), client_a)
soc.sendto(addr_to_bytes(client_a), client_b)
addr = ('0.0.0.0', 4000)
udp_server(addr)
Client:
import socket
from threading import Thread
def udp_client(server):
soc = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
soc.sendto(b'', server)
data, _ = soc.recvfrom(6)
peer = bytes_to_addr(data)
print('peer:', *peer)
Thread(target=soc.sendto, args=(b'hello', peer)).start()
data, addr = soc.recvfrom(1024)
print('{}:{} says {}'.format(*addr, data))
server_addr = ('server_ip', 4000) # the server's public address
udp_client(server_addr)
This code requires for the rendezvous server to have a port open (4000 in this case), and be accessible by both clients. The clients can be on the same or on different local networks. The code was tested on Windows and it works well, either with a local or a public IP.
I have experimented with TCP hole punching but I had limited success (sometimes it seems that it works, sometimes it doesn't). I can include the code if someone wants to experiment. The concept is more or less the same, both clients start sending and receiving simultaneously, and it is described in detail in Peer-to-Peer Communication Across Network Address Translators, section 4, TCP Hole Punching.
If both clients are on the same network, it will be much easier to communicate with each other. They would have to choose somehow which one will be a server, then they can create a normal server-client connection. The only problem here is that the clients must detect if they are on the same network. Again, the server can help with this problem, as it knows the public address of both clients. For example:
def tcp_server(addr):
soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
soc.bind(addr)
soc.listen()
client_a, addr_a = soc.accept()
client_b, addr_b = soc.accept()
client_a.send(addr_to_bytes(addr_b) + addr_to_bytes(addr_a))
client_b.send(addr_to_bytes(addr_a) + addr_to_bytes(addr_b))
def tcp_client(server):
soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
soc.connect(server)
data = soc.recv(12)
peer_addr = bytes_to_addr(data[:6])
my_addr = bytes_to_addr(data[6:])
if my_addr[0] == peer_addr[0]:
local_addr = (soc.getsockname()[0], peer_addr[1])
... connect to local address ...
Here the server sends two addresses to each client, the peer's public address and the client's own public address. The clients compare the two IPs, if they match then they must be on the same local network.

The accepted answer gives the solution. Here is some additional information in the case "Client A and Client B are in the same local network".
This situation can indeed be detected by the server if it notices that both clients have the same public IP.
Then the server can choose Client A as "local server", and Client B as "local client".
The server will then ask Client A for its "local network IP". Client A can find it with:
import socket
localip = socket.gethostbyname(socket.gethostname()) # example: 192.168.1.21
and then send it back to the server. The server will communicate this "local network IP" to Client B.
Then Client A will then run a "local server":
import socket
soc = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
soc.bind(('0.0.0.0', 4000))
data, client = soc.recvfrom(1024)
print("Connected client:", client)
print("Received message:", data)
soc.sendto(b"I am the server", client)
and Client B will run as a "local client":
import socket
soc = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
server = ('192.168.1.21', 4000) # this "local network IP" has been sent Client A => server => Client B
soc.sendto("I am the client", server)
data, client = soc.recvfrom(1024)
print("Received message:", data)

Python TCP socket doesn't close?

Maybe someone here will have a response for this thing which is just driving me insane.
To make it simple, I'm making a kind of proxy. Whenever it receives something, it forwards everything to a server, and sends back the response. So there is one socket always listening on port 4557 for clients, and for each incoming connection, there is a new socket created on a random port to connect to the server port 4556.
Clients <==> Proxy <==> Server
Also, there another socket which is instantiated and listening for requests coming from the server and to be forwarded to the corresponding client.
Here is an example:
Client A connects to proxy on port 4557
Proxy creates a socket to Server on port 4556
Along with that, it creates a socket listening on port 40100
Client sends stuff, forwarded to Server
Client disconnects. Close client connection and socket to server
Some time later, Server sends stuff to proxy on port 40100
Everything's forwarded to Client A (port 40100 corresponding to Client A)
And so on..
So far in my tests, I use a simple python script for sending a unique tcp packet to the proxy, along with a dump server showing received data and echoing back.
So the issue is that when a connection to the proxy is closed, the connection to the Server should also be closed with "sock.close()". However it just seems to be completely ignored. The socket remains as ESTABLISHED.
About the code now.
A few notes.
DTN and Node are respectively Server and Clients.
runCallback is called in a loop until thread dies.
finalCallback is called when the thread is dying.
Associations between remote hosts (Client), proxy ports (to Server) and proxies are kept in the dictionaries: TCPProxyHostRegister (RemoteHost => Proxy), TCPProxyPortRegister (Port => Proxy), TCPPortToHost (Port => RemoteHost).
The first class is TCPListenerThread.
It just listen on a specific port and instantiate proxies (one for each Client=>Server couple and Server=>Client couple) and forward them connections.
class TCPListenerThread(StoppableThread):
def __init__(self, tcp_port):
StoppableThread.__init__(self)
self.tcp_port = tcp_port
self.sock = socket.socket( socket.AF_INET, # Internet
socket.SOCK_STREAM ) # tcp
self.sock.bind( (LOCAL_ADDRESS, self.tcp_port) )
self.sock.listen(1)
def runCallback(self):
print "Listen on "+str(self.tcp_port)+".."
conn, addr = self.sock.accept()
if isFromDTN(addr):
tcpProxy = getProxyFromPort(tcp_port)
if not tcpProxy:
tcpProxy = TCPProxy(host, True)
else:
host = addr[0]
tcpProxy = getProxyFromHost(host)
if not tcpProxy:
tcpProxy = TCPProxy(host, False)
tcpProxy.handle(conn)
def finalCallback(self):
self.sock.close()
Now comes the TCP Proxy:
It associates a remote host (Client) with a port connecting to Server.
If it's a connection coming from a new Client, it will create a new listener (see above) for the Server and create a socket ready to forward everything to Server.
class TCPProxy():
def __init__(self, remote, isFromDTN):
#remote = port for Server or Remote host for Client
self.isFromDTN = isFromDTN
self.conn = None
#add itself to proxy registries
#If listening from a node
if not isFromDTN:
#Set node remote host
self.remoteHost = remote
TCPProxyHostRegister[self.remoteHost] = self
#Set port to DTN interface + listener
self.portToDTN = getNewTCPPort()
TCPPortToHost[self.portToDTN] = self.remoteHost
newTCPListenerThread(self.portToDTN)
#Or from DTN
else:
self.portToDTN = remote
TCPProxyPortRegister[self.portToDTN] = self
self.remoteHost = getRemoteHostFromPortTCP(self.portToDTN)
def handle(self, conn):
print "New connection!"
#shouldn't happen, but eh
if self.conn != None:
self.closeConnections()
self.conn = conn
#init socket with remote
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
if self.isFromDTN:
self.sock.connect((self.remoteHost, 4556)) #TODO: handle dynamic port..
else:
self.sock.connect((DTN_Address, DTN_TCPPort))
#handle connection in a thread
self.handlerThread = newTCPHandlerThread(self)
#handle reply in a therad
self.replyThread = newTCPReplyThread(self)
def closeConnections(self):
try:
if self.conn != None:
print "Close connections!"
self.sock.close()
self.conn.close()
self.conn = None
self.handlerThread.kill()
self.replyThread.kill()
except Exception, err:
print str(err)
#pass
def forward(self, data):
print "TCP forwarding data: "+data
self.sock.send(data)
def forwardBack(self, data):
print "TCP forwarding data back: "+data
self.conn.send(data)
In this proxy class, I instantiate two classes, TCPHandlerThread and TCPReplyThread. They are responsible for forwarding to Server, and forwarding back to Client, respectively.
class TCPHandlerThread(StoppableThread):
def __init__(self, proxy):
StoppableThread.__init__(self)
self.proxy = proxy
def runCallback(self):
test = False
while 1:
data = self.proxy.conn.recv(BUFFER_SIZE)
if test:
self.proxy.sock.close()
test = True
if not data:
break
print "TCP received data:", data
self.proxy.forward(data)
self.kill()
def finalCallback(self):
self.proxy.closeConnections()
class TCPReplyThread(StoppableThread):
def __init__(self, proxy):
StoppableThread.__init__(self)
self.proxy = proxy
def runCallback(self):
while 1:
data = self.proxy.sock.recv(BUFFER_SIZE)
if not data:
break
print "TCP received back data: "+data
self.proxy.forwardBack(data)
self.kill()
def finalCallback(self):
self.proxy.closeConnections()
You see that whenever a connection is closed, the thread dies and the other connection (Client/Server to proxy or Proxy to Server/Client) should be closed in Proxy.closeConnections()
I noticed that when closeConnections() is "data = self.proxy.conn.recv(BUFFER_SIZE)", it goes well, but when it's called even right after the latter statement, it goes wrong.
I wiresharked TCP, and the proxy doesn't send any "bye signal". The socket state doesn't go to TIME_WAIT or whatever, it just remains ESTABLISHED.
Also, I tested it on Windows and Ubuntu.
On Windows it goes exactly as I explained
On Ubuntu, it works well for usually (not always), 2 connections, and the third time I connect with the same client in exactly the same way to the proxy, it goes wrong again exactly as explained.
Here are the three files i'm using so that you can have a look at the whole code. I'm sorry the proxy file might not be really easy to read. Was SUPPOSED to be a quick dev.
http://hognerud.net/stackoverflow/
Thanks in advance..
It's surely something stupid. Please don't hit me too hard when you see it :(

First I'm sorry that I currently have not the time to actually run and test your code.
But the idea came to my mind, that your problem might actually have something todo with using blocking mode vs. non-blocking mode on the socket. In that case you should checkout the "socket" module help in the python documentation, especially socket.setblocking().
My guess is, that the proxy.conn.recv() function only returns, when actually BUFFER_SIZE bytes where received by the socket. Because of this the thread is blocked until enough data was received and therefore the socket doesn't get closed.
As I said first, this is currently just a guess, so please don't vote me down if it doesn't solve the problem...