I tried setting up a TCP socket with Python, and it works totally fine as long as I'm in the same network. But my problem is that I can't get it to work if one of the devices isn't in the same network.
I already tried setting the bind IP to '0.0.0.0', and if I try connecting I get socket error 10060. I think the problem probably has something to do with port-forwarding, but I don't know how to do it in Python.
Here is the code I used to test it, and I don't know how I get the public IP from my device in Python (not the gethostbyname() function, that only returns the private IP).
I hope you can help me.
Server.py
HOST = '0.0.0.0'
PORT = 5000
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind((HOST, PORT))
s.listen()
conn, addr = s.accept()
with conn:
while True:
text = input("Enter text: ").encode("utf-8")
send_msg(conn,text)
echo = recv_msg(conn).decode("utf-8")
print(echo)
client.py
#HOST = '192.168.0.220'
HOST = 'x.x.x.x' #ip address of server
PORT = 5000`
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect((HOST, PORT))
while True:
data = recv_msg(s)
send_msg(s,data)
def send_msg(sock, msg):
msg = struct.pack('>Q', len(msg)) + msg
sock.sendall(msg)
def recv_msg(sock):
raw_msglen = recvall(sock, 8)
if not raw_msglen:
return None
msglen = struct.unpack('>Q', raw_msglen)[0]
return recvall(sock, msglen)
def recvall(sock: socket.socket, n):
data = bytearray()
while len(data) < n:
packet = sock.recv(n - len(data))
if not packet:
return None
data.extend(packet)
return data
The code shown is fine. bind()'ing the server to 0.0.0.0 is typically the correct thing to do, that allows it to listen on all local IPs installed on its machine. Or, you can bind() it to the specific IP of the adapter that is connected to your network. Either way is fine.
On the client side, if the client is on the same network, it can connect() to the server's LAN IP, and all is well.
The problem comes when the client is not on the same network. That means the client simply cannot connect to your server directly. There is no physical route to facilitate that connection. So, to remedy that, you must setup a Port Forwarding rule in your LAN router. You must open a public IP/Port on the router's WAN, and map it to the server's LAN IP/Port. Then the client will be able to connect to the router's WAN IP/Port and let the router forward the packets to the server's LAN IP/Port, and vice versa.
If your router supports uPNP, your server code can setup this Port Forwarding programmably. There are uPNP libraries available, or your OS may even have APIs for that. Otherwise, your router admin must setup the Port Forwarding rule by hand.
To discover your router's WAN IP for the client to connect to, your server code can query an external site like https://whatismyip.com, or even query the router directly (if the router has an API for this purpose). Otherwise, you should subscribe to a 3rd party Dynamic DNS service that assigns you a static hostname that the client can always connect to, and then your server code, or even the router itself, can update the IP for that hostname whenever the router's WAN IP changes.
Related
I have been working on a project to connect computers located in different locations together through Python. Initially, while testing, I used my private IP address (I did not know it was private at the time) to connect computers on the same network as mine. But as soon as I tried doing this with computers located on different networks in different locations, it simply did not work.
And I assume this is because the program is using the local IP address of my computer that can connect only to computers on the same network. Here are my simplified programs:
Here is my server-side script:
server = socket.gethostbyname(socket.gethostname()) # 10.128.X.XXX which is the Internal IP
print(server)
port = 5555
clients = 0
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((server, port))
s.listen(2)
print("Waiting for connection...")
while True:
conn, addr = s.accept()
print("Connected to: ", addr)
conn.send(str.encode(f"{clients}"))
clients += 1
and here is my client side-script:
class Network:
def __init__(self):
self.client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.server = "10.128.0.2"
self.port = 5555
self.addr = (self.server, self.port)
self.id = int(self.connect())
def connect(self):
self.client.connect(self.addr)
return self.client.recv(2048).decode()
network = Network()
print(f"Connected as client {network.id}")
Now when I tried replacing the private IP address with the global IP address (as specified here: How do I get the external IP of a socket in Python?) I got the following error:
# Getting the Global IP Address
from requests import get
server = get("https://api.ipify.org").text
s.bind((server, port))
OSError: [WinError 10049] The requested address is not valid in its context
I have tried searching a lot on how to communicate (transfer small amounts of data as strings) between multiple computers located in different locations using different networks, but I haven't really gotten a solution. Is there a way that I can do this?
In server you always use local IP (it is IP of one of network cards in computer or 0.0.0.0 to use all network cards)
s.bind( (local_IP, port) )
# or
s.bind( ('0.0.0.0', port) )
In client you use external IP
s.connect( (external_IP, port) )
External client uses external IP to connect with your Internet Provider route and this router knows that this external IP is assigned to your computer and it redirects it your server.
At the same time local client can use local IP to connect with the same server.
external_client --> router(externa_IP) --> server(local_IP) <-- local_client
I'm writing a toy meeting-point/relay server listening on port 5555 for two clients "A" and "B".
It works like this: every byte received by the server from the firstly-connected client A will be sent to the secondly-connected client B, even if A and B don't know their respective IP:
A -----------> server <----------- B # they both connect the server first
A --"hello"--> server # A sends a message to server
server --"hello"--> B # the server sends the message to B
This code is currently working:
# server.py
import socket, time
from threading import Thread
socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket.bind(('', 5555))
socket.listen(5)
buf = ''
i = 0
def handler(client, i):
global buf
print 'Hello!', client, i
if i == 0: # client A, who sends data to server
while True:
req = client.recv(1000)
buf = str(req).strip() # removes end of line
print 'Received from Client A: %s' % buf
elif i == 1: # client B, who receives data sent to server by client A
while True:
if buf != '':
client.send(buf)
buf = ''
time.sleep(0.1)
while True: # very simple concurrency: accept new clients and create a Thread for each one
client, address = socket.accept()
print "{} connected".format(address)
Thread(target=handler, args=(client, i)).start()
i += 1
and you can test it by launching it on a server, and do two netcat connections to it: nc <SERVER_IP> 5555.
How can I then pass the information to the clients A and B that they can talk directly to each other without making the bytes transit via the server?
There are 2 cases:
General case, i.e. even if A and B are not in the same local network
Particular case where these two clients are in the same local network (example: using the same home router), this will be displayed on the server when the 2 clients will connect to the server on port 5555:
('203.0.113.0', 50340) connected # client A, router translated port to 50340
('203.0.113.0', 52750) connected # same public IP, client B, router translated port to 52750
Remark: a previous unsuccesful attempt here: UDP or TCP hole punching to connect two peers (each one behind a router)
and UDP hole punching with a third party
Since the server knows the addresses of both clients, it can send that information to them and so they would know each others adress. There are many ways the server can send this data - pickled, json-encoded, raw bytes. I think the best option is to convert the address to bytes, because the client will know exactly how many bytes to read: 4 for the IP (integer) and 2 for the port (unsigned short). We can convert an address to bytes and back with the functions below.
import socket
import struct
def addr_to_bytes(addr):
return socket.inet_aton(addr[0]) + struct.pack('H', addr[1])
def bytes_to_addr(addr):
return (socket.inet_ntoa(addr[:4]), struct.unpack('H', addr[4:])[0])
When the clients receive and decode the address, they no longer need the server, and they can establish a new connection between them.
Now we have two main otions, as far as I know.
One client acts as a server. This client would close the connection to the server and would start listening on the same port. The problem with this method is that it will only work if both clients are on the same local network, or if that port is open for incoming connections.
Hole punching. Both clients start sending and accepting data from each other simultaneously. The clients must accept data on the same address they used to connect to the rendezvous server, which is knwn to each other. That would punch a hole in the client's nat and the clients would be able to communicate directly even if they are on different networks. This proccess is expleined in detail in this article Peer-to-Peer Communication Across Network Address Translators, section 3.4 Peers Behind Different NATs.
A Python example for UDP Hole Punching:
Server:
import socket
def udp_server(addr):
soc = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
soc.bind(addr)
_, client_a = soc.recvfrom(0)
_, client_b = soc.recvfrom(0)
soc.sendto(addr_to_bytes(client_b), client_a)
soc.sendto(addr_to_bytes(client_a), client_b)
addr = ('0.0.0.0', 4000)
udp_server(addr)
Client:
import socket
from threading import Thread
def udp_client(server):
soc = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
soc.sendto(b'', server)
data, _ = soc.recvfrom(6)
peer = bytes_to_addr(data)
print('peer:', *peer)
Thread(target=soc.sendto, args=(b'hello', peer)).start()
data, addr = soc.recvfrom(1024)
print('{}:{} says {}'.format(*addr, data))
server_addr = ('server_ip', 4000) # the server's public address
udp_client(server_addr)
This code requires for the rendezvous server to have a port open (4000 in this case), and be accessible by both clients. The clients can be on the same or on different local networks. The code was tested on Windows and it works well, either with a local or a public IP.
I have experimented with TCP hole punching but I had limited success (sometimes it seems that it works, sometimes it doesn't). I can include the code if someone wants to experiment. The concept is more or less the same, both clients start sending and receiving simultaneously, and it is described in detail in Peer-to-Peer Communication Across Network Address Translators, section 4, TCP Hole Punching.
If both clients are on the same network, it will be much easier to communicate with each other. They would have to choose somehow which one will be a server, then they can create a normal server-client connection. The only problem here is that the clients must detect if they are on the same network. Again, the server can help with this problem, as it knows the public address of both clients. For example:
def tcp_server(addr):
soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
soc.bind(addr)
soc.listen()
client_a, addr_a = soc.accept()
client_b, addr_b = soc.accept()
client_a.send(addr_to_bytes(addr_b) + addr_to_bytes(addr_a))
client_b.send(addr_to_bytes(addr_a) + addr_to_bytes(addr_b))
def tcp_client(server):
soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
soc.connect(server)
data = soc.recv(12)
peer_addr = bytes_to_addr(data[:6])
my_addr = bytes_to_addr(data[6:])
if my_addr[0] == peer_addr[0]:
local_addr = (soc.getsockname()[0], peer_addr[1])
... connect to local address ...
Here the server sends two addresses to each client, the peer's public address and the client's own public address. The clients compare the two IPs, if they match then they must be on the same local network.
The accepted answer gives the solution. Here is some additional information in the case "Client A and Client B are in the same local network".
This situation can indeed be detected by the server if it notices that both clients have the same public IP.
Then the server can choose Client A as "local server", and Client B as "local client".
The server will then ask Client A for its "local network IP". Client A can find it with:
import socket
localip = socket.gethostbyname(socket.gethostname()) # example: 192.168.1.21
and then send it back to the server. The server will communicate this "local network IP" to Client B.
Then Client A will then run a "local server":
import socket
soc = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
soc.bind(('0.0.0.0', 4000))
data, client = soc.recvfrom(1024)
print("Connected client:", client)
print("Received message:", data)
soc.sendto(b"I am the server", client)
and Client B will run as a "local client":
import socket
soc = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
server = ('192.168.1.21', 4000) # this "local network IP" has been sent Client A => server => Client B
soc.sendto("I am the client", server)
data, client = soc.recvfrom(1024)
print("Received message:", data)
I am trying to set up a UDP unicast between two linux-machines on my local network, using the python sockets library. I manage to send and receive the package using the following code:
Send
import socket
HOST = '192.168.1.194' # IP of remote machine
PORT = 47808
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.sendto('Hello UDP', (HOST, PORT))
s.close()
Receive
import socket
HOST = ''
PORT = 47808
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.bind((HOST, PORT))
while True:
try:
data, addr = s.recvfrom(1024)
print 'Received: {0} # {1}'.format(data, addr)
except KeyboardInterrupt:
break
s.close()
However, binding to '' makes the receiving code accept packets from any local interface. If I try to bind to the IP address of the sending machine specifically (changing HOST = ''to HOST = '192.168.1.130' in the receiving code), I get a socket.error: [Errno 99] Cannot assign requested address. No other services are using the port, and I have tried different ports with no change in behaviour. How can I configure my socket to only receive packets from a specific address?
First, let's deal with the error you are seeing. .bind() names the local end of the socket, not the remote. So the host part must refer to the local machine (e.g., 'localhost', '127.0.0.1, '192.168.1.194', or '' (wildcard for all local interfaces).) So, when you specify an address that isn't local to the machine running .bind(), you get an error.
Second, there is no way to "configure my socket to only receive packets from a specific address." As an alternative, you can use the returned address from .recvfrom() to ignore data you don't care about.
data, addr = s.recvfrom(1024)
if addr != '192.168.1.130':
continue
I am new here, so please, don't be angry if I am stupid - but I don't know about it.
I would like to make a python TCP server, which can be accessed from anywhere via external (public) IP. I have done simple server (it works) in local network from this tutorial:
https://www.youtube.com/watch?v=XiVVYfgDolU
The client sends string and the server sends back that string but with uppercase.
Now I want to do the same but accessable from anywhere. I read a lot about it. I have Raspberry Pi, where I set up static IP address and I did the port forward (on port 42424). I am just looking for some tutorial, you can direct me anywhere - thats all I need. Or you can tell me how to do it step by step, but I know that it takes a lot of time to write answer. I tried googling, but I didn't find anything. And if I did, it was a person who didn't know what the external IP and the port forwarding is so the end of the conversation was: Learn what is port forwarding.
So please, give me some tips how to do it, or direct me somewhere. Thanks!
The code
Server:
import socket
def Main():
host = '10.0.0.140'
port = 42424
s = socket.socket()
s.bind((host, port))
s.listen(1)
c, addr = s.accept()
while True:
data = c.recv(1024)
if not data:
break
data = str(data).upper()
c.send(data)
c.close()
if __name__ == '__main__':
Main()
Client:
import socket
def Main():
host = '10.0.0.140'
port = 42424
s = socket.socket()
s.connect((host,port))
message = raw_input("->")
while message != 'q':
s.send(message)
data = s.recv(1024)
message = raw_input("->")
s.close()
if __name__ == '__main__':
Main()
When connecting to a server behind a NAT firewall/router, in addition to port forwarding, the client should be directed to the IP address of the router. As far as the client is concerned, the IP address of the router is the server. The router simply forwards the traffic according to the port forwarding rules.
I want ro receive some data that is sent as a UDP packet over VPN. So wrote (mostly copied) this program in python:
import socket
import sys
HOST = ???????
PORT = 80
# SOCK_DGRAM is the socket type to use for UDP sockets
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((HOST,PORT))
data,addr = sock.recv(1024)
print "Received: %s" % data
print "Addr: %s" % addr
What should I use as host? I know the IP of the sender but it seems anything thats not local gives me socket.error: [Errno 10049]. The IP that the VPN gives me (the same IP that the sender sends to, that is)? Or just localhost?
The host argument is the host IP you want to bind to. Specify the IP of one of your interfaces (Eg, your public IP, or 127.0.0.1 for localhost), or use 0.0.0.0 to bind to all interfaces. If you bind to a specific interface, your service will only be available on that interface - for example, if you want to run something that can only be accessed via localhost, or if you have multiple IPs and need to run different servers on each.
"0.0.0.0" will listen for all incoming hosts. For example,
sock.bind(("0.0.0.0", 999))
data,addr = sock.recv(1024)
Use:
sock.bind(("", 999))