So, me and my friend just wanted to try creating our own personal chat application in python I started by trying to make a CLI application first, I googled a few things but I am unable to find out how to send files over Internet . There are a lot of ways to send it over LAN but me and my friend are on different networks so that won't work. I can't seem to find a way to do it over the internet. I am new to networking so please pardon if I have made any mistake. Here's the code I have to transfer files over LAN:
client.py
import socket
from threading import Thread
from datetime import datetime
# server's IP address
# if the server is not on this machine,
# put the private (network) IP address (e.g 192.168.1.2)
SERVER_HOST = input("Server IP : ")
SERVER_PORT = 5002 # server's port
separator_token = "<SEP>" # we will use this to separate the client name & message
# initialize TCP socket
s = socket.socket()
print(f"[*] Connecting to {SERVER_HOST}:{SERVER_PORT}...")
# connect to the server
s.connect((SERVER_HOST, SERVER_PORT))
print("[+] Connected.")
# prompt the client for a name
name = input("Enter your name: ")
def listen_for_messages():
while True:
message = s.recv(1024).decode()
print("\n" + message)
# make a thread that listens for messages to this client & print them
t = Thread(target=listen_for_messages)
# make the thread daemon so it ends whenever the main thread ends
t.daemon = True
# start the thread
t.start()
while True:
# input message we want to send to the server
to_send = input()
# a way to exit the program
if to_send.lower() == 'q':
break
# add the datetime, name & the color of the sender
date_now = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
to_send = f"[{date_now}] {name}{separator_token}{to_send}"
# finally, send the message
s.send(to_send.encode())
# close the socket
s.close()
server.py
import socket
from threading import Thread
# server's IP address
SERVER_HOST = "0.0.0.0"
SERVER_PORT = 5002 # port we want to use
separator_token = "<SEP>" # we will use this to separate the client name & message
# initialize list/set of all connected client's sockets
client_sockets = set()
# create a TCP socket
s = socket.socket()
# make the port as reusable port
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
# bind the socket to the address we specified
s.bind((SERVER_HOST, SERVER_PORT))
# listen for upcoming connections
s.listen(5)
print(f"[*] Listening as {SERVER_HOST}:{SERVER_PORT}")
def listen_for_client(cs):
"""
This function keep listening for a message from `cs` socket
Whenever a message is received, broadcast it to all other connected clients
"""
while True:
try:
# keep listening for a message from `cs` socket
msg = cs.recv(1024).decode()
except Exception as e:
# client no longer connected
# remove it from the set
print(f"[!] Error: {e}")
client_sockets.remove(cs)
else:
# if we received a message, replace the <SEP>
# token with ": " for nice printing
msg = msg.replace(separator_token, ": ")
# iterate over all connected sockets
for client_socket in client_sockets:
# and send the message
client_socket.send(msg.encode())
while True:
# we keep listening for new connections all the time
client_socket, client_address = s.accept()
print(f"[+] {client_address} connected.")
# add the new connected client to connected sockets
client_sockets.add(client_socket)
# start a new thread that listens for each client's messages
t = Thread(target=listen_for_client, args=(client_socket,))
# make the thread daemon so it ends whenever the main thread ends
t.daemon = True
# start the thread
t.start()
Your code is fine. The issue is the network setup.
I assume you both have typical setups, so you are both behind NAT (network address translation). Other chat apps use a server that you can both connect to. If you put your friend's IP address you will try to open a connection to the router which likely won't answer.
One hack is to setup port forwarding on one of your routers. Say 5002, from your example, and forward all incoming traffic to the right local machine. This should get you running, but I agree it's not a easy thing to get working. An easier setup would be to use upnp which sets up the port forwarding rules automatically, but I don't know which package would do that (there must be one somewhere).
Another way would be to switch to ipv6, if your providers support it. Because of the increase in address space it's common to give out ipv6 ranges to customers, so that local machines can have public ipv6 addresses, which you can use to connect to. Here packages should be routed correctly, but firewalls might interfere.
Yet another hack, would be to bypass routers entirely and plug your machine directly into your uplink connection. This depends on your provider and network setup. This won't work if you have a router that also acts as a modem to connect over TV cable or telephone wires.
Diclaimer: this may not be a true answer (at least it is not the expected one), but it could not fit in a comment.
Luckily you gave some context about your real problem! Because it is neither a Python question nor a file transfer one. It is about how to exchange data between 2 private networks each protected with Network Address Translation by their ISP router. And there is no simple and direct answer, only possible workaround.
In fact the correct solution would be that either you or a friend of yours use a (virtual) machine in the cloud and installs the server there. That way the server woud have a true address and could easily be accessed from any of your clients. It has some drawback and pitfalls because you now have an application publicly accessible and always up, so security has to be considered...
The possible workarounds depends on your ISP routers. Some allow to configure a local address to be a DMZ that is an address that is accessible from both the internet and the rest of your LAN but only on certain ports. The rationale behind is to reduce as much as possible the attack surface: the machine in the DMZ will be vulnerable, but it should not contain sensitive data and should be easy to rebuild from scratch if compromised. But it only makes sense if you (or one of your friends) can dedicate a physical machine (raspberry or equivalent are not so expensive). Others allow to configure a special port to be automatically routed to an internal address. You lose the security benefit of a DMZ but no longer need to dedicate anything.
Unfortunately, as I initialy said, I have no code fix to propose because the probem is not there, nor any direct solution because the workarounds really depend on your ISP routers.
Related
I'm making python binds for Blackmagic's Ethernet Control Protocol ( as documented in https://documents.blackmagicdesign.com/UserManuals/HyperDeckManual.pdf?_v=1528269592000 , page 60). Simple socket connection seems to fail however, because every commands gets rejected with the server's greeting.
This protocol documents how software can communicate with certain blackmagic devices, in this case, Blackmagic's hyperdeck, the device runs a TCP server constantly listening on port 9993, on cmd I can simply telnet to it and issue commands, you'd it expect it to be as straightforward in python, however every command gets ignored for the server's greeting message, the device's information. I have been doing socket's for at least 3 months now and i've tried several methods of code, and all seem to fail.
For the most trivial test i've used:
import socket
HOST = "device's ip"
PORT = 9993
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
s.sendall(b'play')
data = s.recv(1024)
print(data)
and a modified version to try to repeat the command:
import socket
import time
HOST = "device's ip"
PORT = 9993
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
s.sendall(b'play')
data = s.recv(1024)
time.sleep(2)
s.sendall(b'play')
It should start video playback, as documented, and as occurs when I issue the command thru telnet, however the command is completely ignored and data always equals to: b'500 connection info:\r\nprotocol version: 1.9\r\nmodel: HyperDeck Studio Mini\r\n\r\n' , the server's greeting message in byte form, it should instead be 200 ok or some sort of error / acknowledged message, as documented.
This is incredibly annoying and i've thought of using subprocess and issuing commands thru cmd as an alternative, but something tells me there's an easier workaround.
I'm trying to make a Python server where multiple clients can connect but I've run into a problem I tried everything that I found on the internet.
I'm running a laptop whit windows 7 and an I3 processor.
This is the file called tcp:
import socket
def make_server (ip,port):
try:
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((ip, port))
server.listen(1)
return server
except Exception as ex:
print(ex)
return None
def accept(server):
conn, addr = server.accept()
return conn
def make_client():
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
return client
def client_connect(client,ip,port):
client.connect((ip,port))
def sendall(conn,mess):
conn.send(str(mess).encode("utf-8"))
def rec(conn,rate):
mess = conn.recv(rate).decode("utf-8")
return mess
def close(client):
client.close()
This is the server:
from multiprocessing import Process
from random import randint
import tcp
import sys
def start(sip, sport):
print("Making sob server...")
print("id= {}".format(sport))
sserver = tcp.make_server(sip, sport)
print("Sub Server Started!")
sconn = tcp.accept(sserver)
tcp.sendall(sconn, "connected!!")
while True:
try:
tcp.sendall(sconn, randint(0, 100))
except Exception as ex:
print("")
print("From server {} error:".format(port))
print(ex)
print("")
break
ip = "192.168.0.102"
port = 8000
subport = 9000
server = tcp.make_server(ip, port)
if server is None:
sys.exit(0)
print("Started!")
while True:
print("Wating for new connection!")
con = tcp.accept(server)
print("Connected!")
subport = subport + 1
tcp.sendall(con, subport)
print("New Port Sent!")
print("New Port = {}".format(subport))
subs = Process(target=start, args=(ip, subport))
subs.start()
subs.join()
This is the client:
import tcp
import time
nport = 0
ip = "192.168.0.102"
port = 8000
client = tcp.make_client()
tcp.client_connect(client,ip,port)
nport = tcp.rec(client,1024)
print(nport)
tcp.close(client)
nport = int(nport)
time.sleep(1)
print(nport)
client = tcp.make_client()
tcp.client_connect(client,ip,nport)
while True:
mess = tcp.rec(client, 1024)
if(mess):
print(mess)
The error is:
[WinError 10048]Only one usage of each socket address (protocol/network address/port) is normally permitted Python
Feel free to change anything you want.
If you need any info in plus just ask.
You are creating a socket in the client with tcp.make_client. You are then using that socket to connect to the server via tcp.client_connect. Presumably you successfully receive the new port number back from the server. But then you are trying to re-use the same socket to connect to those ports.
This is the proximate cause of your error: A socket can only be used for a single TCP connection. If you want to create a new connection, you must first create a new socket.
That being said, if you are simply trying to create a server that will accept multiple connections, you're making it way too complicated. The server can receive any number of connections on its single listening port, as long as a different address/port combination is used by each client.
One way to structure this in a server is something like this:
# Create and bind listening socket
lsock = socket.socket()
lsock.bind(('', port))
lsock.listen(1)
while True:
csock, addr = lsock.accept()
print("Got connection from {}".format(addr))
# Start sub-process passing it the newly accepted socket as argument
subs = Process(target=start, args=(csock, ))
subs.start()
# Close our handle to the new socket (it will remain open in the
# sub-process which will use it to talk to the client)
csock.close()
# NOTE: do not call subs.join here unless you want the parent to *block*
# waiting for the sub-process to finish (and if so, what is the point in
# creating a sub-process?)
There are several others ways to do it as well: you can create multiple threads to handle multiple connections, or you can handle all connections in a single thread by using select or with asynchronous I/O.
The client is typically much simpler -- as it usually only cares about its own one connection -- and doesn't care which way the server is implemented:
sock = socket.socket()
sock.connect((ip, port))
while True:
sock.send(...)
sock.recv(...)
If the client does wish to connect to the same server again, it simply creates a second socket and call its connect method with the same server IP and port.
Usually, the client never needs to specify its own port, only the server's port. It simply calls connect and the client-side operating system chooses an unused port for it. So the first time, the client creates a socket and connects it (to the server's listening port), the client-side OS may choose port 50001. The next time it creates and connects a socket, it may get 50002 and so on. (The exact port numbers chosen depend on the operating system implementation and other factors, such as what other programs are running and creating connections.)
So, given client IP 192.168.0.101 and server IP 192.168.0.102, and assuming the server is listening on port 8000, this would result in these two connections:
(192.168.0.101/50001) ====> (192.168.0.102/8000)
(192.168.0.101/50002) ====> (192.168.0.102/8000)
I'm writing a toy meeting-point/relay server listening on port 5555 for two clients "A" and "B".
It works like this: every byte received by the server from the firstly-connected client A will be sent to the secondly-connected client B, even if A and B don't know their respective IP:
A -----------> server <----------- B # they both connect the server first
A --"hello"--> server # A sends a message to server
server --"hello"--> B # the server sends the message to B
This code is currently working:
# server.py
import socket, time
from threading import Thread
socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket.bind(('', 5555))
socket.listen(5)
buf = ''
i = 0
def handler(client, i):
global buf
print 'Hello!', client, i
if i == 0: # client A, who sends data to server
while True:
req = client.recv(1000)
buf = str(req).strip() # removes end of line
print 'Received from Client A: %s' % buf
elif i == 1: # client B, who receives data sent to server by client A
while True:
if buf != '':
client.send(buf)
buf = ''
time.sleep(0.1)
while True: # very simple concurrency: accept new clients and create a Thread for each one
client, address = socket.accept()
print "{} connected".format(address)
Thread(target=handler, args=(client, i)).start()
i += 1
and you can test it by launching it on a server, and do two netcat connections to it: nc <SERVER_IP> 5555.
How can I then pass the information to the clients A and B that they can talk directly to each other without making the bytes transit via the server?
There are 2 cases:
General case, i.e. even if A and B are not in the same local network
Particular case where these two clients are in the same local network (example: using the same home router), this will be displayed on the server when the 2 clients will connect to the server on port 5555:
('203.0.113.0', 50340) connected # client A, router translated port to 50340
('203.0.113.0', 52750) connected # same public IP, client B, router translated port to 52750
Remark: a previous unsuccesful attempt here: UDP or TCP hole punching to connect two peers (each one behind a router)
and UDP hole punching with a third party
Since the server knows the addresses of both clients, it can send that information to them and so they would know each others adress. There are many ways the server can send this data - pickled, json-encoded, raw bytes. I think the best option is to convert the address to bytes, because the client will know exactly how many bytes to read: 4 for the IP (integer) and 2 for the port (unsigned short). We can convert an address to bytes and back with the functions below.
import socket
import struct
def addr_to_bytes(addr):
return socket.inet_aton(addr[0]) + struct.pack('H', addr[1])
def bytes_to_addr(addr):
return (socket.inet_ntoa(addr[:4]), struct.unpack('H', addr[4:])[0])
When the clients receive and decode the address, they no longer need the server, and they can establish a new connection between them.
Now we have two main otions, as far as I know.
One client acts as a server. This client would close the connection to the server and would start listening on the same port. The problem with this method is that it will only work if both clients are on the same local network, or if that port is open for incoming connections.
Hole punching. Both clients start sending and accepting data from each other simultaneously. The clients must accept data on the same address they used to connect to the rendezvous server, which is knwn to each other. That would punch a hole in the client's nat and the clients would be able to communicate directly even if they are on different networks. This proccess is expleined in detail in this article Peer-to-Peer Communication Across Network Address Translators, section 3.4 Peers Behind Different NATs.
A Python example for UDP Hole Punching:
Server:
import socket
def udp_server(addr):
soc = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
soc.bind(addr)
_, client_a = soc.recvfrom(0)
_, client_b = soc.recvfrom(0)
soc.sendto(addr_to_bytes(client_b), client_a)
soc.sendto(addr_to_bytes(client_a), client_b)
addr = ('0.0.0.0', 4000)
udp_server(addr)
Client:
import socket
from threading import Thread
def udp_client(server):
soc = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
soc.sendto(b'', server)
data, _ = soc.recvfrom(6)
peer = bytes_to_addr(data)
print('peer:', *peer)
Thread(target=soc.sendto, args=(b'hello', peer)).start()
data, addr = soc.recvfrom(1024)
print('{}:{} says {}'.format(*addr, data))
server_addr = ('server_ip', 4000) # the server's public address
udp_client(server_addr)
This code requires for the rendezvous server to have a port open (4000 in this case), and be accessible by both clients. The clients can be on the same or on different local networks. The code was tested on Windows and it works well, either with a local or a public IP.
I have experimented with TCP hole punching but I had limited success (sometimes it seems that it works, sometimes it doesn't). I can include the code if someone wants to experiment. The concept is more or less the same, both clients start sending and receiving simultaneously, and it is described in detail in Peer-to-Peer Communication Across Network Address Translators, section 4, TCP Hole Punching.
If both clients are on the same network, it will be much easier to communicate with each other. They would have to choose somehow which one will be a server, then they can create a normal server-client connection. The only problem here is that the clients must detect if they are on the same network. Again, the server can help with this problem, as it knows the public address of both clients. For example:
def tcp_server(addr):
soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
soc.bind(addr)
soc.listen()
client_a, addr_a = soc.accept()
client_b, addr_b = soc.accept()
client_a.send(addr_to_bytes(addr_b) + addr_to_bytes(addr_a))
client_b.send(addr_to_bytes(addr_a) + addr_to_bytes(addr_b))
def tcp_client(server):
soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
soc.connect(server)
data = soc.recv(12)
peer_addr = bytes_to_addr(data[:6])
my_addr = bytes_to_addr(data[6:])
if my_addr[0] == peer_addr[0]:
local_addr = (soc.getsockname()[0], peer_addr[1])
... connect to local address ...
Here the server sends two addresses to each client, the peer's public address and the client's own public address. The clients compare the two IPs, if they match then they must be on the same local network.
The accepted answer gives the solution. Here is some additional information in the case "Client A and Client B are in the same local network".
This situation can indeed be detected by the server if it notices that both clients have the same public IP.
Then the server can choose Client A as "local server", and Client B as "local client".
The server will then ask Client A for its "local network IP". Client A can find it with:
import socket
localip = socket.gethostbyname(socket.gethostname()) # example: 192.168.1.21
and then send it back to the server. The server will communicate this "local network IP" to Client B.
Then Client A will then run a "local server":
import socket
soc = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
soc.bind(('0.0.0.0', 4000))
data, client = soc.recvfrom(1024)
print("Connected client:", client)
print("Received message:", data)
soc.sendto(b"I am the server", client)
and Client B will run as a "local client":
import socket
soc = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
server = ('192.168.1.21', 4000) # this "local network IP" has been sent Client A => server => Client B
soc.sendto("I am the client", server)
data, client = soc.recvfrom(1024)
print("Received message:", data)
I'm trying to learn something about socket programming in python.
I have written a server which identifies clients first by their IPs , I want to test the program on localhost, but have no idea how to give different IPs to clients. As I know when we use connect() method, we only specify the host and port of the destination, but what if we want to choose ours?! is there any way?
HOST = '127.0.0.1'
PORT = 8018
TIMEOUT = 5
BUF_SIZE = 1024
class WhatsUpClient():
def __init__(self, host=HOST, port=PORT):
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock.connect((host, port))
logging.info('Connecting to %s:%s' % (host, port))
while 1:
try:
buf = self.sock.recv(BUF_SIZE)
sys.stdout.write(buf)
cmd = raw_input()
if cmd.strip() == '!q':
sys.exit(1)
self.sock.send(cmd)
except:
self.sock.close()
def run(self):
pass
An IP address needs to be unique for the whole world (with a few exceptions), so you can't arbitrarily pick one and use it.
You can look at is private networks (192.168.x.x and 10.x.x.x). The easiest way to set this up is to install a virtual machine (like VirtualBox) on your computer. When you look at the active network interfaces, you'll see that the VM added a few virtual network cards.
If you run your client on the virtual PC, it will get a special, private IP address.
You could also rent a PC in the cloud and run your code there but that costs money.
Another option is to ask your system administrator whether they have any free IP addresses which you can use for testing but usually they don't or they don't want to tell you since it's a bit complicated to set this up.
I'm building a game-server in Python. The functionality is pretty well-defined. The server will listen on the port 6000 and a remote client will send a request. Then the server will establish a connection to the client's port 7000. From then on, the client will keep sending 'requests' (basically, strings such as "UP#", "DOWN#", "SHOOT#" etc.) to server's port 6000.
This is the problem. I have made a 'server' who listens on the port 6000. This means I cannot bind a client to the same port. Is there a way that I can get the data string of an incoming request in a server? So far, I only have this.
What am I doing wrong here? Any workarounds for this issue? In short, can I read the incoming request string from a client in the server code?
Thanks in advance.
def receive_data(self):
errorOccured = False
connection = None
try:
listener = socket.socket() # Create a socket object.
host = socket.gethostname()
port = 6000 # The port that the server keeps listening to.
listener.bind(('', port))
# Start listening
listener.listen(5)
statement = ("I:P0:7,6;8,1;0,4;3,8;3,2;1,6:5,4;9,3;8,7;2,6;1,4;2,7;6,1;6,3:2,1;8,3;5,8;9,8;7,2;0,3;9,4;4,8;7,1;6,8#\n","S:P0;0,0;0#","G:P0;0,0;0;0;100;0;0:4,3,0;5,4,0;3,8,0;2,7,0;6,1,0;5,8,0;1,4,0;1,6,0#", "C:0,5:51224:824#","G:P0;0,0;0;0;100;0;0:4,3,0;5,4,0;3,8,0;2,7,0;6,1,0;5,8,0;1,4,0;1,6,0#","G:P0;0,1;2;0;100;0;0:4,3,0;5,4,0;3,8,0;2,7,0;6,1,0;5,8,0;1,4,0;1,6,0#")
# This is just game specific test data
while True:
c, sockadd = listener.accept() # Establish connection with client.
print 'incoming connection, established with ', sockadd
i = 0 # Just a counter.
while i<len(statement):
try:
self.write_data(statement[i], sockadd[0])
time.sleep(1) # The game sends updates every second to the clients
i = i + 1
#print listener.recv(1024) -- this line doesn't work. gives an error
except:
print "Error binding client"
c.close() # Close the connection
return
except:
print "Error Occurred"
I'm going to answer it because I got some help and figured it out.
The most basic thing I can do is to use the client connection which is c for this purpose. In here, instead of the commented line data=listener.recv(1024) I should have used data= c.recv(1024). Now it works.
Another way is to use SocketServers with a StreamingRequestHandler. While this is ideal for usage of typical servers, if a lot of objects are involved it could reduce the flexibility.