I've made a server (python, twisted) for my online game. Started with TCP, then later added constant updates with UDP (saw a big speed improvement). But now, I need to connect each UDP socket client with each TCP client.
I'm doing this by having each client first connect to the TCP server, and getting a unique ID. Then the client sends this ID to the UDP server, connecting it also. I then have a main list of TCP clients (ordered by the unique ID).
My goal is to be able to send messages to the same client over both TCP and UDP.
What is the best way to link a UDP and TCP socket to the same client?
Can I just take the IP address of a new TCP client, and send them data over UDP to that IP? Or is it necessary for the client to connect twice, once for TCP and once for UDP (by sending a 'connect' message)?
Finally, if anyone with knowledge of TCP/UDP could tell me (i'm new!), will the same client have the same IP address when connecting over UDP vs TCP (from the same machine)? (I need to know this, to secure my server, but I don't want to accidentally block some fair users)
Answering your last question: no. Because:
If client is behind NAT, and the gateway (with NAT) has more than one IP, every connection can be seen by you as connection from different IP.
Another problem is when few different clients that are behind the same NAT will connect with your server, you will have more than one pair of TCP-UDP clients. And it will be impossible to join correct pairs.
Your method seems to be good solution for the problem.
1- Can I just take the IP address of a new TCP client, and send them data over UDP to that IP? NO in the general case, but ...
2- is it necessary for the client to connect twice, once for TCP and once for UDP ? NO, definitively
3- will the same client have the same IP address when connecting over UDP vs TCP (from the same machine)? YES except in special cases
You really need some basic knowledge of the TCP, UDP and IP protocol to go further, and idealy, on the OSI model.
Basics (but you should read articles on wikipedia to have a deeper understanding) :
TCP and UDP are 2 protocol over IP
IP is a routable protocol : it can pass through routers
TCP is a connected protocol : it can pass through gateways or proxies (firewalls and NATs)
UDP in a not connected protocol : it cannot pass through gateways
a single machine may have more than one network interface (hardware slot) : each will have different IP address
a single interface may have more than one IP address
in the general case, client machines have only one network interface and one IP address - anyway you can require that a client presents same address to TCP and UDP when connecting to your server
Network Address Translation is when there is a gateway between a local network and the wild internet that always presents its own IP address and keep track of TCP connections to send back packets to the correct client
In fact the most serious problem is if there is a gateway between the client and your server. While the client and the server are two (virtual) machines for which you have direct keyboard access, no problem, but corporate networks are generally protected by a firewall acting as a NAT, and many domestic ADSL routers also include a firewall and a NAT. In that case just forget UDP. It is possible to instruct a domestic router to pass all UDP traffic to a single local IP, but it is not necessarily an easy job. In addition, that means that if a user of yours has more than one machine at home, he will be allowed to use only one at a time and will have to reconfigure his router to switch to another one !
First of all when you send data with TCP or UDP you have to give the port.
If your client connect with TCP and after your server send a response with UDP the packet will be reject by the client.
Why? Because you have to register a port for connection and you can not be sure the port is correctly open on the client.
So when you begin a connection in TCP the client open a port to send data and receive the response. You have to make the same with UDP. When client begin all communication with server you can be sure all the necessary port are open.
Don't forget to send data on the port which the connection was open.
Can I just take the IP address of a new TCP client, and send them data over UDP to that IP? Or is it necessary for the client to connect twice, once for TCP and once for UDP (by sending a 'connect' message)?
Why you don't want create 2 connections?
You have to use UDP for movement for example. because if you create an FPS you can send the player's position every 50ms so it's really important to use UDP.
It's not just a question of better connection. If you want to have a really good connection between client and server you need to use Async connection and use STREAM. But if you use stream you'r TCP socket do not signal the end of a socket but you have a better transmition. So you have to write something to show the packet end (for example <EOF>).
But you have a problem with this. Every socket you receive you have to analyze the data and split over the <EOF>. It can take a lot a processor.
With UDP the packet always have a end signal. But you need to implement a security check.
Related
I have 2 servers and a client. There is 2 way communication between the server and the client. Would I need multiple sockets on the client to communicate with the servers? I used only one socket and a few of the packets from the servers are missing. How many sockets would I need to communicate with the server?
With UDP you almost always only need a single socket; you can call sendto() and recvfrom() on it to send and receive UDP packets from anywhere.
As for missing UDP packets, that is a fact of life with UDP; UDP packets can and sometimes will get dropped at any step of the path from sender and receiver. You'll need to design your app to tolerate that, or alternatively come up with a mechanism by which the receiver can detect that a packet was lost and request a resend (or otherwise somehow handle that situations).
Since there is no support of IP_PKTINFO in python(for IPv4), are there no multihomed UDP python servers out there in production?
If there are ( and I expect there shall be), how do they handle the problem of sending the response UDP packets to the interface from which they received the request!
EDIT for clarification
Lets say I have a UDP server with multiple interfaces (consider 2 here), each with IP 172.217.163.68 and 172.217.163.69 respectively. The server has socket bind call on 0.0.0.0. Now if a request packet comes on 172.217.163.68, the server processes it, forms a response packet, and then sends on what interface? There is no way it knows about the interface from which request packet arrived, so it can't fill the sending interface IP. This is because there is no support for IP_PKTINFO in python.
Also it should be noted that we can't make use of the default route here. If default route gets used, then from the perspective of the client, it sent request to 172.217.163.68 but is getting response from 172.217.163.69, which is obviously wrong.
UDP servers use recvfrom and sendto to get the source IP address and to send back the response:
Receive:message, address = socket.recvfrom(1024)
Send: socket.sendto(message, address)
You can see an example UDP server in python in this question
If a socket program runs on a port(say 6053) and if the rule is not added in the firewall the functions recv read and recvfrom are blocked.
How do we check this in C or python and report Port not opened error on linux machines.
Try to connect on that port using socket.connect(), if connection is not successful, then show message that Port not opened.
Tools like nmap can help in determining whether the particular port is open or closed.
TCP :
nmap uses techniques like TCP SYN scan or TCP Connect scan where the server will reply with ACK-RST packet for SYN request incase of closed port. You can notice that, it is determined at the time of 3-way handshake (connection establishment) itself.
UDP : nmap also facilitates the UDP scan, where ICMP based 'Port Unreachable' packet shall be returned in case the UDP packet arrives on a closed UDP port (This also depends on the stack in the OS). Unlike TCP, the UDP is not a connection-based protocol and ICMP is also connection-less, so you might need to send some significant number of UDP packets for a short interval and evaluate based on the responses or some similar logic.
You can arrive on similar technique/logic and determine whether the particular port is open or closed and flash appropriate message for user.
I have a client-server "snake" game working really well with TCP connections, and I would like to try it the UDP way.
I wonder how it is supposed to be used ? I know how UDP works, how to make a simple ECHO example, but I wonder how to do the following :
For instance with TCP, every TICK (1/15 second) server sends to the client the new Snake head position.
With UDP, am I supposed to do something like this :
Client SIDE :
client = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
serverAddr = (('localhost', PORT))
while 1:
client.sendto('askForNewHead', serverAddr)
msg, addrServer = client.recvfrom(1024)
game.addPosition(msg)
Server SIDE :
server = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
server.bind((HOST, PORT))
while 1:
data, addr = server.recvfrom(1024)
server.sendto(headPosition, addr)
So here Client has to ask server to get the new head position, and then server sends the answer. I managed to make it work this way, but I can't figure out if it is a good way of doing.
Seems weird that client has to ask udp for an update while with my TCP connection, client has just to wait untill he receives a message.
There are differences between TCP and UDP but not the way you describe. Like with TCP the client can recvfrom to get messages from the server without asking each time for new data. The differences are:
With TCP the initial connect includes a packet exchange between client and server. Unless the client socket was already bound to an IP and port it will be bound to the clients IP and a free port will be allocated. Because of the handshake between client and server the server knows where to contact the client and thus can send data to the packet without getting data from the client before.
With UDP there is no initial handshake. Unless already bound, the socket will be bound to clients IP and a free port when sending the first packet to the server. Only when receiving this packet the server knows the IP and port of the client and can send data back.
Which means, that you don't need to 'askForNewHead' all the time. Instead the client has to send only a single packet to the server so that the server knows where to send all future packets.
But there are other important differences between TCP and UDP:
With UDP packets may be lost or could arrive in a different order. With TCP you have a guaranteed delivery.
With UDP there is no real connection, only an exchange of packets between two peers. With TCP you have the start and end of a connection. This is relevant for packet filters in firewalls or router, which often need to maintain the state of a connection. Because UDP has no end-of-connection the packet filters will just use a simple timeout, often as low as 30 seconds. Thus, if the client is inside a home network and waits passively for data from server, it might wait forever if the packet filter closed the state because of the timeout. To work around this data have to be transmitted in regular intervals so that the state does not time out.
One often finds the argument, that UDP is faster then TCP. This is plain wrong. But you might see latency problems if packets get lost because TCP will notice packet loss and send the packet again and also reduce wire speed to loose less packets. With UDP instead you have to deal with the packet loss and other congestion problems yourself. There are situations like real time audio, where it is ok to loose some packets but low latency is important. These are situations where UDP is good, but in most other situations TCP is better.
UDP is different to TCP, and I believe with python the client does have to ask for an update from the server.
Although it is fun to learn and use a different way of communicating over the internet, for python I would really recommend sticking with TCP.
You don't have to ask the server for a update. But since UDP is connection-less the server can send head-positions without being asked. But the client should send i'm-alive-packets to the server, but this could happen every 10 seconds or so.
I am trying to learn python sockets, but am becoming very confused by the results of the example code from the website (found here).
The only modification I have made is replacing socket.gethostname() in the server with the local IP of my server, to allow me to run this on two computers.
When I connect, attempting to connect on port 12345 as in the example, I get this output:
Got connection from ('10.0.1.10', 37492)
This leads me to believe that it is connecting on port 37492. I would like it to connect on the port I tell it to, so I can port forward. Am I misunderstanding, or is there an extra command to specify it.
Edit: I am uploading my code:
Client.py
#!/usr/bin/python # This is client.py file
import socket # Import socket module
s = socket.socket() # Create a socket object
host = socket.gethostname() # Get local machine name
port = 12345 # Reserve a port for your service.
s.connect(("10.0.1.42", port))
print s.recv(1024)
s.close # Close the socket when done
Server.py
import socket
s = socket.socket() # Create a socket object
host = "10.0.1.42" # Get local machine name
port = 12345 # Reserve a port for your service.
s.bind((host, port)) # Bind to the port
s.listen(5) # Now wait for client connection.
while True:
c, addr = s.accept() # Establish connection with client.
print 'Got connection from', addr
c.send('Thank you for connecting')
c.close() # Close the connection
You have reached that point in your networking life where you need to understand protocol multiplexing. Good for you.
Think of the TCP/IP stack. An application communicates with a remote application by passing application-layer data to the transport (end-to-end) layer, which passes it to the network layer (internetwork layer) which tries, without guarantees, to have packets reach the IP destination host over a sequence of hops determined by cooperating routers that dynamically update their routing tables by talking to connected routers. Each router conversation goes over a physical transport of some kind (ISDN, Ethernet, PPP - in TCP/IP the task of creating packets and transmitting the appropriate bit stream is regarded as a single "subnetwork" layer, but this is ultimately split into two when differentiation is required between the OSI physical layer (Layer 1) and the data link layer (layer 2) for protocols like DHCP.
When TCP and UDP were designed, the designers imagined that each server would listen on a specific port. This typically has the inherent limitation that the port can only handle one version of your service protocol (though protocols like HTTP take care to be backwards-compatible so that old servers/clients can generally interoperate with newer ones). There is often a service called portmapper running on port 111 that allows servers to register the port number they are running on, and clients to query the registered servers by service (program) number and protocol version. This is a part of the Sun-designed RPC protocols, intended to expand the range of listening ports beyond just those that were pre-allocated by standards. Since the preallocated ports were numbered from 1 to 1023, and since those ports typically (on a sensible operating system) require a high level of privilege, RPC also enabled non-privileged server processes as well as allowing a server to be responsive to multiple versions of network application protocols such as NFS.
However the server side works, the fact remains that there has to be some way for the network layer to decide which TCP connection (or UDP listener) to deliver a specific packet to. Similarly for the transport layer (I'll just consider TCP here since it's connection-oriented - UDP is similar, but doesn't mind losing packets). Suppose I'm a server and I get two connections from two different client processes on the same machine. The destination (IP address, port number) will be the same if the clients are using the same version of the same protocol, or if the service only listens on a single port.
The server's network layer looks at the incoming IP datagram and sees that it's addressed to a specific server port. So it hands it off to that port in the transport layer (the layer above the network layer). The server, being a popular destination, may have several connections from different different client processes on the same machine. This is where the magic of ephemeral ports appears.
When the client requests a port to use to connect to a service, the TCP layer guarantees that no other process on that machine (technically, that interface, since different interfaces have unique IP addresses, but that's a detail) will be allocated the same port number while the client process continues to use it.
So protocol multiplexing and demultiplexing relies on five pieces of information:
(sender IP, sender port, protocol, receiver IP, receiver port)
The protocol is a field in the IP header as are the source and destination IP addresses. The sending and receiving port numbers are in the transport layer segment header.
When an incoming packet arrives, the guaranteed uniqueness of different ephemeral ports from the same client (endpoint) allows the transport layer to differentiate between different connections to the same server IP address from the same client IP address and port (the worst case for demultiplexing) by their source IP address and port. The (transport) protocol is included to ensure that TCP and UDP traffic don't get mixed up. The TCP/UDP constraints on uniqueness of ephemeral ports guarantee that any server can only receive one connection from a specific combination of (IP address, port number) and it's that that allows connections from the same machine to be demultiplexed into separate streams corresponding to the different origins.
In Python when you connect a socket to a remote endpoint the socket.accept() call returns the (IP address, port number) pair for the remote endpoint. You can use that to discover who is communicating with you, but if you just want to talk back you can simply write() the socket.
The key word is "from." That's the port that the client is connecting from, 12345 is the one your server is listening on and the client is connecting to.
The message that appears comes from the server. It just gives you information that connection was established from the client's port 37492.
This is what happens:
Your server (server.py) is listening on port 12345. Your client (client.py) connects to the port 12345 of the server. The TCP connection is always established between two ports - source and destination.
So, looking from your client app perspective 12345 is the destination port and 37492 is the source port. In other words client establishes a connection from its local port 37492 to the remote servers port 12345.
If you want to set up port forwarding you may still do it as the port on which server listens is well known (12345) and the source port of the client doesn't really matter in this situation.
The port that you get in your output is the source port. Your client program sends to the server on the port you choose (in this case, 12345), but it also needs a port to receive data sent by the server, so it randomly chooses a source port and tells it to the server.
I suggest you read some more about TCP and ports in general.