tcmpdump can view all the multicast traffic to specific group and port on eth2, but my Python program cannot. The Python program, running on Ubuntu 12.04:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
# Multicast port is 52122
sock.bind(('', 52122))
# Interface eth2 IP is 1.2.3.4, multicast group is 6.7.8.9
mreq = socket.inet_aton('6.7.8.9')+socket.inet_aton('1.2.3.4')
sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)
while True:
print '\nwaiting to receive message'
data, address = sock.recvfrom(1024)
print data
When I use another program to send a multicast packet to eth2, it works and prints the packet. But it fails to see all the current multicast traffic. If I run tcpdump on eth2 on the same port and group as the above program:
sudo tcpdump -i eth2 host 6.7.8.9 and port 52122
it sees both the packets I send from another program AND all the current multicast traffic. It's output looks likes this...
# Packet sent from my other program
09:52:51.952714 IP 1.2.3.4.57940 > 6.7.8.9.52122: UDP, length 19
# Packet send from the outside world
09:52:52.143339 IP 9.9.9.9.39295 > 6.7.8.9.52122: UDP, length 62
Why can't my program see the packets from the outside world? How can I modify it (or something else) to fix this?
Edit:
I should have mentioned, the interface this going over is not eth2 but eth2.200 a VLAN. (The local IP and the tcpdump commands are all run with eth2.200, I just changed that in this question to make it simpler.) Based on this answer that could be the problem?
Edit #2:
netstat -ng when the program is running shows eth2.200 subscribed to 224.0.0.1 and 6.7.8.9`.
tshark -i eth2.200 igmp shows three repeated 1.2.3.4 -> 6.7.8.9 IGMP 46 V2 Membership Report / Join group 6.7.8.9 when the program first starts. When the program process is killed, it shows 1.2.3.4 -> 224.0.0.2 IGMP 46 V2 Leave group 6.7.8.9. There is also an infrequent 1.2.3.1 -> 224.0.0.1 IGMP 60 V2 Membership Query, general, where 1.2.3.1 is 1.2.3.4's gateway.
Not sure if it will help, but the routing table looks like:
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 1.2.5.6 0.0.0.0 UG 0 0 0 eth1
1.2.3.0 0.0.0.0 255.255.255.240 U 0 0 0 eth2.200
Thank you!
Finally! Found this question on ServerFault that addresses the same thing. Basically the kernel was not forwarding on / was filtering out the packets because it thought the sourced address was spoofed.
Changed the settings in /etc/sysctl.conf to match:
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.all.rp_filter = 0
net.ipv4.ip_forward = 1
Rebooted and everything works.
Related
By searching for 'python dual-stack', I found https://bugs.python.org/issue25667 where someone mentions that this was already resolved in https://bugs.python.org/issue17561. In the latter issue, one of the last messages helpfully mentions what the implemented solution is:
# IPv4 only
>>> socket.create_server(addr)
# IPv6 only
>>> socket.create_server(addr, family=socket.AF_INET6)
# IPv4 + IPv6
>>> socket.create_server(addr, family=socket.AF_INET6, dualstack_ipv6=True)
However, I was looking for a generic solution. Or rather, I was looking for UDP but figured that, because this is on the IP layer, any solution would be generic. It turns out that create_server() is TCP-only.
Binding to both v4 and v6 manually means I cannot just call recvfrom() and let it block until a packet comes in, because then a v6 packet might be queued while it's blocking on the v4 recvfrom call or vice versa. So I guess I need to use threading?
What is the best way to make a dual-stack UDP server in python 3.x? For me, a Linux solution would be sufficient, but of course platform-independent answers are even better.
Basically, how to write this in dual-stack:
import socket
server_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
server_socket.bind(('0.0.0.0', 53))
while True:
message, address = server_socket.recvfrom(1400)
At least under Linux, creating an IPv6 socket listens for both IPv6 and IPv4 connections. That is, given this code:
import socket
s = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
s.bind(('::', 9090))
while True:
message, address = s.recvfrom(1024)
print(address, "says: ", message.decode())
I can connect using IPv4:
echo hello world | nc -u 127.0.0.1 9090
Which results in:
('::ffff:127.0.0.1', 36694, 0, 0) says: hello world
Or I can connect using IPv6:
echo hello world | nc -u ::1 9090
Which results in:
('::1', 51880, 0, 0) says: hello world
I have a raspberry pi that is both connected to the internet via Wlan and a local device via Ethernet. So it has two IPs; one for each endpoint.
This is how it looks like simplified when running ifconfig; with different IPs for privacy
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 189.168.200.110 netmask 0.0.0.0 broadcast 255.255.255.255
wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 189.168.201.79 netmask 255.255.255.0 broadcast 192.168.1.255
This is the code that python is using to send a message to the device through the Ethernet with that gateway's ip
TCP_PORT = 3001
SERVER_IP_AD = "189.168.200.110"
CLIENT_IP_AD = "189.168.200.155"
BROADCAST_IP = "255.255.255.255"
def sendMessage(self, file_path, client_ip=CLIENT_IP_AD):
print('message en route')
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((self.SERVER_IP_AD, 0))
s.connect((client_ip, self.TCP_PORT)) #**ERROR IS HERE**
MESSAGE = self.openFile(file_path)
s.send(MESSAGE.encode())
data = s.recv(self.BUFFER_SIZE)
s.close()
return data
Using wireshark I can see that the package is being sent through the Wlan interface instead of the Ethernet interface with the correct IP source and IP destination.
How do I tell python to use the correct interface when sending out the package?
In my opinion, you can establish Tcp connection with Ethernet, cause there isn't shaking hands by Ethernet
And, you shouldn't use s.bind() and s.connect() at the same time. Because the former is for UDP client, and the later is for TCP client. Have a try with only s.bind().
I've been working on a project that requires a bit of networking between a server (hosted on GCE) and multiple clients. I created a Compute Engine Instance to run a Python script as shown in this video: https://www.youtube.com/watch?v=5OL7fu2R4M8.
Here is my server-side script:
server = socket.gethostbyname(socket.gethostname()) # 10.128.X.XXX which is the Internal IP
print(server)
port = 5555
clients = 0
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((server, port))
s.listen(2)
print("Waiting for connection...")
while True:
conn, addr = s.accept()
print("Connected to: ", addr)
conn.send(str.encode(f"{clients}"))
clients += 1
and here is my client side-script:
class Network:
def __init__(self):
self.client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.server = "10.128.0.2"
self.port = 5555
self.addr = (self.server, self.port)
self.id = int(self.connect())
def connect(self):
self.client.connect(self.addr)
return self.client.recv(2048).decode()
network = Network()
print(f"Connected as client {network.id}")
I know this script works because I have tested it with my computer being the server and 1 client, and another computer being the 2nd client. But when I use the GCE as the server, I get this error in the client script:
TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
Could this be because I am using the internal IP address and not the external?
After this, I tried changing the firewall settings (added 'python-socket') of the GCE and this is what they look like:
But the error still persists...
As answered by W_B, I tried to run these commands on my VM and got the following outputs:
From your description it's evident it's the connection problem.
First of all you have to check if the firewall rule you created is still there. If it's "too broad" and allows for very wide access then it might be removed automatically even without you knowing it. It's on you'r screenshot but check it again just to be sure.
If it's there select the protocol you're goint to be using (I assume it's TCP) - some protocols are always blocked by default by GCP (you can't change this) so creating a rule with "any protocol" allowed is risky. Also - put one or two target IP's (not all inside this VPC) - this is not a must but improves security of your network.
Second - make sure port 5555 you're trying to connect to is accessible from other computers. You can scan the target host with nmap -p 5554 put.server.ip.here
You can scan it from the Internet or other VM's in the same VPC network.
You should get something like this:
root#localhost:~$ nmap -p 443 192.168.1.6
Starting Nmap 7.70 ( https://nmap.org ) at 2020-06-25 17:12 UTC
Nmap scan report for 192.168.1.6
Host is up (0.00091s latency).
PORT STATE SERVICE
443/tcp open https
Nmap done: 1 IP address (1 host up) scanned in 0.04 seconds
If you see 5555/tcp filtered freeciv this means that something blocks the port.
Run nmap on the server (I assume you run some version of Linux) and if you don't want to install any non-essencial software you can use sudo netstat -tulpn | grep LISTEN to get a list of open ports (5555 should be on the list).
Also make sure firewall on your server doesn't block this port. You can use iptables for that.
I'm experimenting testing with multicast and I have a couple questions of how I can accomplish this.
First, is it absolutely necessary for me to have a router to accomplish multicast if the two systems are under the same subnet (i.e 10.10.1.10 and 10.10.1.12) or can I just have a switch or even point to point connection? I would say no because of some of the tests I've ran, but it'd be nice to know for sure.
Second, how can I set up, if at all possible, a multicast connection between my Windows 10 PC and a VMWare running CentOS7 on that same PC?
What I have right now:
A VMWare with a bridged network to my External Connection and running CentOS7.
The VM has an IP of 10.10.1.12 and a netmask of 255.255.255.0 and the windows has a IP of 10.10.1.10 and a netmask of 255.255.255.0.
I've written a receive python script based on a something I've found running on the VM
import socket
import struct
MCAST_GRP = '224.0.0.71'
MCAST_PORT = 1000
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(('', MCAST_PORT))
mreq = struct.pack("4sl", socket.inet_aton(MCAST_GRP), socket.INADDR_ANY)
sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)
while True:
print sock.recv(10240)
and a sending python script from my host PC
import socket
MCAST_GRP = '224.0.0.71'
MCAST_PORT = 1000
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
sock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 2)
sock.sendto("robot", (MCAST_GRP, MCAST_PORT))
on my CentOS7 VM I can see the receive script is signing up to the group by running
netstat -g
I can also see the incoming packets by running
sudo tcpdump -i <NIC> host 224.0.0.0/4
which shows
14:28:03.111837 IP 10.10.1.101.60007 > 224.0.0.71.cadlock2: UDP, length 26
I can also see it in Wireshark as an incoming UDP message, but my python receive script running isn't receiving it.
I've also set net.ipv4.all.rp_filter to 0 that I've seen while doing research, but nothing.
However, I can run the send and receive scripts on the Windows PC and receive them just fine. I can even see them in Wireshark and they show up as IP 10.10.1.101 to 224.0.0.71 IPV4mcast.
I can also send messages from the VM to the host PC just fine. I just can't receive on the VM.
Basically, I want to listen to all ports using the socket module. How do I make it so that port is = to all the open ports on the server? Any guides and or resources are highly appreciated Here is my current code.
import socket
def Main():
host = '127.0.0.1'
port = 5000
s = socket.socket()
s.bind((host, port))
s.listen(1)
c, addr = s.accept()
print('Connection from: ' + str(addr))
while True:
data = c.recv(1024)
if not data:
break
print('from connected user: ' + str(data))
data = str(data).upper()
print('sending: ' + str(data))
c.send(data)
c.close()
if __name__ == '__main__':
Main()
You may try all possible ports and store them in a list. Remember ports below 1024 are reserved and some of the ports may be in use. So, you will get some errors and you need to handle those if you cannot bind to that port. Also, you need a socket for each port since a socket can only listen at one port. Create a function create_socket which returns socket, then store them is a list. If you get error while trying to connect, just pass those errors. This may not be a good approach but it will work for you.
def create_socket(port_number):
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_socket.bind(('127.0.0.1', port_number))
server_socket.listen(1)
return server_socket
socket_list = []
for port_number in range(1025,65536):
try:
socket_list.append(create_socket(port_number))
except Exception:
pass
Do Packet Filtering and Translation
An alternative would be to setup packet filtering and translation in the host machine to direct all incoming TCP requests to your process, regardless of the destination port. sshuttle does this to tunnel all requests to an ssh server. This way, your process would not need to open thousands of files.
In freeBSD/macOS the configuration is achieved as follows. Other OSs will have their specific way of doing this (e.g. nftables' nft(8) in Debian).
Configuration File
Create a file (named rules.txt for this example) with the following contents:
# Redirect incoming traffic on interface en0 to localhost:5000
rdr pass on en0 inet proto tcp all -> 127.0.0.1 port 5000
Change en0 to the interface that you wish to intercept incoming connections on. Remove inet or replace with inet6 to accept both IP and IPv6 or just IPv6, respectively. Check pf.conf(5) for exact semantics and syntax of this file.
Enable Rules
With administrative access run the following to load up the rules contained in the previously created file.
Enable packet filtering and translation:
pfctl -e
Flush everything (be careful as this will erase existing routing and translating configurations already set):
pfctl -F a
Load the rules:
pfctl -f rules.txt
Test it out.
Also Include Outgoing Traffic
If you also want to include outgoing traffic, as sshuttle does, you should append the next line to rules.txt:
pass out route-to lo0 inet proto tcp all
You can also tweak this rule to be more selective and avoid setting yourself a networking jail (see entry 1 of notes below).
Notes
If you include outgoing traffic, you will be unable to communicate with the outside world unless you receive a connection first or have an alternative interface. This is because outgoing traffic will be routed to your catchall process (due to the route-to keyword).
Beware that this method allows processes bound to localhost to be reachable from the outside. In the setup above, anyone connecting to any port on en0 will be able to talk to the process bound to 127.0.0.1:5000.
Remember to reload the rules after changing rules.txt for them to take effect.
To disable packet filtering and translation run pfctl -d.