I would like to explorer the possibilites of creating a custom IP procotol.
Perhaps with scapy in Python.
I've got a tunnel where I receive all packets, so would like to see if I can dismiss destination IP, ports etc. and keep an absolut minimum - but with ACK.
Anyway this is possible with scapy or similar?
I've tried some simple stuff like:
p = IP(dst="192.168.0.2")/"My payload"
But it seems not doable in that manner.
What are you trying to achieve? Forgoing transport layer port numbers limits communication to a single process on each node.
You'll need to register a protocol (number) with the IP stack to make it pass everything with that protocol to your handler. Most often, it's much easier to use UDP transport and build your own stuff on top. You can also simply number your UDP datagrams and ACK them on the application level.
Related
So I'm trying to build a packet sniffer in Python to deepen my understanding of networking. Thing is, it has turned out to be a tad bit more confusing than I initially anticipated. The problem is that all resources with thorough explanations cover the scenario of creating sockets for client/server data sending/receiving purposes.
At this point, I've successfully created some classes that handle packet header decoding for IPv4 and ICMP. Now, since my socket code only seemed to capture ICMP packets, I've been trying to configure it so that I can catch all traffic reaching my wifi interface, but I still almost exclusively see ICMP packets (with localhost as both source and destination).
So, I have some questions which I'd like to get answered. But first, my code:
import socket
import sys
from protocols.ipv4 import IPv4
PACKET_SIZE = 65535
sock = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_IP)
sock.bind(("0.0.0.0", 0))
try:
while True:
# read in a packet
raw_buffer = sock.recvfrom(PACKET_SIZE)[0]
# create an IP packet object
ip_header = IPv4(raw_buffer)
# print the packet
print(ip_header)
except KeyboardInterrupt:
print("\nExiting...")
sock.close()
sys.exit(0)
This is how I've understood it:
First I'm creating a socket with socket.socket, where I specify address family, socket type and protocol. In my case, I'm selecting the AF_INET family which I don't really understand very well, but it seems to yield packets from the network layer. The socket type is set to SOCK_RAW meaning that I want the raw sockets as opposed to using SOCK_STREAM for TCP connections and SOCK_DGRAM for UDP. The last argument IPPROTO_IP just indicates that I want IP packets only.
Then, I'm binding the socket to 0.0.0.0 which supposedly means "any address" as described here.
What I don't understand:
Initially, I saw some examples of creating a sniffer socket which used the AF_PACKET address family. I soon found out that this address family is not available on macos (which I'm using). Why is that? What is an address family how does it relate to sockets? Is there an alternative way to catch packets from lower levels? In Wireshark I can see ethernet datagrams, so it seems possible.
As I've stated, I want to sniff all the traffic reaching my wifi interface. How does the socket know which interface I want it to operate on? Also I've learned that network interfaces can be put into different modes like monitor or promiscuous, how does that relate to sockets and my goal of catching packets?
Why am I almost only catching ICMP packets? What is the purpose of these packets with localhost both as destination and source?
I know there are lots of gaps in my current understanding of this. I'm not sure if I'll be able to get this to work, but I'm curious and I'd be grateful for any kind of answer or even just some good resources to check out.
Edit: My main question is where can I find out more about sockets in the context of packet sniffing?
I have a client written using python-twisted (http://pastebin.com/X7UYYLWJ) which sends a UDP packet to a UDP Server written in C using libuv. When the client sends a packet to the server, it is successfully received by the server and it sends a response back to the python client. But the client not receiving any response, what could be the reason ?
Unfortunately for you, there are many possibilities.
Your code uses connect to set up a "connected UDP" socket. Connected UDP sockets filter the packets they receive. If packets are received from any address other than the one to which the socket is connected, they are dropped. It may be that the server sends its responses from a different address than you've connected to (perhaps it uses another port or perhaps it is multi-homed and uses a different IP).
Another possibility is that a NAT device is blocking the return packets. UDP NAT hole punching has come a long way but it's still not perfect. It could be that the server's response arrives at the NAT device and gets discarded or misrouted.
Related to this is the possibility that an intentionally configured firewall is blocking the return packets.
Another possibility is that the packets are simply lost. UDP is not a reliable protocol. A congested router, faulty networking gear, or various other esoteric (often transient) concerns might be resulting in the packet getting dropped at some point, instead of forwarded to the next hop.
Your first step in debugging this should be to make your application as permissive as possible. Get rid of the use of connected UDP so that all packets that make it to your process get delivered to your application code.
If that doesn't help, use tcpdump or wireshark or a similar tool to determine if the packets make it to your computer at all. If they do but your application isn't seeing them, look for a local firewall configuration that might reject them.
If they're not making it to your computer, see if they make it to your router. Use whatever diagnostic tools are available (along the lines of tcpdump) on your router to see whether packets make it that far or not. Or if there are no such tools, remove the router from the equation. If you see packets making it to your router but no further, look for firewall or NAT configuration issues there.
If packets don't make it as far as your router, move to the next hop you have access to. This is where things might get difficult since you may not have access to the next hop or the next hop might be the server (with many intervening hops - which you have to just hope are all working).
Does the server actually generate a reply? What addressing information is on that reply? Does it match the client's expectations? Does it get dropped at the server's outgoing interface because of congestion or a firewall?
Hopefully you'll discover something interesting at one of these steps and be able to fix the problem.
I had a similar problem. The problem was windows firewall. In firewall allowed programs settings, allowing the communication for pythonw/python did solve the problem. My python program was:
from socket import *
import time
address = ( '192.168.1.104', 42) #Defind who you are talking to (must match arduino IP and port)
client_socket = socket(AF_INET, SOCK_DGRAM) #Set Up the Socket
client_socket.bind(('', 45)) # arduino sending to port 45
client_socket.settimeout(1) #only wait 1 second for a response
data = "xyz"
client_socket.sendto(data, address)
try:
rec_data, addr = client_socket.recvfrom(2048) #Read response from arduino
print rec_data #Print the response from Arduino
except:
pass
while(1):
pass
I'm implementing client-server communication using UDP that's used for FTP. First off, you don't need to tell me that UDP is unreliable, I know. My approach is: client asks for a file, server blasts the client with udp packets with sequence numbers, then says "what'd you miss?", resending those. On a local network, packet loss is < 1%. I'm pretty new to socket programming, so I'm not familiar with all the socket options (of which most examples found on google are for tcp).
My problem is why my client's receiving of this data.
PACKET_SIZE = 9216
mysocket.sendto('GO!', server_addr)
while True:
resp = mysocket.recv(PACKET_SIZE)
worker_thread.enqeue_packet(resp)
But by the time it gets back up to .recv(), it's missed a few udp packets (that I've confirmed are being sent using wireshark). I can fix this by making the server send slightly slower (actually, including logging statements is enough of a delay to make everything function).
How can i make sure that socket.recv doesn't miss anything in the time it takes to process a packet? I've tried pushing the data out to a separate thread that pushes it into a queue, but it's still not enough.
Any ideas? select, recv_into, setblocking?
While you already know, that UDP is not reliable, you maybe missed the other advantages of TCP. Relevant for you is that TCP has flow control and automatically scales down if the receiver is unable to cope with the senders speed (e.g. packet loss). So for normal connections TCP should be preferred for data transfer. For high latency connections (satellite link) it behaves too bad in the default configuration, so that some people design there custom transfer protocols (mostly with UDP), while others just tune the existing TCP stack.
I don't know why you use UDP, but if you want to continue to use it you should add some kind of back channel to the sender to inform it from current packet loss, so that it can scale down. Maybe you should have a look at RTCP, which accompanies RTP (used for VoIP etc).
Using a Lantronix UDS-1100 serial to IP converter. The goal is to write a small proof of concept piece in Python to capture serial data output by this device over IP.
I've done a couple test projects using sockets in python, but they were all done between python processes (python > python): listen() on one end, and connect(), sendall() etc on the other.
I think I can use sockets for this project, but before I invest a bunch of time into it, wanted to make sure it is a viable solution.
Can python sockets be used to capture IP traffic when the traffic is originating from a non-python source? I have full control over the IP and port that the device sends the serial data to, but there will be no python connect() initiated by the client. I can pre-pend then serial data with some connect() string if needed.
If sockets won't work, please recommend another solution...guessing it will be REST or similar.
Of course. TCP/IP is supposed to be cross-platform and cross-language, so in theory you should be able to communicate with every kind of device as long as you manage to process and send the expected protocol.
After searching high and low across the google i have not found the definitive answer to the following question:
by roughly following the following guide:
http://twistedmatrix.com/documents/10.2.0/core/howto/udp.html#auto3
How is it possible to bind twisted Multicast listener to ONLY the multicast address and on specific or all interfaces.
While looking at reactor.listenMulticast it does not provide abstraction for hardware interface only an pseudo interface represented by an IP address. I cant find a method to bind only on the multicast address e.g. 224.0.0.1 of a specific interface or all interfaces.
Can anyone provide further information on this?
reactor.listenMulticast returns a twisted.internet.udp.MulticastPort object. That object owns the socket you're listening on. So hang on to the result of reactor.listenMulticast and set the appropriate socket option (in this case it looks like SO.BINDTODEVICE) along with a null terminated device string.
import IN, socket, struct
udp_port = reactor.listenMulticast(8005, MulticastServerUDP())
dev = "eth0" + "\0"
udp_port.socket.setsockopt(socket.SOL_SOCKET, IN.SO_BINDTODEVICE, dev)
reactor.run()
It would be nice if this were exposed directly through the listenMulticast call but assuming this works it would be a pretty easy patch. Let me know if this solves your problem.
The other answers may solve the problem, but there is a "more twisted" (and probably easier) way to listen to a multicast group on multiple interfaces.
Listening to a multicast group on one or more interfaces
To listen to a multicast group on one or more interfaces, provide the IP of each desired interface in multiple calls to the protocol's transport.joinGroup() method as the 'interface' argument.
Below is an example that works on my Linux box, and will make your system listen to multicasts on specific interfaces. Replace the IP addresses with ones belonging to your system.
#!/usr/bin/env python
from twisted.internet import protocol
from twisted.internet import reactor
class MyProtocol(protocol.DatagramProtocol):
def startProtocol(self):
# Join a multicast group, 224.0.0.9, on the interface belonging
# to each of these IPs.
# XXX Replace the interface_ips with one or more IP addresses belonging
# to your system.
for interface_ip in ["192.168.2.2", "10.99.1.100"]:
self.transport.joinGroup("224.0.0.9", interface_ip)
if __name__ == "__main__":
reactor.listenMulticast(1520, MyProtocol())
reactor.run()
You can check that the interface is listening to the new multicast group using the /sbin/ip maddr show command. Find the desired interface name in the command output, and verify that the multicast group shows up beneath it.
The UDP Server example linked in the original post should be able to do the same thing by changing the call to joinGroup() to include the second IP address argument, as above.
Sending multicasts from a particular IP
If you're receiving multicast data on a socket, chances are you will want to send multicast data too -- possibly out of multiple interfaces. Since it's closely related and there are very few examples around I'll throw it in here. Inside a twisted.internet.protocol.DatagramProtocol object you can use the self.transport.setOutgoingInterface() method to control the source IP that will be used for subsequent calls to self.transport.write(). Example showing sending a message from multiple IPs/interfaces:
class MyProtocol(protocol.DatagramProtocol):
# ...
def send_stuff(self, msg):
for src_ip in ["10.0.0.1", "192.168.1.1"]:
self.transport.setOutgoingInterface(src_ip)
self.transport.write(msg, ("224.0.0.9", 1520))
Suppose these IPs were assigned to two different interfaces. Sniffing with Wireshark you would see the message sent out of the first interface, then the second interface, using each IP as a source IP address for the respective transmission.
A simple, if heavy handed approach could be to just specify a route via something like:
sudo route add 239.255.0.0 -interface en0 -netmask 255.255.0.0