So I'm trying to build a packet sniffer in Python to deepen my understanding of networking. Thing is, it has turned out to be a tad bit more confusing than I initially anticipated. The problem is that all resources with thorough explanations cover the scenario of creating sockets for client/server data sending/receiving purposes.
At this point, I've successfully created some classes that handle packet header decoding for IPv4 and ICMP. Now, since my socket code only seemed to capture ICMP packets, I've been trying to configure it so that I can catch all traffic reaching my wifi interface, but I still almost exclusively see ICMP packets (with localhost as both source and destination).
So, I have some questions which I'd like to get answered. But first, my code:
import socket
import sys
from protocols.ipv4 import IPv4
PACKET_SIZE = 65535
sock = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_IP)
sock.bind(("0.0.0.0", 0))
try:
while True:
# read in a packet
raw_buffer = sock.recvfrom(PACKET_SIZE)[0]
# create an IP packet object
ip_header = IPv4(raw_buffer)
# print the packet
print(ip_header)
except KeyboardInterrupt:
print("\nExiting...")
sock.close()
sys.exit(0)
This is how I've understood it:
First I'm creating a socket with socket.socket, where I specify address family, socket type and protocol. In my case, I'm selecting the AF_INET family which I don't really understand very well, but it seems to yield packets from the network layer. The socket type is set to SOCK_RAW meaning that I want the raw sockets as opposed to using SOCK_STREAM for TCP connections and SOCK_DGRAM for UDP. The last argument IPPROTO_IP just indicates that I want IP packets only.
Then, I'm binding the socket to 0.0.0.0 which supposedly means "any address" as described here.
What I don't understand:
Initially, I saw some examples of creating a sniffer socket which used the AF_PACKET address family. I soon found out that this address family is not available on macos (which I'm using). Why is that? What is an address family how does it relate to sockets? Is there an alternative way to catch packets from lower levels? In Wireshark I can see ethernet datagrams, so it seems possible.
As I've stated, I want to sniff all the traffic reaching my wifi interface. How does the socket know which interface I want it to operate on? Also I've learned that network interfaces can be put into different modes like monitor or promiscuous, how does that relate to sockets and my goal of catching packets?
Why am I almost only catching ICMP packets? What is the purpose of these packets with localhost both as destination and source?
I know there are lots of gaps in my current understanding of this. I'm not sure if I'll be able to get this to work, but I'm curious and I'd be grateful for any kind of answer or even just some good resources to check out.
Edit: My main question is where can I find out more about sockets in the context of packet sniffing?
Related
I have an application that sends data over a TCP connection to a production server. I need to sniff the contents of that TCP connection and resend it to a debug server.
I've gotten quite close with this:
from scapy.all import *
packets = 0
def dup_pkt(pkt):
global packets
read = raw(pkt[TCP].payload)
print(str(packets))
s.sendall(read)
print("connecting")
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("x.x.x.x", 12345))
print("connected")
print("sniffing")
pkts = sniff(prn=dup_pkt, filter="tcp dst port 12345 and not host x.x.x.x") # don't sniff the packets we're resending
The problem is that the packets appear to be missing data in the first two headers. I have set the debug server to save each received packet to a file, and set the application to connect directly to the debug server to compare the known good packet data with what the sniffer is sending. In the first packet, the first 1546/2079 bytes are good, but after that, each byte is zero instead of full of the correct data. In the second packet, the first 11 bytes are missing, but the rest is good.
Interestingly, after those initial two big setup packets, the remainder of the TCP packets seem to be sniffed properly - perhaps because they are usually far simpler and less than 40 bytes each.
Is there a better way to read packet data? Am I missing something? Unfortunately I don't have access to the source of the application, so I can't tell if it's doing anything special with those two big packets I'm having trouble with.
The issue with the first packet could indicate a problem in your operating system's TCP stack w.r.t. fragment reassembly.
In any case, try using another tool like tcpdump or wireshark to capture the packets. If they have the same problem, the problem lies with your operating system. If not, it could be a bug or configuration issue with scapy.
It could also mean that your IP packets are fragmented. Scapy does not automatically defragment packets, but fragments them.
you need to use the defrag function, or defragment (have a look at help(defrag) ), on the received packet list.
Maybe the packet you are checking is a fragment
I'm trying to read Ethernet (IEEE 802.2 / 3) frames using primarily socket.
The application shuld just sniff ethernet frames and depending on the content, act on it. However, there are almost no information on how to do this on Windows, the default (unix way) being socket.socket(socket.AF_PACKET, socket.SOCK_RAW, socket.htons(0x0800)). This is nonexistent in winsock apparently. So how do I sniff eth frames?
I suspect I need to bind to a MAC using socket.bind() instead of IP.
My current piece of code:
def _receive(interface): #Receive Eth packets.
#Interface = '192.168.0.10'
sock2 = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_RAW)
sock2.setsockopt(socket.SOL_IP, socket.IP_HDRINCL, 1))
sock2.bind((interface, 0))
while True:
data, sender = sock2.recvfrom(1500)
handle_data(sender, data)
Gets me nowhere. I see packets on Local connection in Wireshark, but it's not picked up in python..
On linux, I can do sock_raw = socket(AF_PACKET, SOCK_RAW, htons(ETH_P_802_2)) , then bind and setsockopt(sock_raw, SOL_PACKET, PACKET_ADD_MEMBERSHIP, &mreq, sizeof(mreq))
I would like to not have to depend on too many external libraries becuase this is supposed to be distributed and thus pretty lightweight. pip install-able packages are OK though, they can be bundled with the installer..
Python's socket doesn't come with sniffing capabilites. Simple as that.
The idea of having a network stack in your operating system is that programs "register" for specific types of packets to be delivered to them – typically, this is something like listening on a IP port, ie. one to two levels above raw ethernet packets.
To get all raw ethernet packets, your operating system's network stack needs some kind of driver/interface to support you with that. That's why wireshark needs WinPcap.
My guess is you're going to be pretty happy with pypcap, which probably is PyPi/pip installable.
I have a client written using python-twisted (http://pastebin.com/X7UYYLWJ) which sends a UDP packet to a UDP Server written in C using libuv. When the client sends a packet to the server, it is successfully received by the server and it sends a response back to the python client. But the client not receiving any response, what could be the reason ?
Unfortunately for you, there are many possibilities.
Your code uses connect to set up a "connected UDP" socket. Connected UDP sockets filter the packets they receive. If packets are received from any address other than the one to which the socket is connected, they are dropped. It may be that the server sends its responses from a different address than you've connected to (perhaps it uses another port or perhaps it is multi-homed and uses a different IP).
Another possibility is that a NAT device is blocking the return packets. UDP NAT hole punching has come a long way but it's still not perfect. It could be that the server's response arrives at the NAT device and gets discarded or misrouted.
Related to this is the possibility that an intentionally configured firewall is blocking the return packets.
Another possibility is that the packets are simply lost. UDP is not a reliable protocol. A congested router, faulty networking gear, or various other esoteric (often transient) concerns might be resulting in the packet getting dropped at some point, instead of forwarded to the next hop.
Your first step in debugging this should be to make your application as permissive as possible. Get rid of the use of connected UDP so that all packets that make it to your process get delivered to your application code.
If that doesn't help, use tcpdump or wireshark or a similar tool to determine if the packets make it to your computer at all. If they do but your application isn't seeing them, look for a local firewall configuration that might reject them.
If they're not making it to your computer, see if they make it to your router. Use whatever diagnostic tools are available (along the lines of tcpdump) on your router to see whether packets make it that far or not. Or if there are no such tools, remove the router from the equation. If you see packets making it to your router but no further, look for firewall or NAT configuration issues there.
If packets don't make it as far as your router, move to the next hop you have access to. This is where things might get difficult since you may not have access to the next hop or the next hop might be the server (with many intervening hops - which you have to just hope are all working).
Does the server actually generate a reply? What addressing information is on that reply? Does it match the client's expectations? Does it get dropped at the server's outgoing interface because of congestion or a firewall?
Hopefully you'll discover something interesting at one of these steps and be able to fix the problem.
I had a similar problem. The problem was windows firewall. In firewall allowed programs settings, allowing the communication for pythonw/python did solve the problem. My python program was:
from socket import *
import time
address = ( '192.168.1.104', 42) #Defind who you are talking to (must match arduino IP and port)
client_socket = socket(AF_INET, SOCK_DGRAM) #Set Up the Socket
client_socket.bind(('', 45)) # arduino sending to port 45
client_socket.settimeout(1) #only wait 1 second for a response
data = "xyz"
client_socket.sendto(data, address)
try:
rec_data, addr = client_socket.recvfrom(2048) #Read response from arduino
print rec_data #Print the response from Arduino
except:
pass
while(1):
pass
About UDP receivers.
Consider the sample code below, do I have to consider that I might receive multiple UDP messaged in the recfrom method ?
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.bind(("", 99))
while 1:
data, addr = s.recvfrom(1024)
someFunction(data)
No to the question in the title.
Yes! to the question in the question. UDP packets often arrive twice (and out of order).
Wikipedia says:
UDP uses a simple transmission model with a minimum of protocol
mechanism. It has no handshaking dialogues, and thus exposes any
unreliability of the underlying network protocol to the user's
program. As this is normally IP over unreliable media, there is no
guarantee of delivery, ordering or duplicate protection. UDP provides
checksums for data integrity, and port numbers for addressing
different functions at the source and destination of the datagram.
I have seen several examples of creating sockets to sniffing for IP Packets, for example using:
s = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_IP)
What I am trying to achieve, is sniffing for Ethernet Frames and analysing the data received in Windows. The packets I am interested in are PPPoE Frames not containing IP.
In Linux (using python) I was able to achieve this using :
s = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, socket.htons(3))
s.setsockopt(socket.SOL_SOCKET, IN.SO_BINDTODEVICE, struct.pack("%ds"%(len("eth0")+1,),"eth0"))
while condition:
pkt = s.recvfrom(1500)
addToQueue(filter(pkt))
Now due to the differences betweeen linux sockets and WinSock2 API, I am having the following compatibility issues :
There is no IN package for windows. That means the SO_BINDTODEVICE is not present. How do I sniff everything coming on eth0 interface?
What should I use for protocol option in socket() constructor as I dont want to limit it to IPPROTO_IP.
Can anyone point me to the right direction ? I went through similar questions but none of them really solved my problem as they were all concerned with IP Packet sniffing
Note: I know libraries like Scapy could be used for sniffing, but it loses packets if we are trying to do any elaborate filtering (or use the prn function) and does not suit what I am trying to do. Raw sockets fit my need perfectly.
I can't verify this without a Windows box but I think all you need is ...
HOST = socket.gethostbyname(socket.gethostname())
s = socket.socket(socket.AF_INET, socket.SOCK_RAW)
s.bind((HOST, 0))
s.ioctl(socket.SIO_RCVALL, socket.RCVALL_ON)
while condition:
pkt = s.recvfrom(1500)
addToQueue(filter(pkt))
Additionally, I'd recommend that you look in to using something like pypcap (or another libpcap wrapper) instead.
FTR
Note: I know libraries like Scapy could be used for sniffing, but it loses packets if we are trying to do any elaborate filtering (or use the prn function) and does not suit what I am trying to do. Raw sockets fit my need perfectly.
If you get Scapy and set conf.use_pcap = False, you can create a Windows raw socket by using sock = conf.L2socket() which according to yourself wont "lose packets".
You can then call recv() or recv_raw() on it like a regular socket, if really you want not to use Scapy's dissection.