Is there an automated way (either in tcpdump or via a helper app Out There) to generate a pcap file that contains only Ethernet, IP and Layer 4 (TCP in my case) headers, so that there is no payload/application data in the resulting pcap? I've found that since header sizes often vary, it's impossible to pick a capture size that won't catch any payload data.
You can strip out the TCP payload very easily with Python's scapy module
BEFORE
[mpenning#hotcoffee tshark_wd]$ tcpdump -n -r sample.pcap
reading from file sample.pcap, link-type EN10MB (Ethernet)
00:25:42.443559 IP 192.168.12.237.1052 > 192.168.12.236.22: Flags [P.],
seq 2445372969:2445373021, ack 1889447842, win 63432, length 52
00:25:42.443607 IP 192.168.12.236.22 > 192.168.12.237.1052: Flags [.],
ack 52, win 65535, length 0
00:25:42.443980 IP 192.168.12.236.22 > 192.168.12.237.1052: Flags [P.],
seq 1:389, ack 52, win 65535, length 388
PAYLOAD STRIPPING
Running this as root in linux...
#!/usr/bin/env python
from scapy.all import *
INFILE = 'sample.pcap'
OUTFILE = 'stripped.pcap'
paks = rdpcap(INFILE)
for pak in paks:
pak[TCP].remove_payload()
wrpcap(OUTFILE, paks)
AFTER
[mpenning#hotcoffee tshark_wd]$ tcpdump -n -r stripped.pcap
reading from file sample.pcap, link-type EN10MB (Ethernet)
00:25:42.443559 IP truncated-ip - 52 bytes missing! 192.168.12.237.1052
> 192.168.12.236.22: Flags [P.], seq 2445372969:2445373021,
ack 1889447842, win 63432, length 52
00:25:42.443607 IP 192.168.12.236.22 > 192.168.12.237.1052: Flags [.],
ack 52, win 65535, length 0
00:25:42.443980 IP truncated-ip - 388 bytes missing! 192.168.12.236.22
> 192.168.12.237.1052: Flags [P.], seq 1:389,
ack 52, win 65535, length 388
In the tcpdump above, notice the "XX bytes missing!" messages. That is because we have removed the TCP payload.
If simple truncate would work for you, you could use:
tcpdump -i eth0 -s 96 -w test1.pcap
Later on you can analyze it with wireshark.
My solution was as follows. I'd love to hear how others do it without external libraries or truncation. I'd love to hear how others performed this, because I was unable to find the remove_payload() function in the Scapy documentation, making this answer unusable.
#read pcap file
pkts = rdpcap("packet-capture.pcap")
#write packet with payload "XXXXXXXXXX"
for pkt in pkts:
pkt.load = "XXXXXXXXXX"
#write new pcap
wrpcap("new.pcap", pkts)
The problem with this is that, when read with tcpdump, it leaves a bytes missing! for the src IP. I can verify the infromation is still there using scapy via
pkts[_packet_num].load
Is there a way to regenerate the whole capture so it looks as if it was unaltered?
Related
I tried to simplify my problem with the following setup.
A simple netcat UDP listener on Port 1337 on my local interface (192.168.183.130)
A simple netcat UDP client connecting to the listener on port 1337 (from 192.168.183.128)
A very basic scapy sniffer running on 192.168.183.130
Scapy sniffer running with root privileges:
from scapy.all import sniff, IP, UDP
def print_package(packet):
packet.show()
sniff(filter="ip dst host 192.168.183.130 and dst port 1337", iface="ens33", prn=print_package)
Sending IP packets / UDP frames with the 1500 Bytes MTU limit works like a charm and the packets are printed to std-out as expected. As soon as I succeed the limit and the IP protocol creates fragments, the sniffer only catches the first packet (correct flags, len etc.)
In the following example I sent a simple string 'next message will be 3200 * "A"' from the nc client to the listener before sending 3200 * "A" with netcat. The packet gets fragmented into three IP packets and correctly reassembled by the stack, before the UDP socket netcat is using receives it, so everything works as i would expect it network-wise. Scapy only sniffs the first of the three packets and I do not understand why this happens.
The screenshot shows the expected/correct handling of the text message and the three IP fragments in wireshark:
The following snippet shows the scapy output to stdout:
sudo python3 scapy_test.py
###[ Ethernet ]###
dst = 00:0c:29:fa:93:72
src = 00:0c:29:15:2a:11
type = IPv4
###[ IP ]###
version = 4
ihl = 5
tos = 0x0
len = 59
id = 18075
flags = DF
frag = 0
ttl = 64
proto = udp
chksum = 0x3c3
src = 192.168.183.128
dst = 192.168.183.130
\options \
###[ UDP ]###
sport = 59833
dport = 1337
len = 39
chksum = 0xdaa0
###[ Raw ]###
load = 'next message will be 3200 * "A"\n'
###[ Ethernet ]###
dst = 00:0c:29:fa:93:72
src = 00:0c:29:15:2a:11
type = IPv4
###[ IP ]###
version = 4
ihl = 5
tos = 0x0
len = 1500
id = 20389
flags = MF
frag = 0
ttl = 64
proto = udp
chksum = 0x1518
src = 192.168.183.128
dst = 192.168.183.130
\options \
###[ UDP ]###
sport = 59833
dport = 1337
len = 3209
chksum = 0x25bd
###[ Raw ]###
load = 'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA'
Why are the other IP fragments missing and how can I sniff them?
I know about the session parameter in sniff but did I not have any luck with actually reassembling the packets with session=IPSession. (This is not what I want to achieve anyway, for my application I want to sniff all fragments, change parts of them and forward them to another address/socket.)
I finally figured this out myself, so I am gonna leave an answer:
The problem lies in the filter of the sniffer:
sniff(filter="ip dst host 192.168.183.130 and dst port 1337", iface="ens33", prn=print_package)
IP fragments after the first do not have a UDP part and therefore do not have a destination port, therefore the scapy filter does not catch them. To work around this problem I extended the filter to catch dst port 1337 or Fragments with an offset. I stumbled across this cheatsheet https://github.com/SergK/cheatsheat-tcpdump/blob/master/tcpdump_advanced_filters.txt, that offers a valid berkeley syntax for this problem and ended up with this filter (for the simplified problem).
sniff(filter="ip dst host 192.168.183.130 and ((src port 1337) or (((ip[6:2] > 0) or (ip[7] > 0)) and (not ip[6] = 64))", iface="ens33", prn=print_package)
This checks if the fragment offset of the packet is >0 (anything after the first three bit of the sixth byte (flags) or the seventh byte are >0) and if the "don't fragment" bit is not set. If this is true, it is an IP fragment and the sniffer shall sniff it.
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("HOST", PORT))
This code surely send a SYN packet to HOST, but does it complete the three-way handshake? Does it send the ACK packet to HOST?
If not, how can I make socket not sending the ACK packet?
That's because I'm trying to study the syn flood flaws and how this attack works. So SYN packets are sent but no ACK packets response are sent.
The .connect() call is asking the kernel to setup a usable socket
with the standard 3-way handshake:
SYN →
← SYN+ACK
ACK →
To send packets, without creating a usable socket, call hping3 instead:
$ sudo hping3 -i u1 -S -p 80 192.168.1.1
I've been running in the circles with the following scenario:
A client sends GET request, requesting resource of the size of a few full MSS 1460 bytes TCP packets, let's say 6.
How to force scapy to send a TCP segment with a FIN flag BEFORE all packets are delivered by the server ie. after packet 1,2,3,4,5
Was trying the combination of simple curl requests/loop and a python/scapy script:
from scapy.all import * from time import sleep
os.system("iptables -I OUTPUT -p tcp --tcp-flags ALL RST,ACK -j DROP")
os.system("iptables -I OUTPUT -p tcp --tcp-flags ALL RST -j DROP")
def packet(pkt):
if pkt[TCP].flags == 16:
print('ACK packet detected port : ' + str(pkt[TCP].sport) + ' from IP Src : ' + pkt[IP].src)
send(IP(dst=pkt[IP].src, src=pkt[IP].dst)/TCP(dport=pkt[TCP].sport, sport=pkt[TCP].dport,ack=pkt[TCP].seq + 1, flags='F')) sniff(iface="ens160", prn=packet, filter="host 1.1.1.1",count=10)
But segment containing FIN flag arrives always late and out of TCP seq/ack sync.
I was recently trying to write a Scapy script that performs a full TCP handshake. The idea was that I connect two Qemu VMs using -net socket userspace interface (which seems to handle raw IP/ethernet fine) and instruct machine B to block all input from A (to prevent it from sending the RSTs). Then, I used telnet to connect() from machine A to B and ran the following script on machine B:
#!/usr/bin/python
import scapy.all as scapy
filter = "port 31337"
iface = "eth0"
def prepare_response(t):
print("Received: %s" % repr(t))
t.src, t.dst = t.dst, t.src # swap ethernet addresses
ip = t.getlayer("IP")
ip.src, ip.dst = ip.dst, ip.src
t.dport, t.sport = t.sport, t.dport
t.ack = t.seq
t.ack += 1
syn = scapy.sniff(filter=filter, count=1, iface=iface)[0]
print(syn.sprintf('%TCP.flags%'))
syn_ack = syn
prepare_response(syn_ack)
syn_ack.getlayer("TCP").flags |= 0x10 # set the ACK flag
print(syn_ack.sprintf('%TCP.flags%'))
print("Sending: %s" % repr(syn_ack))
scapy.sendp(syn_ack, iface=iface, verbose=False)
ack = scapy.sniff(filter=filter, count=1, iface=iface)[0]
assert(ack.flags & 0x10)
The problem is that instead of receiving an ACK from A to B, I seem to get a SYN retransmission as if SYN+ACK wasn't interpreted correctly:
tcp on machine A confirms that SYN+ACK reached the machine:
05:47:03.925100 IP 10.0.0.1.39634 > debian.31337: Flags [S], seq 2426802888, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0
05:47:03.927515 IP debian.31337 > 10.0.0.1.39634: Flags [S.], seq 2426802888, ack 2426802889, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0
Here's the PCAP file from machine B's perspective in Base64 form:
1MOyoQIABAAAAAAAAAAAAP//AAABAAAAYlilUwieDgARAQAAEQEAAAEAXgAA+1JUABI0VggARQABA2UUQAD/ESrYCgAAAuAAAPsU6RTpAO/r/QAAAAAAAwAAAAUAAAE2ATUBNAEzATIBMQFlAWYBZgFmATABMAE0ATUBMAE1ATABMAEwATABMAEwATABMAEwATABMAEwATABOAFlAWYDaXA2BGFycGEAAP8AAQtkZWJpYW4tMTA5MwVsb2NhbAAA/wABATIBMAEwAjEwB2luLWFkZHLAUAD/AAHAWgANAAEAAAB4AAsESTY4NgVMSU5VWMBaAAEAAQAAAHgABAoAAALAcQAMAAEAAAB4AALAWsBaABwAAQAAAHgAEP6AAAAAAAAAUFQA//4SNFbADAAMAAEAAAB4AALAWmJYpVMJoA4AnAAAAJwAAAABAF4AAPtSVAASNFYIAEUAAI4GlEAA/xGJzgoAAAHgAAD7FOkU6QB6hFgAAIQAAAAAAQAAAAABNgE1ATQBMwEyATEBZQFmAWYBZgEwATABNAE1ATABNQEwATABMAEwATABMAEwATABMAEwATABMAEwATgBZQFmA2lwNgRhcnBhAAAMgAEAAAB4ABIKZGViaWFuLTQwNwVsb2NhbABnWKVTvIYIAEIAAABCAAAAUlQAEjRWUlQAEjRWCABFAAA0HdtAAEAGCOcKAAABCgAAAprbemmul/p8AAAAAIACOQhjsAAAAgQFtAEBBAIBAwMEZ1ilU5COCABCAAAAQgAAAFJUABI0VlJUABI0VggARQAANB3bQABABgjnCgAAAgoAAAF6aZrbrpf6fK6X+n2AEjkIY7AAAAIEBbQBAQQCAQMDBGhYpVPTfggAQgAAAEIAAABSVAASNFZSVAASNFYIAEUAADQd3EAAQAYI5goAAAEKAAACmtt6aa6X+nwAAAAAgAI5CGOwAAACBAW0AQEEAgEDAwRqWKVTrI4IAEIAAABCAAAAUlQAEjRWUlQAEjRWCABFAAA0Hd1AAEAGCOUKAAABCgAAAprbemmul/p8AAAAAIACOQhjsAAAAgQFtAEBBAIBAwME
And one from A to B's perspective:
1MOyoQIABAAAAAAAAAAAAP//AAABAAAAVVilU9NXCABCAAAAQgAAAFJUABI0VlJUABI0VggARQAANB3bQABABgjnCgAAAQoAAAKa23pprpf6fAAAAACAAjkIFCkAAAIEBbQBAQQCAQMDBFVYpVPIYAgAQgAAAEIAAABSVAASNFZSVAASNFYIAEUAADQd20AAQAYI5woAAAIKAAABemma266X+nyul/p9gBI5CGOwAAACBAW0AQEEAgEDAwRWWKVT008IAEIAAABCAAAAUlQAEjRWUlQAEjRWCABFAAA0HdxAAEAGCOYKAAABCgAAAprbemmul/p8AAAAAIACOQgUKQAAAgQFtAEBBAIBAwMEWFilU4FfCABCAAAAQgAAAFJUABI0VlJUABI0VggARQAANB3dQABABgjlCgAAAQoAAAKa23pprpf6fAAAAACAAjkIFCkAAAIEBbQBAQQCAQMDBA==
At first I thought that this is somehow related to some Linux TCP/IP quirk, so I experimented with turning off TCP timestamps and SYN cookies. I also tried increasing IP ID, which didn't help either. Both machines are running Debian 7.5 with linux-image-3.2.0-4-686-pae under qemu 1.6.2. What am I missing?
That's a checksum issue.
In the IP layer it happens to be OK since you're just swapping the source and destination addresses, but in the TCP layer the original checksum becomes wrong when you change the flags value.
The best option is to let Scapy compute the correct checksum value for you, by adding del(t[TCP].chksum) in prepare_response().
tcmpdump can view all the multicast traffic to specific group and port on eth2, but my Python program cannot. The Python program, running on Ubuntu 12.04:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
# Multicast port is 52122
sock.bind(('', 52122))
# Interface eth2 IP is 1.2.3.4, multicast group is 6.7.8.9
mreq = socket.inet_aton('6.7.8.9')+socket.inet_aton('1.2.3.4')
sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)
while True:
print '\nwaiting to receive message'
data, address = sock.recvfrom(1024)
print data
When I use another program to send a multicast packet to eth2, it works and prints the packet. But it fails to see all the current multicast traffic. If I run tcpdump on eth2 on the same port and group as the above program:
sudo tcpdump -i eth2 host 6.7.8.9 and port 52122
it sees both the packets I send from another program AND all the current multicast traffic. It's output looks likes this...
# Packet sent from my other program
09:52:51.952714 IP 1.2.3.4.57940 > 6.7.8.9.52122: UDP, length 19
# Packet send from the outside world
09:52:52.143339 IP 9.9.9.9.39295 > 6.7.8.9.52122: UDP, length 62
Why can't my program see the packets from the outside world? How can I modify it (or something else) to fix this?
Edit:
I should have mentioned, the interface this going over is not eth2 but eth2.200 a VLAN. (The local IP and the tcpdump commands are all run with eth2.200, I just changed that in this question to make it simpler.) Based on this answer that could be the problem?
Edit #2:
netstat -ng when the program is running shows eth2.200 subscribed to 224.0.0.1 and 6.7.8.9`.
tshark -i eth2.200 igmp shows three repeated 1.2.3.4 -> 6.7.8.9 IGMP 46 V2 Membership Report / Join group 6.7.8.9 when the program first starts. When the program process is killed, it shows 1.2.3.4 -> 224.0.0.2 IGMP 46 V2 Leave group 6.7.8.9. There is also an infrequent 1.2.3.1 -> 224.0.0.1 IGMP 60 V2 Membership Query, general, where 1.2.3.1 is 1.2.3.4's gateway.
Not sure if it will help, but the routing table looks like:
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 1.2.5.6 0.0.0.0 UG 0 0 0 eth1
1.2.3.0 0.0.0.0 255.255.255.240 U 0 0 0 eth2.200
Thank you!
Finally! Found this question on ServerFault that addresses the same thing. Basically the kernel was not forwarding on / was filtering out the packets because it thought the sourced address was spoofed.
Changed the settings in /etc/sysctl.conf to match:
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.all.rp_filter = 0
net.ipv4.ip_forward = 1
Rebooted and everything works.