Find how much data has been transferred from pcap data - python

I have a pcap file which contains traffic trace from an experiment in a binary format. What I'm trying to do is to find out how much data different hosts are exchanging with each other, but I'm pretty new to working with pcap and I've been searching and trying different things without success.
Can tcpdump be useful here? I've processed the raw file with it and got something like this:
2009-12-17 17:26:04.398500 IP 41.177.117.184.1618 > 41.177.3.224.51332: Flags [P.], seq 354231048:354231386, ack 3814681859, win 65535, length 338
2009-12-17 17:26:04.398601 IP 90.218.72.95.10749 > 244.3.160.239.80: Flags [P.], seq 1479609190:1479610159, ack 3766710729, win 17520, length 969
2009-12-17 17:26:04.398810 IP 244.3.160.239.80 > 90.218.72.95.10749: Flags [.], ack 969, win 24820, length 0
2009-12-17 17:26:04.398879 IP 41.177.3.224.51332 > 41.177.117.184.1618: Flags [P.], seq 1:611, ack 338, win 65535, length 610
Are the "length" values at the end of each line good indicators of how much data two hosts have transferred to each other?
The problem is that if I look at the raw file with Wireshark it seems like this length is actually the TCP header length, however the data/payload size is specified separately in Wireshark (38 bytes for the first of these four packets) which is confusing me.
So to sum up - Wireshark says (about the first packet):1) "396 bytes on wire", 2) "96 bytes captured", 3) "len: 338", 4) "Data (38 bytes)".
Tcpdump says: "length 338"
How do I find payload size? I'm willing to use Python if possible as I'll be working with a huge capture file.

Can tcpdump be useful here?
Yes.
Are the "length" values at the end of each line good indicators of how much data two hosts have transferred to each other?
Yes. That's the amount of bytes transferred sans headers.
How do I find payload size? I'm willing to use Python if possible as I'll > be working with a huge capture file.
You didn't specify a protocol so let's assume by "payload size" you mean "everything after the IP header". This is easy to do with Python and dpkt. As per Jon's tutorial, assuming IP packets with no options, some code that does probably what you want looks like this:
#!/usr/bin/env python
import dpkt
from socket import inet_ntoa
with open("sample.pcap") as f:
pcap = dpkt.pcap.Reader(f)
for ts, buf in pcap:
ip = dpkt.ethernet.Ethernet(buf).data
print "{} --> {} {}".format(inet_ntoa(ip.src), inet_ntoa(ip.dst), ip.len)

Related

How to get TCP-Timestamp (TSval) using python

I've searched in several places, but I didn't find a simple answer to this question -
I have a .pcap file, generated using Wireshark, with several packets in it, and I wish to extract from each packet it's TCP-Timestamp (TSval).
I've managed to open each packet using scapy
packets = rdpcap('pcap_file.pcap')
for packet in packets:
print(packet.payload.id)
but I can't find the TSval of the packet (even though I can see the TSval field in the packet when I open it with Wireshark as shown in the picture below).
Packets can be accessed like dictionaries whose keys are protocols and values are payloads. For instance you can print the TCP payload in a packet like this:
if TCP in packet:
packet[TCP].show()
Now to get the TSval of the payload you have to look in TCP options. Each TCP option is encoded by scapy as a couple (option name, option value). For the timestamp option, the option value is itself a couple (TSval, TSecr). So you can basically get what you want doing the following:
from scapy.all import TCP, rdpcap
packets = rdpcap('packets.pcapng')
for packet in packets:
if TCP in packet: # ignore packets without TCP payload
for opt, val in packet[TCP].options: # consider all TCP options
if opt == 'Timestamp':
TSval, TSecr = val # decode the value of the option
print('TSval =', TSval)

Read/capture packets larger than 65535 bytes with scapy

I am reading USBPcap files created in Wireshark with large, isochronous packets up to 494439 bytes. I am then trying to parse the packets in python with scapy. Unfortunately, scapy 2.4.3 defaults to ignoring bytes past the 65535th byte for packets greater than 65535 bytes.
Is there a way to read packets larger than 65535 bytes?
In order to instruct scapy to read packets larger than 65535 bytes, you must read packets one at a time off the many pcap reading generators found in scapy.utils API. Specifically, the following classes within scapy.utils support reading large packets from a pcap or pcapng file:
class scapy.utils.PcapReader(filename)
class scapy.utils.PcapNgReader(filename)
class scapy.utils.RawPcapReader(filename)
class scapy.utils.RawPcapNgReader(filename)
Note that scapy.utils.rdpcap(filename, count=- 1) does not support this functionality.
My solution:
from scapy.all import *
packet_reader = RawPcapNgReader('my_pcapng_file.pcapng')
while True:
try:
# created my own usb_packet class to parse packet header on __init__
p = usb_packet(packet_reader.read_packet(size=500000)) # read packet up to 500 kB large
if p.filter(): # filter packets based on usb_packet custom filter method
print(len(p.hex_str), len(p.data_packets)) # do stuff to the filtered packets
except EOFError:
break
For capturing packets via sniff or similar, one might set the buffer size via class scapy.config.Conf.bufsize which defaults to 65536 to allow a 65535 byte packet to be captured (see docs). For pragmatic programming, one may set this configuration when reading pcaps as well rather than maintaining the size argument wherever read_packet() is called on a generator.
An interesting note: This default appears to be due to IPv4 packet structure in which the packet length is represented by a 16 bit number. In other words, the largest packet this header value can depict is 1111111111111111 bytes, or 2^16=65536. Counting from zero, we have the largest IPv4 packet being 65535 bytes. For USB, as in my case, larger packets are possible.

Parsing the pcap file through scapy

I like to read a pcap file through scapy and use a filter of source address,destination address and length of the packet above or equal to 400,
After matching those packets, i would like to remove the first 16 bytes and then extract the remaining bytes sequentially.
file=rdpcap(pcap)
for pkt in file:
if pkt[0].src=='198.18.32.1' and pkt[0].dst=='198.18.50.97':
This is more than normal. pkt[0] does not really mean anything...
When doing pkt.src, you’ll get the Ethernet address (of the first layer), which is no IP address. You would need pkt[IP].src to get the IP address

Getting a particular package from the pcap file using the Scapy module (python)

Is there a way to load a particular package from the pcap file using Scapy?
I know that I can load a specific number of packages using the' sniff' function and count attribute, e. g.'
sniff(offline='file.pcap', prn=action, count=31)
However, I need to get a 30th packet without loading the previous packets.
In other words, I am not satisfied with such an example:
packages = (pkt for pkt in sniff (offline=path, prn=action, count=31)
print(packages[30])
The attempt to load a millionth of a package is too long.
Each packet header states how long it is. Once the parser has read that header, it can calculate the position of the next one. So as far as I know, you cannot open a pcap file and instantly locate packet 30; you'll need to parse the headers of the first 29.
But you don't have to keep all packets in memory either, as long as you process them while receiving.
i = 0
for pkt in sniff(offline=path, prn=action):
if i == 30:
print pkt
break

Remove tzsp headers to normalize pacquet

im trying to create a little sniffer on `python, to remove the tzsp encapulation that my router uses to send me
a copy all all the packet that flows through it. The problem is that i want to use ossim, and snort to analyze
the traffic, and woul like to de-encapsulate the packet and send it to a virtual interface where snort is
listening.
I allready have the sniffer, but didnt find the way to strip the tzsp headers to get the original packet and
send it after. I see that other people have done this on perl, but dont know how to do the same on scapy.
Its basically remove the firts 5 bytes.
# --- Cut TZSP bytes
my $tzspheader = substr $udp->{data}, 0, 5;
my $tzspdata = substr $udp->{data}, 5;
Full perl script -> http://wiki.mikrotik.com/wiki/Calea_perl_trafr
Can i use the pkt scapy format as an array ? If i remove the headers the packet will be as received by
the router, can i use scapy to sned it directly or do i have to create a emtpy packet according to the original
packet protocol and copy each field to the new one ?
Thank you All

Categories