Is there a way to load a particular package from the pcap file using Scapy?
I know that I can load a specific number of packages using the' sniff' function and count attribute, e. g.'
sniff(offline='file.pcap', prn=action, count=31)
However, I need to get a 30th packet without loading the previous packets.
In other words, I am not satisfied with such an example:
packages = (pkt for pkt in sniff (offline=path, prn=action, count=31)
print(packages[30])
The attempt to load a millionth of a package is too long.
Each packet header states how long it is. Once the parser has read that header, it can calculate the position of the next one. So as far as I know, you cannot open a pcap file and instantly locate packet 30; you'll need to parse the headers of the first 29.
But you don't have to keep all packets in memory either, as long as you process them while receiving.
i = 0
for pkt in sniff(offline=path, prn=action):
if i == 30:
print pkt
break
I have about 10GB pcap data with IPv6 traffic to analyze infos stored in IPv6 header and other extension header. To do this I decided to use Scapy framework. I tried rdpcap function , but for such big files it is not recommended. It tries to load all file into memory and get stuck in my case.
I found in the Net that in such situation sniff is recommended, my code look like:
def main():
sniff(offline='traffic.pcap', prn=my_method,store=0)
def my_method(packet):
packet.show()
In function called my_method I receive each packet separately and I can parse them, but....
When I call show function with is in-build framework method I got sth like this:
When opened in wireshark I got properly looking packet:
Could you tell me how to parse this packets in scapy to get proper results?
EDIT:
According to the discussion in comments I found a way to parse PCAP file with Python. In my opinion the easies way is to use pyshark framework:
import pyshark
pcap = pyshark.FileCapture(pcap_path) ### for reading PCAP file
It is possible to easily iterate read file with for loop
for pkt in pcap:
#do what you want
For parsing IPv6 header following methods may be useful:
pkt['ipv6'].tclass #Traffic class field
pkt['ipv6'].tclass_dscp #Traffic class DSCP field
pkt['ipv6'].tclass_ecn #Traffic class ECN field
pkt['ipv6'].flow #Flow label field
pkt['ipv6'].plen #Payload length field
pkt['ipv6'].nxt #Next header field
pkt['ipv6'].hlim #Hop limit field
Update
The latest scapy versions now support ipv6 parsing.
So to parse an ipv6 ".pcap" file with scapy now it can be done like so:
from scapy.all import *
scapy_cap = rdpcap('file.pcap')
for packet in scapy_cap:
print packet[IPv6].src
Now as I had commented back when this question was originally asked, for older
scapy versions (that don't support ipv6 parsing):
pyshark can be used instead (pyshark is a tshark wrapper) like so:
import pyshark
shark_cap = pyshark.FileCapture('file.pcap')
for packet in shark_cap:
print packet.ipv6.src
or even of course tshark (kind of the terminal version of wireshark):
$ tshark -r file.pcap -q -Tfields -e ipv6.src
If you want to keep using scapy and read the file Iteratively I'd recommend you to give it a shot to PcapReader()
It would do the same you tried to do with pyshark but in Scapy
from scapy.all import *
for packet in PcapReader('file.pcap')
try:
print(packet[IPv6].src)
except:
pass
I'd recommend wrapping this around just as a failsafe if you have any packet that does not have an IPv6 address.
I have a problem when going througt an openfpc pcap file(1gb) with pyshark. When i use (in console, as root):
lst = pyshark.FileCapture(TMP_NAME):
for p in lst:
if(p.highest_layer=="FTP"):
print "win"
It does not print anything, even if i am sure there are ftp packets in the file. If i go like this:
lst = pyshark.FileCapture(TMP_NAME, display_filter="ftp"):
for p in lst:
if(p.highest_layer=="FTP"):
print "win"
It prints "win".
My questions are:
1) How can we explain this behaviour?
2) How can I see the ftp packets without filter?
EDIT: I have seen that this only happens when using .pcap files.The first code works flawlessly when using .cap files downloaded from the wireshark website. Which I find wierd since pyshark uses TShark, and should be able to read pcaps no problem.
Thanks
I am looking for the specific task:
Grab the payload/data from a packet -> Append to a file... BUT. I want specifically to follow packets according to flags/ICMP types/etc... So lets say I want specifically to take the payload of "echo" packets and not the rest.
My (ineffective) code is the following:
from scapy.all import *
f= open('filecaptured', 'a+')
def pkt_diam(pkt):
raw = pkt.getlayer(Raw).load
print raw
# pkt.show()
# fo = open("payload", "wb")
f.write(raw);
sniff (filter="icmp" , store=0, prn=pkt_diam, timeout = 120 )
The problem here is that I cannot find a way to sniff specifically for "type = echo request" and the only parameters that I can use is 'protocol' and host or 'and not host 127.0.0.1'.
Is there a way around this?
I think for this one I need to use ctypes and libpcap.so but I am not sure... (I didnt find any [other] libraries for python - sniffing )
I don't have scapy installed right now, but what if you simply check for the type echo-reply in your callback-function pkt_diam:
if not "echo-reply" in pkt.show():
return
Try filter="icmp[0]=8" for filtering during capture or
if pkt[ICMP].type==8:
in callback function.
I'm trying to write a secure transfer file program using Python and AES and i've got a problem i don't totally understand. I send my file by parsing it with 1024 bytes chunks and sending them over but the server side who receive the data crashes ( I use AES CBC therefore my data length must be a multiple of 16 bytes ) and the error i get says that it is not.
I tried to print the length of the data sent by the client on the client side and the length of the data received on the server and it shows that the client is sending exactly 1024 bytes each time like it's supposed to, but the server side shows that at some point in time, a received packet is not and so less than 1024 bytes ( for example 743 bytes ).
I tried to put a time.sleep(0.5) between each socket send on the client side and it seems to work. Is it possible that it is some kind of socket buffer failure on the server side ? That too much data is being send too fast by the client and that it breaks somehow the socket buffer on the server side so the data is corrupted or vanish and the recv(1024) only receive a broken chunk? That's the only thing i could think of, but this may also be completely false, if anyone has an idea of why this is not working properly it would be great ;)
Following my idea i tried :
self.s.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 32768000)
print socket.SO_RCVBUF
I tried to put a 32mbytes buffer on the server side but On Windows XP it shows 4098 on the print and on linux it shows only 8. I don't know how i must interpret this, the only thing i know is that it seems that it doesn't have a 32mbytes buffer so the code doesn't work.
Well it's been a really long post, i hope some of you had the courage to read it all to here ! i'm totally lost there so if anyone has any idea about this please share it :D
Thanks to Faisal my code is here :
Server Side: ( count is my filesize/1024 )
while 1:
txt=self.s.recv(1024)
if txt == " ":
break
txt = self.cipher.decrypt(txt)
if countbis == count:
txt = txt.rstrip()
tfile.write(txt)
countbis+=1
Client side :
while 1:
txt= tfile.read(1024)
if not txt:
self.s.send(" ")
break
txt += ' ' * (-len(txt) % 16)
txt = self.cipher.encrypt(txt)
self.s.send(txt)
Thanks in advance,
Nolhian
Welcome to network programming! You've just fallen into the same mistaken assumption that everyone makes the first time through in assuming that client sends & server recives should be symmetric. Unfortunately, this is not the case. The OS allows reception to occur in arbitrarily sized chunks. It's fairly easy to work around though, just buffer your data until the amount you've read in equals the amount you wish to receive. Something along the lines of this will do the trick:
buff=''
while len(buff) < 1024:
buff += s.recv( 1024 - len(buff) )
TCP is a stream protocol, it doesn't conserve message boundaries, as you have just discovered.
As others have pointed out you're probably processing an incomplete message. You need to either have fixed sized messages or have a delimiter (don't forget to escape your data!) so you know when a complete message has been received.
What TCP can guarantee is that all your data arrives, in the right order, at some point. (Unless something unexpected happens, by which it won't arrive.) But it's very possible that the data you send will still arrive in chunks. Much of it is because of limited send- and receive-buffers. What you should do is to continue doing your recv calls until you have enough data to process it. You might might have to call send multiple times; use its return value to keep track of how much data has been sent/buffered so far.
When you do print socket.SO_RCVBUF, you actually print the symbolic SO_RCVBUF contant (except that Python doesn't really have constants); the one used to tell setsockopt what you want to change. To get the current value, you should instead call getsockopt.
Not related to TCP (as that has been answered already), but appending to a string repeatedly will be rather inefficient if you're expecting to receive a lot. It might be better to append to a list and then turn the list into a string when you finished receiving by using ''.join(list).
For many applications, the complexities of TCP are neatly abstracted by Python's asynchat module.
Here is the nice snippet of code that I wrote some time ago, may be not the best , but it could be good example of big files transfer over the local network. http://setahost.com/sending-files-in-local-network-with-python/
As mentioned above
TCP is a stream protocol
You can try this code, where the data is your original data, you can read it from the file or user input
Sender
import socket as s
sock = s.socket(s.AF_INET, s.SOCK_STREAM)
sock.connect((addr,5000))
sock.sendall(data)
finish = t.time()
Receiver
import socket as s
sock = s.socket(s.AF_INET, s.SOCK_STREAM)
sock.setsockopt(s.SOL_SOCKET, s.SO_REUSEADDR, 1)
sock.bind(("", 5000))
sock.listen(1)
conn, _ = sock.accept()
pack = []
while True:
piece = conn.recv(8192)
if not piece:
break
pack.append(piece.decode())