I want to print out the packet data from a pcap file. The code below stores the packet data in an array but I can't figure out how to print each element of the array and then split up the data from there. A for loop on the array just returns an error.
import dpkt
import socket
f = open('test.pcap', 'r')
pcap = dpkt.pcap.Reader(f)
info = []
for ts, buf in pcap:
eth = dpkt.ethernet.Ethernet(buf)
if not hasattr(eth, 'ip'):
continue
info.append(eth.ip)
f.close()
print info
It is not clear what exactly you want to print. It depends on what you are looking for.
If you want to print the tcp data, then here is how you do it:
import dpkt
f = open('test.pcap')
pcap = dpkt.pcap.Reader(f)
for ts, buf in pcap:
eth = dpkt.ethernet.Ethernet(buf)
ip = eth.data
tcp = ip.data
If you want to print data in a higher layer, you can extend the above example to do so. HTTP, for example, is as follows:
import dpkt
f = open('test.pcap')
pcap = dpkt.pcap.Reader(f)
for ts, buf in pcap:
eth = dpkt.ethernet.Ethernet(buf)
ip = eth.data
tcp = ip.data
if tcp.dport == 80:
http = dpkt.http.Request(tcp.data)
print http.data
For more details, see the example here, or the examples in the dpkt project.
Related
I'm trying to capture a fast stream of UDP packets without missing any.
Packet size is known and fixed at 1472 bytes.
I'm generating UDP transmissions with another Python application and the first two bytes have a incremental counter so I can check if any have been lost on capture.
Approach 1: read packet -> write to file
Packets are being written to file one at a time as they come in.
Result: 100 to 200 packets lost out of 10,000 received
import socket
import capture_verify
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
address = ("192.168.252.147", 23850)
sock.bind(address)
previous_first_val = 0
out_file = open("out_bin.bin", "wb")
packet_count = 1E5
while(packet_count):
data, addr = sock.recvfrom(2048)
out_file.write(data)
packet_count -= 1
capture_verify.verify_file()
Approach 2: read packets to memory buffer, write to file once 10K captured
Packets are being stored to a pre-allocated buffer. Then written to file once 10K packets received.
Result: 7 to 15 packets lost out of 10,000 received
import socket
import constants as cs
import capture_verify
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
address = ("192.168.252.147", 23850)
sock.bind(address)
out_file = open("out_bin.bin", "wb")
packet_count = 1E5
bytes_to_read = int(cs.packet_size * packet_count)
in_buf = bytearray(bytes_to_read)
view = memoryview(in_buf)
while packet_count:
# nbytes = sock.recv_into(view, bytes_to_read)
sock.recvfrom_into(view, cs.packet_size)
packet_count -= 1
nbytes = cs.packet_size
view = view[nbytes:]
out_file.write(in_buf)
capture_verify.verify_file()
Is there anything I can do to speed up the code and ensure no packet loss?
*UDP protocol is a requirement, can't throttle down transmission speed. Currently using Python 3.7 on Windows 7
I would like to extract date time from PCAP files only for ARP packets and would like to save as csv/txt. i did uses below code for extract time. print command is working fine with time. but when its save at csv file its only one date and one time (example 14:59:58) save in to csv file. can any one suggest for modify the codes for extract ARP times from pcap and save to csv CORRECTLY. Thank you.
with open("../data/" + filename + ".pcap", 'rb') as f:
pcap = dpkt.pcap.Reader(f)
requests = []
replies = []
for ts, buf in pcap:
eth = dpkt.ethernet.Ethernet(buf)
# If the packet is not arp
if eth.type != 2054:
continue
try:
arp = eth.arp
except Exception as e:
continue
src = dpkt.socket.inet_ntoa(arp.spa)
tgt = dpkt.socket.inet_ntoa(arp.tpa)
if arp.op == 2:
count_duplication(replies, src, tgt)
elif arp.op == 1:
count_duplication(requests, src, tgt)
packet_time = datetime.datetime.utcfromtimestamp(ts).strftime("%m/%d/%Y, %H:%M:%S")
print (packet_time)
save_packets(sorted(requests, key=lambda x: -x[2]), '../tmp/count-requests-xyz' + '.csv', packet_time)
# Save Packets
def save_packets(packets,filename,tcp,ts, degree_sorted):
with open(filename, 'w') as f:
for packet in packets:
data = ''
for item in packet:
data = data + str(item) + ','
f.write(data + tcp + datetime.datetime.utcfromtimestamp(ts).strftime("%m/%d/%Y, %H:%M:%S") + degree_sorted + '\n')
import socket
import datetime
import dpkt
def _inet_to_str(inet):
try:
return socket.inet_ntop(socket.AF_INET, inet)
except ValueError:
return socket.inet_ntop(socket.AF_INET6, inet)
def arp(pcap_path):
def _is_arp(packet):
return True
with open(pcap_path, 'rb') as f:
pcap = dpkt.pcap.Reader(f)
for ts, buf in pcap:
eth = dpkt.ethernet.Ethernet(buf)
if not isinstance(eth.data, dpkt.ip.IP):
continue
if not _is_arp(eth):
continue
ip = eth.data
# write to file instead of printing
print('{},{},{}'.format(_inet_to_str(ip.src), _inet_to_str(ip.dst),
datetime.datetime.utcfromtimestamp(ts).strftime("%m/%d/%Y, %H:%M:%S")))
To achive this, i have written socket_client.py and socket_server.py. I put all my affort, but i could not achieve the result.
In the current code, i want that a socket_client.py should read block device data and send to socket_server.py with the offest value from where data has been read. In socket program i just want to unpack the data contained in struct and display on the screen.
Note:
Data returned after unpack will be returned in tuple.
My code contains lots of debugging lines. Please let me know if you
need any other details.
Please let me know if there is any better way to do this. I am new to pyhon. So as per my understanding i tried to write this code to achive the result.
Need help from all of you.
socket_client.py:
import socket
import sys
import os,pickle
import struct as st
dev_read_data = os.open("/dev/sdc1",os.O_RDONLY)
buffer_size = 230400 #1048576
offset = 0
sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
server_address = ('localhost',10000)
print ('connecting to {} port {}'.format(server_address[0],server_address[1]))
sock.connect(server_address)
data_list = []
sent_data_size = 0
try:
while True:
data = os.pread(dev_read_data,buffer_size,offset)
#Data should be binary as its string
packed_data = st.pack("%dsi" %(len(data),),data,offset)
st_size = st.calcsize("%dsi" %(len(data)))
print ('st_size {}'.format(st_size))
packed_st_size = st.pack("i",st_size)
sock.send(packed_st_size)
#Sending data
while sent_data_size < st_size: #Send data till sent_data_size < st_size i.e. struct size
print ('{} size expected to recev'.format(st_size))
print ('value of sent_data_size before send command {}'.format(sent_data_size))
sock.send(packed_data)
sent_data_size = sock.recv(16) # rec the amount of data sent from socket_client.py to socket_server.py. This info is sent by socket_server.py
print ('value of sent_data_size sock.send command {}'.format(sent_data_size))
sent_data_size = st.unpack("Q",sent_data_size) #unpacking sent_data_size so that it can be compared in while loop condition
print ('value of sent_data_size sock.send command {}'.format(sent_data_size))
print ('Inside while loop')
#print ('Sent from offset {} data size {}'.format(offset,buffer_size))
print ('Outside while loop')
offset += buffer_size # Changing the offset to read new data using pread.
if offset == 10036977152:
break
finally:
print ('Closing socket')
sock.close()
socket_server.py:
import sys
import socket
import os
import pickle
import struct as st
#create a tcp/ip socket
dev_write_data = os.open("/dev/sdb1",os.O_WRONLY)
buffer_size = 230400 #1048576
sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
server_address = ('localhost',10000)
print ('Starting up on {} port {}'.format(server_address[0],server_address[1]))
sock.bind(server_address)
sock.listen(5)
data_size = 0
while True:
print ('waiting for a connection')
connection,client_address = sock.accept()
try:
print ('connection from {}'.format(client_address))
packed_st_size = connection.recv(16) #rec size of struct sent from socket_client.py
print ('Struct size {}'.format(packed_st_size))
st_size = st.unpack("i",packed_st_size)[0] # After unpacking data returned is tuple. So this stmt takes 0th index data from tuple and initializing to st_size
print ('Outside while loop struct size {}'.format(st_size))
print ('Outside while Data_Size {}'.format(data_size))
while data_size <= st_size: #Keep looping till rec data_size <= st_size i.e size of struct which was actually sent.
print ('Inside while loop struct size {}'.format(st_size))
print ('Outside while Data_Size {}'.format(data_size))
if data_size == 0: # if data_size is 0, data variable is initialized with data send by socket_client.py
data = connection.recv(st_size)
else:
data += connection.recv(st_size) #Otherwise byte stream data is concatinated with sequentially sent data by socket_client.py
x = len(data) #get data len.
data_len_rec = st.pack("Q",x) #get the data len rec in bytes using struct pack method
connection.send(data_len_rec) #Sending the len of data recv to socket_client.py
print ('{} bytes received.'.format(x))
data_size += len(data) # Adding length of rec data to data_size for iteration
#final_data = st.unpack("%dsi" %(len(data)),data)
#print ('Final Data recv {}'.format(final_data))
finally:
connection.close()
I am building a UDP server to parse and verify incoming UDP packets. I am able to receive and parse packets but the header values are not what I expected.
This is structure of incoming packet
Packet ID ( 4 bytes )
Packet Sequence ( 4 bytes )
XOR Key ( 2 bytes )
Number of Checksums in packet ( 2 bytes )
Cyclic checksum CRC32 (variable)
To send the packet,
with open('payloadfile.bin') as op:
payload = pickle.load(op)
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
for i in payload:
sock.sentto(payload, ('127.0.0.1',4545))
To receive and parse this packet
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind('127.0.0.1',4545)
while 1:
packet = sock.recvfrom(65565)
packet = packet[0]
# parse IP
ip_header = packet[0:20]
iph = struct.unpack('!BBHHHBBH4s4s' , ip_header)
#all the following values are incorrect
version_ihl = iph[0]
version = version_ihl >> 4
ihl = version_ihl & 0xF
ttl = iph[5]
protocol = iph[6]
s_addr = socket.inet_ntoa(iph[8]);
d_addr = socket.inet_ntoa(iph[9]);
# parse UDP
packet = packet[20:28]
data = packet[header_length:]
source_port, dest_port, data_length, checksum = struct.unpack("!HHHH", header)
From what I understand so far, this should be the general structure
IP_HEADER ( UDP_HEADER ( PAYLOAD )))
I want to parse the headers correctly, and then extract the payload.
Unfortunately the standard socket interface doesn't give you access to the data frames that your data arrive in, neither does it include the IP Datagram headers nor the TCP/UDP headers from the transport layer.
To get hold of lower-level data you are forced to use the so-called raw socket interface, which Windows for one tries to block you from using because you might be a hacker. This article might give you some clues.
How can I parse a ICMP packet (using dpkt) to check if it is a request or a response coming from A to B?
I found some examples for TCP and UDP packets (below) but I can't find anything for IP packets.
import dpkt
f = open('test.pcap')
pcap = dpkt.pcap.Reader(f)
for ts, buf in pcap:
eth = dpkt.ethernet.Ethernet(buf)
ip = eth.data
tcp = ip.data
if tcp.dport == 80 and len(tcp.data) > 0:
http = dpkt.http.Request(tcp.data)
print http.uri
f.close()
Also, is there any good tutorial for dpkt?
This is an old question but for folks coming across this, I've added an ICMP example to the dpkt repo. The docs can be found here: http://dpkt.readthedocs.io/en/latest/print_icmp.html and the example code can be found in dpkt/examples/print_icmp.py
# For each packet in the pcap process the contents
for timestamp, buf in pcap:
# Unpack the Ethernet frame (mac src/dst, ethertype)
eth = dpkt.ethernet.Ethernet(buf)
# Make sure the Ethernet data contains an IP packet
if not isinstance(eth.data, dpkt.ip.IP):
print 'Non IP Packet type not supported %s\n' % eth.data.__class__.__name__
continue
# Now grab the data within the Ethernet frame (the IP packet)
ip = eth.data
# Now check if this is an ICMP packet
if isinstance(ip.data, dpkt.icmp.ICMP):
icmp = ip.data
# Pull out fragment information
do_not_fragment = bool(ip.off & dpkt.ip.IP_DF)
more_fragments = bool(ip.off & dpkt.ip.IP_MF)
fragment_offset = ip.off & dpkt.ip.IP_OFFMASK
# Print out the info
print 'Timestamp: ', str(datetime.datetime.utcfromtimestamp(timestamp))
print 'Ethernet Frame: ', mac_addr(eth.src), mac_addr(eth.dst), eth.type
print 'IP: %s -> %s (len=%d ttl=%d DF=%d MF=%d offset=%d)' % \
(inet_to_str(ip.src), inet_to_str(ip.dst), ip.len, ip.ttl,
do_not_fragment, more_fragments, fragment_offset)
print 'ICMP: type:%d code:%d checksum:%d data: %s\n' % (icmp.type,
icmp.code, icmp.sum, repr(icmp.data))
Example Output
Timestamp: 2013-05-30 22:45:17.283187
Ethernet Frame: 60:33:4b:13:c5:58 02:1a:11:f0:c8:3b 2048
IP: 192.168.43.9 -> 8.8.8.8 (len=84 ttl=64 DF=0 MF=0 offset=0)
ICMP: type:8 code:0 checksum:48051 data: Echo(id=55099, data='Q\xa7\xd6}\x00\x04Q\xe4\x08\t\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !"#$%&\'()*+,-./01234567')
Timestamp: 2013-05-30 22:45:17.775391
Ethernet Frame: 02:1a:11:f0:c8:3b 60:33:4b:13:c5:58 2048
IP: 8.8.8.8 -> 192.168.43.9 (len=84 ttl=40 DF=0 MF=0 offset=0)
ICMP: type:0 code:0 checksum:50099 data: Echo(id=55099, data='Q\xa7\xd6}\x00\x04Q\xe4\x08\t\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !"#$%&\'()*+,-./01234567')