I am trying to write a python program using scapy to get various details of the packet such as src/dest IP/port #. As part of this I need to get the packet time of arrival and the time between packets in each direction, and the total number of packets and bytes in each direction.
It is very similar to Can reading sessions from a pcap file with Scapy be made more memory efficient
In the code below, pkttime is not working. How do I get the packet arrival time? Thanks for your help.
from scapy.all import *
pcap_file = "test.pcap"
a = rdpcap(pcap_file)
sessions = a.sessions()
for k, v in sessions.items():
tot_packets = len(v)
proto, source, dir, target = k.split()
srcip, srcport = source.split(":")
dstip, dstport = target.split(":")
if dir == '>':
direction="outbound"
else:
direction="inbound"
pkttime = v.time
print('%s,%s,%s,%s,%s,%s,%s,%s\n' % (srcip, dstip, proto, srcport,
dstport, tot_packets, direction, pkttime))
Related
Below is the code of NetworkScanner that I tried to build as my first Python project.
#!/usr/bin/env python
import scapy.all as scapy
import optparse
def scan(ip):
packet1 = scapy.ARP(pdst=ip)
etherpacket = scapy.Ether(dst='ff:ff:ff:ff:ff:ff')
broadcast_packet = etherpacket / packet1
ans = scapy.srp(broadcast_packet, timeout=60, verbose=False)[0]
ret_list = list()
for item in ans:
dic = {}
dic['ip'] = item[1].pdst
dic['mac'] = item[1].hwdst
ret_list.append(dic)
print(ret_list)
return ret_list
def printfun(returnlist):
print("IP\t\t\tMAC Address\n----------------------------------------------")
for elem in returnlist:
print(elem["ip"] + "\t\t" + elem["mac"])
def getip():
parser = optparse.OptionParser()
parser.add_option('-i', "--ip", dest = 'received_ip', help="Please enter the ip you want to scan")
(option, arguments) = parser.parse_args()
return option.received_ip
ip = getip()
if ip:
result = scan(ip)
printfun(result)
else:
print("no ip given")
Now I did follow some tutorials and learned to build parallelly and it seems right to me but I am not good.
but when I execute the program, it only scans the IP of virtual Host itself on which it is executed.
/PycharmProjects/Networkscanner$ sudo python networkscanner.py -i 192.168.1.1/24
[{'ip': '192.168.1.205', 'mac': '08:00:27:1f:30:76'}, {'ip': '192.168.1.205', 'mac': '08:00:27:1f:30:76'}]
IP MAC Address
----------------------------------------------
192.168.1.205 08:00:27:1f:30:76
192.168.1.205 08:00:27:1f:30:76
when I use the inbuild network scanner of python it gives these results.
Currently scanning: Finished! | Screen View: Unique Hosts
5 Captured ARP Req/Rep packets, from 4 hosts. Total size: 300
_____________________________________________________________________________
IP At MAC Address Count Len MAC Vendor / Hostname
-----------------------------------------------------------------------------
192.168.1.1 a0:47:d7:36:2a:c0 2 120 Best IT World (India) Pvt Lt
192.168.1.203 e4:42:a6:30:93:64 1 60 Intel Corporate
192.168.1.205 30:b5:c2:10:05:3b 1 60 TP-LINK TECHNOLOGIES CO.,LTD
192.168.1.207 30:b5:c2:10:05:3b 1 60 TP-LINK TECHNOLOGIES CO.,LTD
Edit:
I tried the Monitor mode, but that does not help.
I also tried to run it on main windows with also an external WiFi adaptor, still same issue
can someone please assist what is wrong in my code?
Determining the problem
If we name variables more appropriately, it becomes apparent what the problem is:
for sent_recv in ans:
dic = {}
return_packet = sent_recv[1]
dic['ip'] = return_packet.pdst
dic['mac'] = return_packet.hwdst
ret_list.append(dic)
Each item is a tuple of a sent packet and a received packet. Naming it as such helps to identify it.
The 2nd element of sent_recv is the returned packet. Let's name it as such.
The destination IP and MAC address of the returning packet are going to be that of our machine. This makes sense in the context of your output: You get your own IP/MAC for every response.
So if we change it to the src IP/MAC of the returning packet, we'll get the information we're after:
for sent_recv in ans:
dic = {}
return_packet = sent_recv[1]
# Difference is dst => src here
dic['ip'] = return_packet.psrc
dic['mac'] = return_packet.hwsrc
ret_list.append(dic)
Note: An ARP packet should not take 60s to return - it's more on the scale of 10-100ms depending on network size. A timeout of 2s is more appropriate here.
Testing the fix
Running this with modified code, we get the desired result:
$ python script.py -i "192.168.1.0/24"
[{'ip': '192.168.1.48', 'mac': '00:1b:78:20:ee:40'},
{'ip': '192.168.1.67', 'mac': '6c:33:a9:42:6a:18'},
...
IP MAC Address
----------------------------------------------
192.168.1.48 00:1b:78:20:ee:40
192.168.1.67 6c:33:a9:42:6a:18
...
In l3_learning.py, there is a method in class l3_switch named _handle_PacketIn.
Now I understand that this event is when a switch contacts controller when it receives a packet corresponding to which it has no entries in its table.
What I don't understand is that here
packet = event.parsed
Now what does packet.next mean in isinstance(packet.next, ipv4)?
def _handle_PacketIn (self, event):
dpid = event.connection.dpid
inport = event.port
packet = event.parsed
if not packet.parsed:
log.warning("%i %i ignoring unparsed packet", dpid, inport)
return
if dpid not in self.arpTable:
# New switch -- create an empty table
self.arpTable[dpid] = {}
for fake in self.fakeways:
self.arpTable[dpid][IPAddr(fake)] = Entry(of.OFPP_NONE,
dpid_to_mac(dpid))
if packet.type == ethernet.LLDP_TYPE:
# Ignore LLDP packets
return
if isinstance(packet.next, ipv4):
log.debug("%i %i IP %s => %s", dpid,inport,
packet.next.srcip,packet.next.dstip)
# Send any waiting packets...
self._send_lost_buffers(dpid, packet.next.srcip, packet.src, inport)
# Learn or update port/MAC info
if packet.next.srcip in self.arpTable[dpid]:
if self.arpTable[dpid][packet.next.srcip] != (inport, packet.src):
log.info("%i %i RE-learned %s", dpid,inport,packet.next.srcip)
else:
log.debug("%i %i learned %s", dpid,inport,str(packet.next.srcip))
self.arpTable[dpid][packet.next.srcip] = Entry(inport, packet.src)
# Try to forward
dstaddr = packet.next.dstip
if dstaddr in self.arpTable[dpid]:
# We have info about what port to send it out on...
prt = self.arpTable[dpid][dstaddr].port
mac = self.arpTable[dpid][dstaddr].mac
I think I have figured it out.
packet is entire packet that data-link layer sends to the physical layer. packet.next removes the encapsulation of data-link layer and reveals the IP packet (the packet sent by IP layer to data-link layer). So to get the source MAC address we use packet.src and to get the IP address of the source we use packet.next.srcip
I am trying to measure the responses back from DNS servers. Making a sniffer for a typical DNS response that is less than 512 bytes is no big deal. My issue is receiving large 3000+ byte responses - in some cases 5000+ bytes. I haven't been able to get a socket working that can receive that data reliably. Is there a way with Python sockets to receive from a specific source address?
Here is what I have so far:
import socket
import struct
def craft_dns(Qdns):
iden = struct.pack('!H', randint(0, 65535))
QR_thru_RD = chr(int('00000001', 2)) # '\x01'
RA_thru_RCode = chr(int('00100000', 2)) # '\x00'
Qcount = '\x00\x01' # question count is 1
ANcount = '\x00\x00'
NScount = '\x00\x00'
ARcount = '\x00\x01' # additional resource count is 1
pad = '\x00' #
Rtype_ANY = '\x00\xff' # Request ANY record
PROtype = '\x00\x01' # Protocol IN || '\x00\xff' # Protocol ANY
DNSsec_do = chr(int('10000000', 2)) # flips DNSsec bit to enable
edns0 = '\x00\x00\x29\x10\x00\x00\x00\x00\x00\x00\x00' # DNSsec disabled
domain = Qdns.split('.')
quest = ''
for x in domain:
quest += struct.pack('!B', len(x)) + x
packet = (iden+QR_thru_RD+RA_thru_RCode+Qcount+ANcount+NScount+ARcount+
quest+pad+Rtype_ANY+PROtype+edns0) # remove pad if asking <root>
return packet
def craft_ip(target, resolv):
ip_ver_len = int('01000101', 2) # IPvers: 4, 0100 | IP_hdr len: 5, 0101 = 69
ipvers = 4
ip_tos = 0
ip_len = 0 # socket will put in the right length
iden = randint(0, 65535)
ip_frag = 0 # off
ttl = 255
ip_proto = socket.IPPROTO_UDP # dns, brah
chksm = 0 # socket will do the checksum
s_addr = socket.inet_aton(target)
d_addr = socket.inet_aton(resolv)
ip_hdr = struct.pack('!BBHHHBBH4s4s', ip_ver_len, ip_tos, ip_len, iden,
ip_frag, ttl, ip_proto, chksm, s_addr, d_addr)
return ip_hdr
def craft_udp(sport, dest_port, packet):
#sport = randint(0, 65535) # not recommended to do a random port generation
udp_len = 8 + len(packet) # calculate length of UDP frame in bytes.
chksm = 0 # socket fills in
udp_hdr = struct.pack('!HHHH', sport, dest_port, udp_len, chksm)
return udp_hdr
def get_len(resolv, domain):
target = "10.0.0.3"
d_port = 53
s_port = 5353
ip_hdr = craft_ip(target, resolv)
dns_payload = craft_dns(domain) # '\x00' for root
udp_hdr = craft_udp(s_port, d_port, dns_payload)
packet = ip_hdr + udp_hdr + dns_payload
buf = bytearray("-" * 60000)
recvSock = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0800))
recvSock.settimeout(1)
sendSock = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_RAW)
sendSock.settimeout(1)
sendSock.connect((resolv, d_port))
sendSock.send(packet)
msglen = 0
while True:
try:
pkt = recvSock.recvfrom(65535)
msglen += len(pkt[0])
print repr(pkt[0])
except socket.timeout as e:
break
sendSock.close()
recvSock.close()
return msglen
result = get_len('75.75.75.75', 'isc.org')
print result
For some reason doing
pkt = sendSock.recvfrom(65535)
Recieves nothing at all. Since I'm using SOCK_RAW the above code is less than ideal, but it works - sort of. If the socket is extremely noisy (like on a WLAN), I could end up receiving well beyond the DNS packets, because I have no way to know when to stop receiving packets when receiving a multipacket DNS answer. For a quiet network, like a lab VM, it works.
Is there a better way to use a receiving socket in this case?
Obviously from the code, I'm not that strong with Python sockets.
I have to send with SOCK_RAW because I am constructing the packet in a raw format. If I use SOCK_DGRAM the custom packet will be malformed when sending to a DNS resolver.
The only way I could see is to use the raw sockets receiver (recvSock.recv or recvfrom) and unpack each packet, look if the source and dest address match within what is supplied in get_len(), then look to see if the fragment bit is flipped. Then record the byte length of each packet with len(). I'd rather not do that. It just seems there is a better way.
Ok I was stupid and didn't look at the protocol for the receiving socket. Socket gets kind of flaky when you try to receive packets on a IPPROTO_RAW protocol, so we do need two sockets. By changing to IPPROTO_UDP and then binding it, the socket was able to follow the complete DNS response over multiple requests. I got rid of the try/catch and the while loop, as it was no longer necessary and I'm able to pull the response length with this block:
recvSock = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_UDP)
recvSock.settimeout(.3)
recvSock.bind((target, s_port))
sendSock = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_RAW)
#sendSock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sendSock.settimeout(.3)
sendSock.bind((target, s_port))
sendSock.connect((resolv, d_port))
sendSock.send(packet)
pkt = recvSock.recvfrom(65535)
msglen = len(pkt[0])
Now the method will return the exact bytes received from a DNS query. I'll leave this up in case anyone else needs to do something similar :)
I'm trying to check for errors in a log file of a running embedded system.
I already have implemented paramiko in my scripts, as I've been told this is the best way to use ssh in python.
Now when I tail the log file I see that there is a big delay build up. Which increases with about 30 seconds per minute.
I already used a grep to decrease the number of lines which is printed as I thought I was receiving too much input but that isn't the case.
How can I decrease this delay or stop the delay from increasing during runtime. I want to tail for hours...
def mkssh_conn(addr):
"""returns an sshconnection"""
paramiko.util.logging.getLogger('paramiko').setLevel(logging.WARN)
sshcon = paramiko.SSHClient()
sshcon.set_missing_host_key_policy(paramiko.AutoAddPolicy())
sshcon.connect(addr , username, password)
return sshcon
while True:
BUF_SIZE = 1024
client = mkssh_conn() #returns a paramiko.SSHClient()
transport = client.get_transport()
transport.set_keepalive(1)
channel = transport.open_session()
channel.settimeout(delta)
channel.exec_command( 'killall tail')
channel = transport.open_session()
channel.settimeout(delta)
cmd = "tail -f /log/log.log | grep -E 'error|statistics'"
channel.exec_command(cmd)
while transport.is_active():
print "transport is active"
rl, wl, xl = select.select([channel], [], [], 0.0)
if len(rl) > 0:
buf = channel.recv(BUF_SIZE)
if len(buf) > 0:
lines_to_process = LeftOver + buf
EOL = lines_to_process.rfind("\n")
if EOL != len(lines_to_process)-1:
LeftOver = lines_to_process[EOL+1:]
lines_to_process = lines_to_process[:EOL]
else:
LeftOver = ""
for line in lines_to_process.splitlines():
if "error" in line:
report_error(line)
print line
client.close()
I've found a solution:
It seems that if I lower BUF_SIZE to 256 the delay decreases. Obviously.
I need to recheck if the delay still increases during runtime or not.
BUFFER_SIZE should be on the higher end to reduce the cpu cycles ( and in turn to reduce the overall delay due to network latency ) in case if you are working with high throughput tailed pipe.
Further making the BUFFER_SIZE to higher number, should not reduce the performance. ( if paramiko doesn't wail till the buffer fills out in low throughput pipe )
Contradiction between #studioj 's answer and this, probably due to the upgrade of paramiko ( fixed now )
I have created a simple RAW socket based packet sniffer. But when I run it, it rarely captures up a packet. First I created this to capture packets in 1 second time intervals, but seeing no packets are captured I commented that line. I was connected to internet and a lot of http traffic are going here and there, but I could not capture a one. Is there a problem in this in the code where I created the socket? Please someone give me a solution. I am fairly new to python programming and could not understand how to solve this.
import socket, binascii, struct
import time
sock = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, socket.htons(0x800))
print "Waiting.."
pkt = sock.recv(2048)
print "received"
def processEth(data):
#some code to process source mac and dest. mac
return [smac, dmac]
def processIP(data):
sip = str(binascii.hexlify(data[1]))
dip = str(binascii.hexlify(data[2]))
return [sip, dip]
def processTCP(data):
sport = str(data[0])
dport = str(data[1])
return [sport, dport]
while len(pkt) > 0 :
if(len(pkt)) > 54:
pkt = sock.recv(2048)
ethHeader = pkt[0][0:14]
ipHeader = pkt[0][14:34]
tcpHeader = pkt[0][34:54]
ethH = struct.unpack("!6s6s2s",ethHeader)
ethdata = processEth(ethH)
ipH = struct.unpack("!12s4s4s",ipHeader)
ipdata = processIP(ipH)
tcpH = struct.unpack("!HH16", tcpHeader)
tcpdata = processTCP(tcpH)
print "S.mac "+ethdata[0]+" D.mac "+ethdata[1]+" from: "+ipdata[0]+":"+tcpdata[0]+" to: "+ipdata[1]+":"+tcpdata[1]
#time.sleep(1);
else:
continue
If you showed all the code, you are running into an endless loop.
Whenever a paket is coming in which has not a length greater then 54 bytes, you end up reading the same packet all the time.
Additionally, socket.recv() returns a string/byte sequence; your approach of accessing the data is wrong. pkt[0] returns a string with length 1; pkt[0][x:y] will not return something useful.
I am not familiar with using sockets, but with some changes I got output that might look similar to what you intended (there is something missing in processEth() I think...).
[...]
while len(pkt) > 0:
print "Waiting.."
pkt = sock.recv(2048)
print "received"
if(len(pkt)) > 54:
ethHeader = pkt[0:14]
ipHeader = pkt[14:34]
tcpHeader = pkt[34:38]
ethH = struct.unpack("!6s6s2s",ethHeader)
ethdata = processEth(ethH)
ipH = struct.unpack("!12s4s4s",ipHeader)
ipdata = processIP(ipH)
tcpH = struct.unpack("!HH16", tcpHeader)
tcpdata = processTCP(tcpH)
print "S.mac "+ethdata[0]+" D.mac "+ethdata[1]+" from: "+ipdata[0]+":"+tcpdata[0]+" to: "+ipdata[1]+":"+tcpdata[1]
#time.sleep(1);
else:
continue