We Keep Coding

Sniff network traffic with scapy and regular expressions?

I had my first glance on python's scapy package and wanted to test my first little script. In order to test the code below I sent myself an E-Mail containing this string "10000000000000". My expectation was that below minimal example (with my network card in monitoring mode and execution as su) would give an command line output for that E-Mail specifically but it doesn't. I am sure that this is caused by an misunderstanding of network traffic and tcp by myself -- could anyone elucidate?
The virtual environment I set-up is Python 3.6.8
(I have tested that I can sniff network traffic in general using the commented out line, only when I try to filter their content via regular expressions the result is not as I expected.)
import optparse
import scapy.all as sca
import re
from scapy import packet
from scapy.layers.dns import DNS
from scapy.layers.dot11 import Dot11Beacon, Dot11ProbeReq
from scapy.layers.inet import TCP
class SniffDataTraffic:
#staticmethod
def find_expr(pack: packet) -> str:
regex_dict = {'foo': r"1[0-9]{13}",'bar': r"2[1-5]{14}"}
raw_pack = pack.sprintf('%Raw.load%')
found = {iter: re.findall(regex_dict[iter], raw_pack) for iter in regex_dict.keys()}
for name, value in found.items():
if value:
return "[+] found {}: {}".format(name, value[0])
def main():
try:
print("[*] Starting sniffer")
#sca.sniff(iface="xxxxxmon", prn=lambda x: x.summary(), store=False)
sca.sniff(prn=SniffDataTraffic.find_expr, filter='tcp', iface="xxxxxmon", store=False)
except KeyboardInterrupt:
exit(0)
if __name__ == '__main__':
main()

GUID number of windows interface giving error: ValueError: Unknown network interface '{1619EEF1-4D71-4831-87AC-8E5DC3AA516A}'

Import scapy version 2.4.0. I am only using version 2.4.0 for my project
import scapy.all as scapy
import sys
by using IP address this function return related MAC address of the target
def get_mac(ip):
arp_request = scapy.ARP(pdst=ip)
broadcast = scapy.Ether(dst="ff:ff:ff:ff:ff:ff")
arp_request_broadcast = broadcast/arp_request
answered_list = scapy.srp(arp_request_broadcast, timeout=1, verbose=False)[0]
return answered_list[0][1].hwsrc
def sniff(interface):
scapy.sniff(iface=interface, store=False, prn=process_sniffed_packet)
This function checks whether default gateway MAC address is equal to my PC's MAC address table. if not it says "[+] You are under attack!!
def process_sniffed_packet(packet):
if packet.haslayer(scapy.ARP) and packet[scapy.ARP].op == 2:
count = 1
try:
real_mac = get_mac(packet[scapy.ARP].psrc)
response_mac = packet[scapy.ARP].hwsrc
if real_mac != response_mac:
count = count+1
print(str(count) + "[+] You are under attack!!")
sys.stdout.flush()
except IndexError:
pass
in Linux, we can use a value like 'etho' but In windows, I have to use GUID value to get the result. I am running this code in Windows Machine.
sniff('{1619EEF1-4D71-4831-87AC-8E5DC3AA516A}')
But this code return error
This is the Error that got raised
raise ValueError("Unknown network interface %r" % name)
ValueError: Unknown network interface '{1619EEF1-4D71-4831-87AC-
8E5DC3AA516A}'

On Windows, you need to provide a complete interface name / object, to be able to sniff on it.
First, have a look at what is available using IFACES.show() in a Scapy shell.
Then to get the interface, you can either use:
iface = IFACES.dev_from_name("...") (or dev_from_pcapname, dev_from_id... have a look at help(IFACES) to see what’s available)
iface = "the full name as printed above"
Then use it via sniff(iface=iface).
You could provide the pcap_name, but not the GUID: for instance, it would be something like \\Device\\NPF_{...} rather than just {...}.
Also, please use scapy 2.4.3rc1 (or at least 2.4.2) to be sure you’re up-to-date

I solved the scapy error ValueError: Unknown network interface on windows by installing npcap

how can the directory of a usb drive connected to a system be obtained?

I need to obtain the path to the directory created for a usb drive(I think it's something like /media/user/xxxxx) for a simple usb mass storage device browser that I am making. Can anyone suggest the best/simplest way to do this? I am using an Ubuntu 13.10 machine and will be using it on a linux device.
Need this in python.

This should get you started:
#!/usr/bin/env python
import os
from glob import glob
from subprocess import check_output, CalledProcessError
def get_usb_devices():
sdb_devices = map(os.path.realpath, glob('/sys/block/sd*'))
usb_devices = (dev for dev in sdb_devices
if 'usb' in dev.split('/')[5])
return dict((os.path.basename(dev), dev) for dev in usb_devices)
def get_mount_points(devices=None):
devices = devices or get_usb_devices() # if devices are None: get_usb_devices
output = check_output(['mount']).splitlines()
is_usb = lambda path: any(dev in path for dev in devices)
usb_info = (line for line in output if is_usb(line.split()[0]))
return [(info.split()[0], info.split()[2]) for info in usb_info]
if __name__ == '__main__':
print get_mount_points()
How does it work?
First, we parse /sys/block for sd* files (courtesy of https://stackoverflow.com/a/3881817/1388392) to filter out usb devices.
Later you call mount and parse output for lines only for those devices.
Of course they might be some edge cases, when this won't work, portability issues etc. Or better ways to do it. But for more information you should rather seek help on SuperUser or ServerFault, with more experienced linux hackers.

I had to modify #m.wasowski 's code to make it work on Python3.5.4 as follows.
def get_mount_points(devices=None):
devices = devices or get_usb_devices() # if devices are None: get_usb_devices
output = check_output(['mount']).splitlines()
output = [tmp.decode('UTF-8') for tmp in output]
def is_usb(path):
return any(dev in path for dev in devices)
usb_info = (line for line in output if is_usb(line.split()[0]))
return [(info.split()[0], info.split()[2]) for info in usb_info]

Using m.wasowski code, unexpected behavior can occur:
return [(info.split()[0], info.split()[2]) for info in usb_info]
This part of code can produce bug, if your USB device name has white space character in it. I got that behavior with device named "USB DEVICE".
info.split()[2]
Returned media/home/USB for me, when it is media/home/USB DEVICE.
I modified that part, so it was founding word "type", and replaced that line with this:
#return [(info.split()[0], info.split()[2]) for info in usb_info]
fullInfo = []
for info in usb_info:
print(info)
mountURI = info.split()[0]
usbURI = info.split()[2]
print(info.split().__sizeof__())
for x in range(3, info.split().__sizeof__()):
if info.split()[x].__eq__("type"):
for m in range(3, x):
usbURI += " "+info.split()[m]
break
fullInfo.append([mountURI, usbURI])
return fullInfo

I had to further modify #nick-sikrier and #m-wasowski response to handle LUKs encrypted devices.
def get_usb_devices():
sdb_devices = map(os.path.realpath, glob('/sys/block/sd*'))
usb_devices = (dev for dev in sdb_devices
if any(['usb' in dev.split('/')[5],
'usb' in dev.split('/')[6]]))
return dict((os.path.basename(dev), dev) for dev in usb_devices)
def get_mount_points(
devices = get_usb_devices()
fullInfo = []
for dev in devices:
output = subprocess.check_output(['lsblk', '-lnpo', 'NAME,MOUNTPOINT', '/dev/' + dev]).splitlines()
for mnt_point in output:
mnt_point_split = mnt_point.split(' ', 1)
if len(mnt_point_split) > 1 and mnt_point_split[1].strip():
fullInfo.append([mnt_point_split[0], mnt_point_split[1]])
return fullInfo

With a simple shell pipe executed in python:
import subprocess
driver_name = "my_usb_stick"
path = subprocess.check_output("cat /proc/mounts | grep '"+driver_name+"' | awk '{print $2}'", shell=True)
path = path.decode('utf-8') # convert bytes in string
>>> "/media/user/my_usb_stick"
Explanations
/proc/mounts/ : Is a file listing all mounted devices
The 1st column specifies the device that is mounted.
The 2nd column reveals the mount point.
The 3rd column tells the file-system type.
The 4th column tells you if it is mounted read-only (ro) or read-write (rw).
The 5th and 6th columns are dummy values designed to match the format used in /etc/mtab
More details see this answer : How to interpret /proc/mounts?
grep returns the line containing your driver's name
awk returns the 2nd columns, aka the mount point, aka your path.

Listing available com ports with Python

I am searching for a simple method to list all available com port on a PC.
I have found this method but it is Windows-specific: Listing serial (COM) ports on Windows?
I am using Python 3 with pySerial on a Windows 7 PC.
I have found in the pySerial API (http://pyserial.sourceforge.net/pyserial_api.html) a function serial.tools.list_ports.comports() that lists com ports (exactly what I want).
import serial.tools.list_ports
print(list(serial.tools.list_ports.comports()))
But it seems that it doesn't work. When my USB to COM gateway is connected to the PC (I see the COM5 in the Device Manager), this COM port isn't included in the list returned by list_ports.comports(). Instead I only get COM4 which seems to be connected to a modem (I don't see it in the COM&LPT section of Device Manager)!
Do you know why it doesn't work? Have you got another solution which is not system specific?

This is the code I use.
Successfully tested on Windows 8.1 x64, Windows 10 x64, Mac OS X 10.9.x / 10.10.x / 10.11.x and Ubuntu 14.04 / 14.10 / 15.04 / 15.10 with both Python 2 and Python 3.
import sys
import glob
import serial
def serial_ports():
""" Lists serial port names
:raises EnvironmentError:
On unsupported or unknown platforms
:returns:
A list of the serial ports available on the system
"""
if sys.platform.startswith('win'):
ports = ['COM%s' % (i + 1) for i in range(256)]
elif sys.platform.startswith('linux') or sys.platform.startswith('cygwin'):
# this excludes your current terminal "/dev/tty"
ports = glob.glob('/dev/tty[A-Za-z]*')
elif sys.platform.startswith('darwin'):
ports = glob.glob('/dev/tty.*')
else:
raise EnvironmentError('Unsupported platform')
result = []
for port in ports:
try:
s = serial.Serial(port)
s.close()
result.append(port)
except (OSError, serial.SerialException):
pass
return result
if __name__ == '__main__':
print(serial_ports())

Basically mentioned this in pyserial documentation
https://pyserial.readthedocs.io/en/latest/tools.html#module-serial.tools.list_ports
import serial.tools.list_ports
ports = serial.tools.list_ports.comports()
for port, desc, hwid in sorted(ports):
print("{}: {} [{}]".format(port, desc, hwid))
Result :
COM1: Communications Port (COM1) [ACPI\PNP0501\1]
COM7: MediaTek USB Port (COM7) [USB VID:PID=0E8D:0003 SER=6 LOCATION=1-2.1]

You can use:
python -c "import serial.tools.list_ports;print serial.tools.list_ports.comports()"
Filter by know port:
python -c "import serial.tools.list_ports;print [port for port in serial.tools.list_ports.comports() if port[2] != 'n/a']"
See more info here:
https://pyserial.readthedocs.org/en/latest/tools.html#module-serial.tools.list_ports

one line solution with pySerial package.
python -m serial.tools.list_ports

A possible refinement to Thomas's excellent answer is to have Linux and possibly OSX also try to open ports and return only those which could be opened. This is because Linux, at least, lists a boatload of ports as files in /dev/ which aren't connected to anything. If you're running in a terminal, /dev/tty is the terminal in which you're working and opening and closing it can goof up your command line, so the glob is designed to not do that. Code:
# ... Windows code unchanged ...
elif sys.platform.startswith ('linux'):
temp_list = glob.glob ('/dev/tty[A-Za-z]*')
result = []
for a_port in temp_list:
try:
s = serial.Serial(a_port)
s.close()
result.append(a_port)
except serial.SerialException:
pass
return result
This modification to Thomas's code has been tested on Ubuntu 14.04 only.

refinement on moylop260's answer:
import serial.tools.list_ports
comlist = serial.tools.list_ports.comports()
connected = []
for element in comlist:
connected.append(element.device)
print("Connected COM ports: " + str(connected))
This lists the ports that exist in hardware, including ones that are in use. A whole lot more information exists in the list, per the pyserial tools documentation

Probably late, but might help someone in need.
import serial.tools.list_ports
class COMPorts:
def __init__(self, data: list):
self.data = data
#classmethod
def get_com_ports(cls):
data = []
ports = list(serial.tools.list_ports.comports())
for port_ in ports:
obj = Object(data=dict({"device": port_.device, "description": port_.description.split("(")[0].strip()}))
data.append(obj)
return cls(data=data)
#staticmethod
def get_description_by_device(device: str):
for port_ in COMPorts.get_com_ports().data:
if port_.device == device:
return port_.description
#staticmethod
def get_device_by_description(description: str):
for port_ in COMPorts.get_com_ports().data:
if port_.description == description:
return port_.device
class Object:
def __init__(self, data: dict):
self.data = data
self.device = data.get("device")
self.description = data.get("description")
if __name__ == "__main__":
for port in COMPorts.get_com_ports().data:
print(port.device)
print(port.description)
print(COMPorts.get_device_by_description(description="Arduino Leonardo"))
print(COMPorts.get_description_by_device(device="COM3"))

Please, try this code:
import serial
ports = serial.tools.list_ports.comports(include_links=False)
for port in ports :
print(port.device)
first of all, you need to import package for serial port communication,
so:
import serial
then you create the list of all the serial ports currently available:
ports = serial.tools.list_ports.comports(include_links=False)
and then, walking along whole list, you can for example print port names:
for port in ports :
print(port.device)
This is just an example how to get the list of ports and print their names, but there some other options you can do with this data. Just try print different variants after
port.

something simple but I use it a lot.
import serial.tools.list_ports as ports
com_ports = list(ports.comports()) # create a list of com ['COM1','COM2']
for i in com_ports:
print(i.device) # returns 'COMx'

try this code
import serial.tools.list_ports
for i in serial.tools.list_ports.comports():
print(i)
it returns
COM1 - Port de communication (COM1)
COM5 - USB-SERIAL CH340 (COM5)
if you just wont the name of the port for exemple COM1
import serial.tools.list_ports
for i in serial.tools.list_ports.comports():
print(str(i).split(" ")[0])
it returns
COM1
COM5
as in my case
py 3.7 64bits

Works only on Windows:
import winreg
import itertools
def serial_ports() -> list:
path = 'HARDWARE\\DEVICEMAP\\SERIALCOMM'
key = winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE, path)
ports = []
for i in itertools.count():
try:
ports.append(winreg.EnumValue(key, i)[1])
except EnvironmentError:
break
return ports
if __name__ == "__main__":
ports = serial_ports()

Several options are available:
Call QueryDosDevice with a NULL lpDeviceName to list all DOS devices. Then use CreateFile and GetCommConfig with each device name in turn to figure out whether it's a serial port.
Call SetupDiGetClassDevs with a ClassGuid of GUID_DEVINTERFACE_COMPORT.
WMI is also available to C/C++ programs.
There's some conversation on the win32 newsgroup and a CodeProject, er, project.

One thing to note, codes like this:
for i in serial.tools.list_ports.comports():
print(i)
Return the following:
COM7 - Standard Serial over Bluetooth link (COM7) COM1 - Communications Port (COM1) COM8 - Standard Serial over Bluetooth link (COM8) COM4 - USB-SERIAL CH340 (COM4)
If you want the ports listed in order, and only the ones available to you, try:(credit to tfeldmann)
def serial_ports():
""" Lists serial port names
:raises EnvironmentError:
On unsupported or unknown platforms
:returns:
A list of the serial ports available on the system
"""
if sys.platform.startswith('win'):
ports = ['COM%s' % (i + 1) for i in range(256)]
elif sys.platform.startswith('linux') or sys.platform.startswith('cygwin'):
# this excludes your current terminal "/dev/tty"
ports = glob.glob('/dev/tty[A-Za-z]*')
elif sys.platform.startswith('darwin'):
ports = glob.glob('/dev/tty.*')
else:
raise EnvironmentError('Unsupported platform')
result = []
for port in ports:
try:
s = serial.Serial(port)
s.close()
result.append(port)
except (OSError, serial.SerialException):
pass
return result
This returns the following:
['COM1', 'COM4', 'COM8']
So unlike the first example, where the result was ['COM7', 'COM1', 'COM8', 'COM4'], this time I get all of the com ports in order, and only the ones available. Very handy if you need them in order, and tested to see if they're available.

Simplest way to publish over Zeroconf/Bonjour?

I've got some apps I would like to make visible with zeroconf.
Is there an easy scriptable way to do this?
Is there anything that needs to be done by my network admin to enable this?
Python or sh would be preferrable. OS-specific suggestions welcome for Linux and OS X.

pybonjour doesn't seem to be actively maintained. I'm using python-zeroconf.
pip install zeroconf
Here is an excerpt from a script I use to announce a Twisted-Autobahn WebSocket to an iOS device:
from zeroconf import ServiceInfo, Zeroconf
class WebSocketManager(service.Service, object):
ws_service_name = 'Verasonics WebSocket'
wsPort = None
wsInfo = None
def __init__(self, factory, portCallback):
factory.protocol = BroadcastServerProtocol
self.factory = factory
self.portCallback = portCallback
self.zeroconf = Zeroconf()
def privilegedStartService(self):
self.wsPort = reactor.listenTCP(0, self.factory)
port = self.wsPort.getHost().port
fqdn = socket.gethostname()
ip_addr = socket.gethostbyname(fqdn)
hostname = fqdn.split('.')[0]
wsDesc = {'service': 'Verasonics Frame', 'version': '1.0.0'}
self.wsInfo = ServiceInfo('_verasonics-ws._tcp.local.',
hostname + ' ' + self.ws_service_name + '._verasonics-ws._tcp.local.',
socket.inet_aton(ip_addr), port, 0, 0,
wsDesc, hostname + '.local.')
self.zeroconf.register_service(self.wsInfo)
self.portCallback(port)
return super(WebSocketManager, self).privilegedStartService()
def stopService(self):
self.zeroconf.unregister_service(self.wsInfo)
self.wsPort.stopListening()
return super(WebSocketManager , self).stopService()

Or you can just use bash:
dns-sd -R <Name> <Type> <Domain> <Port> [<TXT>...]
This works by default on OS X. For other *nixes, refer to the avahi-publish man page (which you may need to install via your preferred package manager).

I'd recommend pybonjour.

Although this answer points you in the right direction, it seems that python-zeroconf (0.39.4) had some changes making the example above not work (for me) anymore.
Also I think a more minimal, self-contained, answer would be nice, so here goes:
from zeroconf import ServiceInfo, Zeroconf
PORT=8080
zeroconf = Zeroconf()
wsInfo = ServiceInfo('_http._tcp.local.',
"myhost._http._tcp.local.",
PORT, 0, 0, {"random_key": "1234", "answer": "42"})
zeroconf.register_service(wsInfo)
import time
time.sleep(1000);
Note that anything beyond PORT is optional for ServiceInfo().
You can run multiple of these programs at the same time; they will all bind to the same UDP port without a problem.

Through the Avahi Python bindings, it's very easy.