I am using Scapy to capture Wi-Fi client request frames. I am only interested in the MAC and requested SSID address of the clients. I do something like the following.
sniff(iface="mon0", prn=Handler)
def Handler(pkt):
if pkt.hasLayer(Dot11):
if pkt.type == 0 and pkt.subtype == 4:
print pkt.addr2 + " " + pkt.info
My issue is that I am doing this on a embedded device with limited processing power. When I run my script, my processor utilization rises to nearly 100%. I assume this is because of the sheer volume of frames that Scapy is sniffing and passing to my Python code. I also assume that if I could employ the right filter in my sniff command, I could eliminate many of the frames that are not being used and thus reduce the processor load.
Is there a filter statement that could be used to do this?
With Scapy it is possible to sniff with an BPF Filter applied to the capture. This will filter out packets at a much lower level then your Handler function is doing, and should significantly improve performance.
There is a simple example in the Scapy documentation.
# Could also add a subtype to this.
sniff(iface="mon0", prn=Handler, filter="type mgt")
Filter pieced together from here specifically.
Unfortunately I can't test this right now, but this information should provide you with a stepping stone to your ultimate solution, or someone else to post exactly what you need. I believe you will also need to set the interface to monitor mode.
You may also find this question of interest - Accessing 802.11 Wireless Management Frames from Python.
Scapy is extremely slow due to the way it decodes the data. You may
Use a BPF-filter on the input to only get the frames you are looking for before handing them to scapy. See this module for example. The module uses libpcap to get the data from the air or from a file first and passes it through a dynamically updated BPF-filter to keep unwanted traffic out
Write your own parser for wifi in c (which is not too hard, given the limited amount of information you need, there are things like prismhead though)
Use tshark from wireshark as a subprocess and collect data from there
I highly recommend the third approach although wiresharek comes with a >120mb library that your embedded device might not be able to handle.
Related
I have a similar question as this one but the solution there did not apply to my problem. I can connect and send commands to my Keysight B1500 mainframe, via pyvisa/GPIB. The B1500 is connected via Keysight's IO tool "Connection Expert"
rman = pyvisa.ResourceManager()
keyS = rman.open_resource('GPIB0::18::INSTR')
keyS.timeout = 20000 # time in ms
keyS.chunk_size = 8204800 # 102400 is 100 kB
keyS.write('*rst; status:preset; *cls')
print('variable keyS is being assigned to ',keyS.query('*IDN?'))
Using this pyvisa object I can query without issues (*IDN? above provides the expected output), and I have also run and extracted data from a different type of IV curve on the same tool.
However, when I try to run a pulsed voltage sweep (change voltage of pulses as function of time and measure current) I do not get the measured data out from the tool. I can hook the output lead from the B1500 to an oscilloscope and can see that my setup has worked and the tool is behaving as expected, right up until I try to extract sweep data.
Again, I can run a standard non-pulsed sweep on the tool and the data extraction works fine using [pyvisaobject].read_raw() - so something is different with the way I'm pulsing the voltage.
What I'm looking for is a way to interrogate the connection in cases where the data transfer is unsuccessful.
Here, in no particular order, are the ways I've tried to extract data. These methods are suggested in this link:
keyS.query_ascii_values('CURV?')
or
keyS.read_ascii_values()
or
keyS.query_binary_values('CURV?')
or
keyS.read_binary_values()
This link from the vendor does cover the extraction of data, but also doesn't yield data in the read statement in the case of pulsed voltage sweep:
myFieldFox.write("TRACE:DATA?")
ff_SA_Trace_Data = myFieldFox.read()
Also tried (based on tab autocompletion on iPython):
read_raw() # This is the one that works with non-pulsed sweep
and
read_bytes(nbytes)
The suggestion from #Paul-Cornelius is a good one, I had to include an *OPC? to get the previous data transfer to work as well. So right before I attempt the data transfer, I send these lines:
rep = keyS.query('NUB?')
keyS.query('*OPC?')
print(rep,'AAAAAAAAAAAAAAAAAAAAA') # this line prints!
mretholder = keyS.read_raw() # system hangs here!
In all the cases the end result is the same - I get a timeout error:
pyvisa.errors.VisaIOError: VI_ERROR_TMO (-1073807339): Timeout expired before operation completed.
The tracebacks for all of these show that they are all using the same basic framework from:
chunk, status = self.visalib.read(self.session, size)
Hoping someone has seen this before, or at least has some ideas on how to troubleshoot. Thanks in advance!
I don't have access to that instrument, but to read a waveform from a Tektronics oscilloscope I had to do the following (pyvisa module):
On obtaining the Resource, I do this:
resource.timeout = 10000
In some cases, I have to wait for a command to complete before proceeding like this:
resource.query("*opc?")
To transfer a waveform from the scope to the PC I do this:
ascii_waveform = resource.query("wavf?")
The "wavf?" query is specifically for this instrument, but the "*opc?" query is generic ("wait for operation complete"), and so is the method for setting the timeout parameter.
There is a lot of information in the user's guide (on the same site you have linked to) about reading data from various devices. I have used the visa library a few times on different devices, and it always requires some fiddling to get it to work.
It looks like you don't have any time delays in your program. In my experience they are almost always necessary, somehow, either by using the resource.timeout feature, the *opc? query, or both.
From time to time I suddenly have a need to connect to a device's console via its serial port. The problem is, I never remember what port settings (baud rate, data bits, stop bits, etc...) to use with each particular device, and documentation never seems to be lying around when it's really needed.
I wrote a Python script, which uses a simple brute-force method (i.e. iterates over all possible settings, sends some test input and displays the response for a human to decide if it makes sense ), but:
it takes a long time to complete
does not always work (perhaps port reset/timeout issues)
just does not seem like a proper way to do this :)
So the question is: does anyone know of a procedure to auto-detect what port settings the remote device is using?
Although part 1 is no direct answer to your question:
There are devices, which just have a autodetection (called Auto-bauding) method included, that means: Send a character using your current settings (9k6, 115k2, ..) to the device and chances are high that the device will answer with your (!) settings. I've seen this on HP switches.
Second approach: try to re-order the connection possibilities. E.g. chances are high that the other end uses 9k6 with no hardware handshake, but less that it uses 38k4 with software Xon/Xoff.
If you break down your tries into just a few, the "brute force" method will be much more efficient.
I have to filter and modify network traffic using Linux kernel libnetfilter_queue (precisely the python binding) and dpkt, and i'm trying to implement delayed packet forward.
Normal filtering works really well, but if i try to delay packets with function like this
def setVerdict(pkt, nf_payload):
nf_payload.set_verdict_modified(nfqueue.NF_ACCEPT, str(pkt), len(pkt))
t = threading.Timer(10, setVerdict, [pkt, nf_payload])
t.start()
It crashs throwing no exception (surely is a low level crash). Can i implement delay using directly libnetfilter like this or I must copy pkt, drop it and send the copy using standard socket.socket.send()?
Thank you
Sorry for the late reply, but I needed to do something like this, although slightly more complicated. I used the C-version of the library and I copied packets to a buffer inside my program, and then issued a DROP verdict. After a timeout relating to your delay, I reinject the packet using a raw socket. This works fine, and seems quite efficient.
I think the reason for your crash was due to the fact that you didnt issue a verdict fast enough.
I can't answer your question, but why not use the "netem" traffic-queue module on the outgoing interface to delay the packet?
It is possible to configure tc queues to apply different policies to packets which are "marked" in some way; the normal way to mark such packets is with a netfilter module (e.g. iptables or nfqueue).
I am looking to create a client/server application that I can use to slit network packets in half, tunnel each half of the packet over a separate udp connection (because each udp connection will be going over a different wifi link) and then reassemble the split packets on the other end. In addition to splitting the packets each half packet will also have to have an ID and sequence number so that they can be reassembled properly.
Basically I am trying to do something similar to MLPPP
I am looking to do this using python and the TUN/TAP network driver.
I have found the following python code samples and modules that I think might be helpful for this project.
Python tun/tap
http://www.secdev.org/projects/tuntap_udp/files/tunproxy.py
http://twistedmatrix.com/trac/browser/trunk/twisted/pair/tuntap.py
http://pastebin.com/gMB8zUfj
Python raw packet manipulation
http://libdnet.sourceforge.net/
http://pypi.python.org/pypi/pyip/
http://code.google.com/p/python-packet/
My question is can the necessary packet modification be done using python and what would be a possible way to approach this? Can I use the modules above to do this or is there a better solution? I am looking for some input that will steer me in the right direction as I am not an experienced programmer. Any code samples or additional links are welcome.
We are doing something like this in production and it works quite well. We don't split individual packets though. We set fractional weights for each connection (unlimited) and send the packets out. We have some code in place to deal with different latencies on each line. On the other end we buffer them and reorder. Performance is pretty good - we have sites with 5+ ADSL lines and get good speeds, 40+ Mbps on the download.
Splitting packets (eg 1500/2 = 750) would introduce unnecessary overhead... keep your packets as big as possible.
We have developed our own protocol (header format) for the UDP packets. We have done loopback testing on the tun/tap up to 200 Mbps, so definitely the kernel to user space interaction works well. Previously we used NFQUEUE but that had reliability issues.
All of the above was written in Python.
It looks perfectly possible to me.
The tun/tap modules you've discovered look like they would do the job. Twisted will be high performance and the cost of hurting your head working it all out.
As for splitting the packets, you don't need to interpret the data in any way, just treat it as a blob of binary data, split it in two and add a header - I wouldn't use any 3rd party modules for that, just normal python string handling.
Or you could use netstrings if you want an easy to use packet encapsulation format.
I don't suppose it would go like a rocket, but I'm sure you would learn lots doing it!
I have a simple problem which I'm sure someone here has done before...
I want to rewrite Layer 4 TCP/IP streams (Not lower layer individual packets or frames.) Ettercap's etterfilter command lets you perform simple live replacements of Layer 4 TCP/IP streams based on fixed strings or regexes. Example ettercap scripting code:
if (ip.proto == TCP && tcp.dst == 80) {
if (search(DATA.data, "gzip")) {
replace("gzip", " ");
msg("whited out gzip\n");
}
}
if (ip.proto == TCP && tcp.dst == 80) {
if (search(DATA.data, "deflate")) {
replace("deflate", " ");
msg("whited out deflate\n");
}
}
http://ettercap.sourceforge.net/forum/viewtopic.php?t=2833
I would like to rewrite streams based on my own filter program instead of just simple string replacements. Anyone have an idea of how to do this? Is there anything other than Ettercap that can do live replacement like this, maybe as a plugin to a VPN software or something?
I would like to have a configuration similar to ettercap's silent bridged sniffing configuration between two Ethernet interfaces. This way I can silently filter traffic coming from either direction with no NATing problems. Note that my filter is an application that acts as a pipe filter, similar to the design of unix command-line filters:
>[eth0] <----------> [my filter] <----------> [eth1]<
My filter will be a userspace Python function.
What I am already aware of, but are not suitable:
Tun/Tap - Works at the lower packet layer, I need to work with the higher layer streams.
Ettercap - I can't find any way to do replacements other than the restricted capabilities in the example above.
Hooking into some VPN software? - I just can't figure out which or exactly how.
libnetfilter_queue - Works with lower layer packets, not TCP/IP streams.
Again, the rewriting should occur at the transport layer (Layer 4) as it does in this example, instead of a lower layer packet-based approach. Exact code will help immensely!
Thanks!
Take a look on Scapy, or another packet crafting tool. There are not much of this type out there.
Ettercap is seemingly an open source project, since it is hosted on SourceForge. Perhaps you should look at how it does it.
At the time I was writing a network traffic analysis tool using libpcap for the capture and libnids for the stream assembly.
I haven't tried reinjecting traffic, but you can use TAP to forward traffic to a user program that will use libnids to assemble the packets, put out streams to a filter code, then take the stream and disassemble them (I'm pretty sure libnids has this capability as well) and reinject them into where you need it to go.
If you want python, pynids seems to do what you need, but I have no experience with it.
It looks like you would be able to write an ettercap plugin that loads filters written in python. Or write your custom filter in C instead.
Why not just access DATA.data directly with whatever filters you want?
I don't think you have to use search() you'll just have to build your own analyzer / state machine.
e.g.
for (int i = 0; i < DATA.len; i++) {
if ( DATA.data[i] == 'c' ) {
DATA.data[i] = q
}
}
Final Thoughts;
(1) I don't know that DATA.len exists but seems likely there's something.
(2) I know I replicated your search / replace but wanted to show how you may be able to do the equiv. 'manually'. You can get fancy w/ stuff later.
(3) If you were to adjust the length of the packet (or even the contents for that matter) you probably need to consider shifting window sizes / CRC checks, etc.
e.g. DATA.data[i] = "cc" would overwrite two characters (possibly not what you want) or would change the side of the packet.
I'd assume there are some library calls to set things right again.
PyPCAP is worth checking out since this all hinges around libpcap underneath to begin with. Cut out the middle man!
Since Ettercap is opensource you can change its source code to do what you want. Running your own C code to rewrite a TCP stream should be relatively easy. The hard work has already been done.
Take a look at the etterfilter man page to get started. Search in the source code for the filter engine (apparently it's a JIT interpreter).
I would also try to mail this question to the etterfilter authors, maybe they like stackoverflow :-)
Note: to use Python instead of C, see http://docs.python.org/release/2.5.2/ext/embedding.html