I'm trying to make a program which takes an executable name as an argument, runs the executable and reports the inputs and outputs for that run. For example consider a child program named "circle". The following would be desired run for my program:
$ python3 capture_io.py ./circle
Enter radius of circle: 10
Area: 314.158997
[('output', 'Enter radius of circle: '), ('input', '10\n'), ('output', 'Area: 314.158997\n')]
I decided to use pexpect module for this job. It has a method called interact which lets the user interact with the child program as seen above. It also takes 2 optional parameters: output_filter and input_filter. From the documentation:
The output_filter will be passed all the output from the child process. The input_filter will be passed all the keyboard input from the user.
So this is the code I wrote:
capture_io.py
import sys
import pexpect
_stdios = []
def read(data):
_stdios.append(("output", data.decode("utf8")))
return data
def write(data):
_stdios.append(("input", data.decode("utf8")))
return data
def capture_io(argv):
_stdios.clear()
child = pexpect.spawn(argv)
child.interact(input_filter=write, output_filter=read)
child.wait()
return _stdios
if __name__ == '__main__':
stdios_of_child = capture_io(sys.argv[1:])
print(stdios_of_child)
circle.c
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char* argv[]) {
float radius, area;
printf("Enter radius of circle: ");
scanf("%f", &radius);
if (radius < 0) {
fprintf(stderr, "Negative radius values are not allowed.\n");
exit(1);
}
area = 3.14159 * radius * radius;
printf("Area: %f\n", area);
return 0;
}
Which produces the following output:
$ python3 capture_io.py ./circle
Enter radius of circle: 10
Area: 314.158997
[('output', 'Enter radius of circle: '), ('input', '1'), ('output', '1'), ('input', '0'), ('output', '0'), ('input', '\r'), ('output', '\r\n'), ('output', 'Area: 314.158997\r\n')]
As you can observe from the output, input is processed character by character and also echoed back as output which creates such a mess. Is it possible to change this behaviour so that my input_filter will run only when Enter is pressed?
Or more generally, what would be the best way to achieve my goal (with or without pexpect)?
When I started to write a helper, I realized that the main problem is that the input should be logged line buffered, so the backspace and other editing is done before the input reaches the program, but the output should be unbuffered in order to log the prompt which is not terminated by a new line.
To capture the output for the purpose of logging, a pipe is needed, but that automatically turns on line buffering. It is known that a pseudoterminal solves the problem (the expect module is built around a pseudoterminal), but a terminal has both the input and the output and we want to unbuffer only the output.
Fortunately there is the stdbuf utility. On Linux it alters the C library functions of dynamically linked executables. Not universally usable.
I have modified a Python bidirectional copy program to log the data it copies. Combined with the stdbuf it produces the desired output.
import select
import os
STDIN = 0
STDOUT = 1
BUFSIZE = 4096
def main(cmd):
ipipe_r, ipipe_w = os.pipe()
opipe_r, opipe_w = os.pipe()
if os.fork():
# parent
os.close(ipipe_r)
os.close(opipe_w)
fdlist_r = [STDIN, opipe_r]
while True:
ready_r, _, _ = select.select(fdlist_r, [], [])
if STDIN in ready_r:
# STDIN -> program
data = os.read(STDIN, BUFSIZE)
if data:
yield('in', data) # optional: convert to str
os.write(ipipe_w, data)
else:
# send EOF
fdlist_r.remove(STDIN)
os.close(ipipe_w)
if opipe_r in ready_r:
# program -> STDOUT
data = os.read(opipe_r, BUFSIZE)
if not data:
# got EOF
break
yield('out', data)
os.write(STDOUT, data)
os.wait()
else:
# child
os.close(ipipe_w)
os.close(opipe_r)
os.dup2(ipipe_r, STDIN)
os.dup2(opipe_w, STDOUT)
os.execlp(*cmd)
# not reached
os._exit(127)
if __name__ == '__main__':
log = list(main(['stdbuf', 'stdbuf', '-o0', './circle']))
print(log)
It prints:
[('out', b'Enter radius of circle: '), ('in', b'12\n'), ('out', b'Area: 452.388947\n')]
Is it possible to change this behaviour so that my input_filter will run only when Enter is pressed?
Yes, you can do it by inheriting from pexpect.spawn and overwriting the interact method. I will come to that soon.
As VPfB pointed out in their answer, you can't use a pipe and I think it's worth to mentioning that this issue is also addressed in the pexpect's documentation.
You said that:
... input is processed character by character and also echoed back as output ...
If you examine the source code of the interact you can see this line:
tty.setraw(self.STDIN_FILENO)
This will set your terminal to raw mode:
input is available character by character, ..., and all special processing of terminal input and output characters is disabled.
That is why your input_filter function is running for every key press and it sees backspace or other special characters. If you could comment out this line, you would see something like this when you run your program:
$ python3 test.py ./circle
Enter radius of circle: 10
10
Area: 314.158997
[('output', 'Enter radius of circle: '), ('input', '10\n'), ('output', '10\r\n'), ('output', 'Area: 314.158997\r\n')]
And this would also let you edit the input (i. e. 12[Backspace]0 would give you same result). But as you can see, it still echoes the input. This can be disabled by setting a simple flag for child's terminal:
mode = tty.tcgetattr(self.child_fd)
mode[3] &= ~termios.ECHO
tty.tcsetattr(self.child_fd, termios.TCSANOW, mode)
Running with the latest changes:
$ python3 test.py ./circle
Enter radius of circle: 10
Area: 314.158997
[('output', 'Enter radius of circle: '), ('input', '10\n'), ('output', 'Area: 314.158997\r\n')]
Bingo! Now you can inherit from pexpect.spawn and override interact method with these changes or implement the same thing using the builtin pty module of Python:
with pty:
import os
import pty
import sys
import termios
import tty
_stdios = []
def _read(fd):
data = os.read(fd, 1024)
_stdios.append(("output", data.decode("utf8")))
return data
def _stdin_read(fd):
data = os.read(fd, 1024)
_stdios.append(("input", data.decode("utf8")))
return data
def _spawn(argv):
pid, master_fd = pty.fork()
if pid == pty.CHILD:
os.execlp(argv[0], *argv)
mode = tty.tcgetattr(master_fd)
mode[3] &= ~termios.ECHO
tty.tcsetattr(master_fd, termios.TCSANOW, mode)
try:
pty._copy(master_fd, _read, _stdin_read)
except OSError:
pass
os.close(master_fd)
return os.waitpid(pid, 0)[1]
def capture_io_and_return_code(argv):
_stdios.clear()
return_code = _spawn(argv)
return _stdios, return_code >> 8
if __name__ == '__main__':
stdios, ret = capture_io_and_return_code(sys.argv[1:])
print(stdios)
with pexpect:
import sys
import termios
import tty
import pexpect
_stdios = []
def read(data):
_stdios.append(("output", data.decode("utf8")))
return data
def write(data):
_stdios.append(("input", data.decode("utf8")))
return data
class CustomSpawn(pexpect.spawn):
def interact(self, escape_character=chr(29),
input_filter=None, output_filter=None):
self.write_to_stdout(self.buffer)
self.stdout.flush()
self._buffer = self.buffer_type()
mode = tty.tcgetattr(self.child_fd)
mode[3] &= ~termios.ECHO
tty.tcsetattr(self.child_fd, termios.TCSANOW, mode)
if escape_character is not None and pexpect.PY3:
escape_character = escape_character.encode('latin-1')
self._spawn__interact_copy(escape_character, input_filter, output_filter)
def capture_io_and_return_code(argv):
_stdios.clear()
child = CustomSpawn(argv)
child.interact(input_filter=write, output_filter=read)
child.wait()
return _stdios, child.status >> 8
if __name__ == '__main__':
stdios, ret = capture_io_and_return_code(sys.argv[1:])
print(stdios)
I don't think you'll be able to do that easily, however, I think this should work for you:
output_buffer=''
def read(data):
output_buffer+=data
if data == '\r':
_stdios.append(("output", output_buffer.decode("utf8")))
output_buffer = ''
return data
Related
We have an old, legacy database that needs input from another system. SendInput method of data input into database forms is slow and unreliable, setting clipboard and then ^v is not reliable either (I have no idea why, but database interface is very old, early 2000s). After a lot of fiddling I discovered that using SendMessage to set text and then sending VK_RETURN is fast (much faster than SendInput/keybd_event) and reliable with our database. Now this code in plain C works:
HWND fghwnd = GetForegroundWindow();
DWORD threadId = GetWindowThreadProcessId(fghwnd, NULL);
DWORD myId = GetCurrentThreadId();
if (AttachThreadInput(myId, threadId, true)) {
HWND ctrl = GetFocus();
SendMessage(ctrl, WM_SETTEXT, 0, (LPARAM) sendbuf); // TESTING
PostMessage(ctrl, WM_KEYDOWN, VK_RETURN, 0);
PostMessage(ctrl, WM_KEYUP, VK_RETURN, 0);
AttachThreadInput(myId, threadId, false);
} else {
printf("\nError: AttachThreadInput failure!\n");
}
But this one in python does not:
foregroundHwnd = win32gui.GetForegroundWindow()
foregroundThreadID = win32process.GetWindowThreadProcessId(foregroundHwnd)[0]
ourThreadID = win32api.GetCurrentThreadId()
if foregroundThreadID != ourThreadID:
win32process.AttachThreadInput(foregroundThreadID, ourThreadID, True)
focus_whd = win32gui.GetFocus()
win32gui.SendMessage(focus_whd, win32con.WM_SETTEXT, None, "test text")
win32gui.PostMessage(focus_whd, win32con.WM_KEYDOWN, win32con.VK_RETURN, None)
win32gui.PostMessage(focus_whd, win32con.WM_KEYUP, win32con.VK_RETURN, None)
win32process.AttachThreadInput(foregroundThreadID, ourThreadID, False)
The trouble is, most of our new logic in python. I turned that C code into a small python module and it works, but as result now I've got dependency on Microsoft's huge compiler and a lot of fiddling with module building. I'd like to have a python-only solution.
Any ideas why this python code does not work? These system calls look the same...
Yes, AttachThreadInput failed. According to the comment here https://toster.ru/q/79336 win32process.GetWindowThreadProcessId returns wrong value, ctypes must be used. This code works:
"""
Fast "paste" implemented via calls to Windows internals, sends parameter
string and RETURN after that
Usage:
from paste import paste
paste("test")
"""
import time
import random
import string
from ctypes import windll
import ctypes
import win32con
def random_string(string_length=10):
"""Generate a random string of fixed length """
letters = string.ascii_lowercase
return ''.join(random.choice(letters) for i in range(string_length))
ERROR_INVALID_PARAMETER = 87
def paste(text_to_paste):
"""Fast "paste" using WM_SETTEXT method + Enter key"""
current_hwnd = windll.user32.GetForegroundWindow()
current_thread_id = windll.kernel32.GetCurrentThreadId()
thread_process_id = windll.user32.GetWindowThreadProcessId(current_hwnd, None)
if thread_process_id != current_thread_id:
res = windll.user32.AttachThreadInput(thread_process_id, current_thread_id, True)
# ERROR_INVALID_PARAMETER means that the two threads are already attached.
if res == 0 and ctypes.GetLastError() != ERROR_INVALID_PARAMETER:
print("WARN: could not attach thread input to thread {0} ({1})"
.format(thread_process_id, ctypes.GetLastError()))
return
focus_whd = windll.user32.GetFocus()
windll.user32.SendMessageW(focus_whd, win32con.WM_SETTEXT, None, text_to_paste)
windll.user32.PostMessageW(focus_whd, win32con.WM_KEYDOWN, win32con.VK_RETURN, None)
windll.user32.PostMessageW(focus_whd, win32con.WM_KEYUP, win32con.VK_RETURN, None)
res = windll.user32.AttachThreadInput(thread_process_id, current_thread_id, True)
if __name__ == '__main__':
time.sleep(5) # time to switch to the target
# paste random 150 char string
paste(random_string(150))
I am working on my own project. In which these steps have to be performed:
Connect to remote server.
Get pid, process name, cpu usage, swap memory usage by each running process on remote server daily on some specific time(say at 4'0 clock).
I have to compare every day's result with previous day's result (e.g. day1-pid with day2 pid and day1 process name with day2 process name etc.)
So far I have done up to step-2. Now I want to know that how to extract the pid, process name, cpu usage, swap memory usage from remote server and store it in some iterable variable. So that I can compare it for checking memory spike?
Any other way apart from my idea will be appreciable.
My code sample is like this:
import paramiko
import re
import psutil
class ShellHandler:
def __init__(self, host, user, psw):
self.ssh = paramiko.SSHClient()
self.ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
self.ssh.connect(host, username=user, password=psw, port=22)
channel = self.ssh.invoke_shell()
self.stdin = channel.makefile('wb')
self.stdout = channel.makefile('r')
def __del__(self):
self.ssh.close()
#staticmethod
def _print_exec_out(cmd, out_buf, err_buf, exit_status):
print('command executed: {}'.format(cmd))
print('STDOUT:')
for line in out_buf:
print(line, end="")
print('end of STDOUT')
print('STDERR:')
for line in err_buf:
print(line, end="")
print('end of STDERR')
print('finished with exit status: {}'.format(exit_status))
print('------------------------------------')
#print(psutil.pids())
pass
def execute(self, cmd):
"""
:param cmd: the command to be executed on the remote computer
:examples: execute('ls')
execute('finger')
execute('cd folder_name')
"""
cmd = cmd.strip('\n')
self.stdin.write(cmd + '\n')
finish = 'end of stdOUT buffer. finished with exit status'
echo_cmd = 'echo {} $?'.format(finish)
self.stdin.write(echo_cmd + '\n')
shin = self.stdin
self.stdin.flush()
shout = []
sherr = []
exit_status = 0
for line in self.stdout:
if str(line).startswith(cmd) or str(line).startswith(echo_cmd):
# up for now filled with shell junk from stdin
shout = []
elif str(line).startswith(finish):
# our finish command ends with the exit status
exit_status = int(str(line).rsplit(maxsplit=1)[1])
if exit_status:
# stderr is combined with stdout.
# thus, swap sherr with shout in a case of failure.
sherr = shout
shout = []
break
else:
# get rid of 'coloring and formatting' special characters
shout.append(re.compile(r'(\x9B|\x1B\[)[0-?]*[ -/]*[#-~]').sub('', line).replace('\b', '').replace('\r', ''))
# first and last lines of shout/sherr contain a prompt
if shout and echo_cmd in shout[-1]:
shout.pop()
if shout and cmd in shout[0]:
shout.pop(0)
if sherr and echo_cmd in sherr[-1]:
sherr.pop()
if sherr and cmd in sherr[0]:
sherr.pop(0)
self._print_exec_out(cmd=cmd, out_buf=shout, err_buf=sherr, exit_status=exit_status)
return shin, shout, sherr
obj=ShellHandler('Servername','username','password')
pID=[]
## I want this(pid, cmd, swap memory) to store in a varible which would be iterable.
pID=ShellHandler.execute(obj,"ps -eo pid,cmd,lstart,%mem,%cpu|awk '{print $1}'")
print(pID[0])##---------------------------------Problem not giving any output.
Your ShellHandler's execute method returns three items, the first of which is the input you sent to it.
You should probably call it directly like this, anyway:
obj = ShellHandler('Servername','username','password')
in, out, err = obj.execute("ps -eo pid,lstart,%mem,%cpu,cmd")
for line in out.split('\n'):
pid, lstartwd, lstartmo, lstartdd, lstartm, lstartyy, mem, cpu, cmd = line.split(None, 8)
I moved cmd last because it might contain spaces. The lstart value also contains multiple space-separated fields. Here's what the output looks like in Debian:
19626 Tue Jan 15 15:03:57 2019 0.0 0.0 less filename
There are many questions about how to parse ps output in more detail; I'll refer you to them for figuring out how to handle the results from split exactly.
Splitting out the output of ps using Python
Is there any way to get ps output programmatically?
ps aux command should have all the info you need (pid, process name, cpu, memory)
I am making an application in PySide and I want to add a Console/Terminal like screen, where you have a prompt and you can type commands. How would I be able to accomplish this. I am guessing some combination of QPlainTextEdit/QTextEdit for the output and QLineEdit for the actual prompt. Is there a better way to do this?
You could check out Spyder. They use PyQt (which is similar) and have a terminal. I think you can import their terminal widget, but I haven't played with it.
https://code.google.com/p/spyderlib/
Also, it is my favorite python editor by far!
I've spent a fair deal of time trying to find something like this, but to no avail. Good luck!
I did this with a custom QPlainTextEdit and custom QLineEdit. I also had an indicator label that would display ">>>" on the terminal to show user input. It needs more work. The best way would be to create your own custom widget based off of a QTextEdit and your own io handler. Below is an example of my execute method with self.input being the QLineEdit and self.view being the QTextEdit. It should get you the general idea.
import io, subprocess, shlex, code, rlcompleter, platform
def execute(self, currentText=None):
"""Execute runs the command given based on the console type.
If the console type is "both" then execute will run python commands
unless the user give the input ::os::command or ("::(platform.syste())::command").
Otherwise the console type will determine the what the input will execute with.
Args:
currentText(str): Text to execute. None will run the text from the QLineEdit self.input.
"""
# Check for given text
if currentText is None:
currentText = self.input.text()
self.input.clear()
self.view.display(currentText, "input")
else:
cmd = re.search("^>>>", currentText) # search for start only
if cmd is None:
currentText = ">>>" + currentText
else:
self.view.display(currentText, "input")
# end
# Variables
self.completer.clear()
cmdString = re.sub("^>>>", "", currentText)
result = None
displayType = "output"
run, cmdString = self.getRunType(cmdString)
try:
# Check where the output is going
sys.stdout = self.out = io.StringIO()
sys.stderr = sys.stdout
if run == "python": # Run python command
result = self._runInterpreter(cmdString)
displayType = "python"
elif run == "os": # Run os command
result = self._runSubprocess(cmdString)
displayType = "os"
except Exception as err:
result = str(err)
displayType = "Error"
notFoundPython = "NameError:" in result and "is not defined" in result
notFoundWindows = "returned non-zero exit status" in result
if notFoundPython or notFoundWindows:
result = "Command not found"
finally:
sys.stdout = self.old_stdout
sys.stderr = self.old_stdout
self.display(result, displayType)
# end execute
def getRunType(self, cmdString):
run = self._consoleType
# Check the run type
if self._consoleType == "both":
if re.search("^::python::", cmdString) is not None:
cmdString = re.sub("^::[a-z]*::", "", cmdString)
run = "python"
elif re.search("^(::os::|::"+platform.system()+"::)", cmdString) is not None:
cmdString = re.sub("^::[a-z]*::", "", cmdString)
run = "os"
else:
run = "python"
# end
return run, cmdString
# end getRunType
def _runInterpreter(self, cmdString, outIO=None, run=None):
# Check for a proper console type
if(self._consoleType != "both" and self._consoleType != "python"):
return
# Get the IO
if outIO is None:
outIO = sys.stdout
# Run python command
self.interpreter.push(cmdString)
# Check outIO
result = "Unreadable buffer: Check python's sys.stdout"
if isinstance(outIO, io.StringIO):
result = outIO.getvalue()
else:
if outIO.readable():
result = str(outIO.readlines())
# Check for error
if re.search("^Traceback", result) or re.search("[a-zA-z]*Error:", result):
raise ValueError(result)
return result
# end _runInterpreter
def _runSubprocess(self, cmdString, run=None):
# Check for a proper console type
if(self._consoleType != "both" and self._consoleType != "os"):
return
# Run OS command
cmd = shlex.split(cmdString)
result = subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT).decode("utf-8")
# Check for error
if re.search("^Traceback", result) or re.search("[a-zA-z]*Error:", result):
raise ValueError(result)
return result
# end _runSubprocess
I need to print some information directly (without user confirmation) and I'm using Python and the win32print module.
I've already read the whole Tim Golden win32print page (even read the win32print doc, which is small) and I'm using the same example he wrote there himself, but I just print nothing.
If I go to the interactive shell and make one step at a time, I get the document on the printer queue (after the StartDocPrinter), then I get the document size (after the StartPagePrinter, WritePrinter, EndPagePrinter block) and then the document disappear from the queue (after the EndDocPrinter) without printing.
I'm aware of the ShellExecute method Tim Golden showed. It works here, but it needs to create a temp file and it prints this filename, two things I don't want.
Any ideas? Thanks in advance.
This is the code I'm testing (copy and paste of Tim Golden's):
import os, sys
import win32print
import time
printer_name = win32print.GetDefaultPrinter()
if sys.version_info >= (3,):
raw_data = bytes ("This is a test", "utf-8")
else:
raw_data = "This is a test"
hPrinter = win32print.OpenPrinter (printer_name)
try:
hJob = win32print.StartDocPrinter (hPrinter, 1, ("test of raw data", None, "RAW"))
try:
win32print.StartPagePrinter (hPrinter)
win32print.WritePrinter (hPrinter, raw_data)
win32print.EndPagePrinter (hPrinter)
finally:
win32print.EndDocPrinter (hPrinter)
finally:
win32print.ClosePrinter (hPrinter)
[EDIT]
I installed a pdf printer in my computer to test with another printer (CutePDF Writer) and I could generate the test of raw data.pdf file, but when I look inside there is nothing. Meaning: all commands except WritePrinter appears to be doing what they were supposed to do. But again, as I said in the comments, WritePrinter return the correct amount of bytes that were supposed to be written to the printer. I have no other idea how to solve this, but just comproved there is nothing wrong with my printer.
I'm still looking for the best way to do this, but I found an answer that satisfy myself for the problem that I have. In Tim Golden's site (linked in question) you can find this example:
import win32ui
import win32print
import win32con
INCH = 1440
hDC = win32ui.CreateDC ()
hDC.CreatePrinterDC (win32print.GetDefaultPrinter ())
hDC.StartDoc ("Test doc")
hDC.StartPage ()
hDC.SetMapMode (win32con.MM_TWIPS)
hDC.DrawText ("TEST", (0, INCH * -1, INCH * 8, INCH * -2), win32con.DT_CENTER)
hDC.EndPage ()
hDC.EndDoc ()
I adapted it a little bit after reading a lot of the documentation. I'll be using win32ui library and TextOut (device context method object).
import win32ui
# X from the left margin, Y from top margin
# both in pixels
X=50; Y=50
multi_line_string = input_string.split()
hDC = win32ui.CreateDC ()
hDC.CreatePrinterDC (your_printer_name)
hDC.StartDoc (the_name_will_appear_on_printer_spool)
hDC.StartPage ()
for line in multi_line_string:
hDC.TextOut(X,Y,line)
Y += 100
hDC.EndPage ()
hDC.EndDoc ()
I searched in meta stackoverflow before answering my own question and here I found it is an encouraged behavior, therefore I'm doing it. I'll wait a little more to see if I get any other answer.
# U must install pywin32 and import modules:
import win32print, win32ui, win32con
# X from the left margin, Y from top margin
# both in pixels
X=50; Y=50
# Separate lines from Your string
# for example:input_string and create
# new string for example: multi_line_string
multi_line_string = input_string.splitlines()
hDC = win32ui.CreateDC ()
# Set default printer from Windows:
hDC.CreatePrinterDC (win32print.GetDefaultPrinter ())
hDC.StartDoc (the_name_will_appear_on_printer_spool)
hDC.StartPage ()
for line in multi_line_string:
hDC.TextOut(X,Y,line)
Y += 100
hDC.EndPage ()
hDC.EndDoc ()
#I like Python
The problem is the driver Version. If the Version is 4 you need to give XPS_PASS instead of RAW, here is a sample.
drivers = win32print.EnumPrinterDrivers(None, None, 2)
hPrinter = win32print.OpenPrinter(printer_name)
printer_info = win32print.GetPrinter(hPrinter, 2)
for driver in drivers:
if driver["Name"] == printer_info["pDriverName"]:
printer_driver = driver
raw_type = "XPS_PASS" if printer_driver["Version"] == 4 else "RAW"
try:
hJob = win32print.StartDocPrinter(hPrinter, 1, ("test of raw data", None, raw_type))
try:
win32print.StartPagePrinter(hPrinter)
win32print.WritePrinter(hPrinter, raw_data)
win32print.EndPagePrinter(hPrinter)
finally:
win32print.EndDocPrinter(hPrinter)
finally:
win32print.ClosePrinter(hPrinter)
I'm trying to transmit TCP/IP over a radio that is connected to my computer (specifically, the USRP). Right now, it's done very simply using Tun/Tap to set up a new network interface. Here's the code:
from gnuradio import gr, gru, modulation_utils
from gnuradio import usrp
from gnuradio import eng_notation
from gnuradio.eng_option import eng_option
from optparse import OptionParser
import random
import time
import struct
import sys
import os
# from current dir
from transmit_path import transmit_path
from receive_path import receive_path
import fusb_options
#print os.getpid()
#raw_input('Attach and press enter')
# Linux specific...
# TUNSETIFF ifr flags from <linux/tun_if.h>
IFF_TUN = 0x0001 # tunnel IP packets
IFF_TAP = 0x0002 # tunnel ethernet frames
IFF_NO_PI = 0x1000 # don't pass extra packet info
IFF_ONE_QUEUE = 0x2000 # beats me ;)
def open_tun_interface(tun_device_filename):
from fcntl import ioctl
mode = IFF_TAP | IFF_NO_PI
TUNSETIFF = 0x400454ca
tun = os.open(tun_device_filename, os.O_RDWR)
ifs = ioctl(tun, TUNSETIFF, struct.pack("16sH", "gr%d", mode))
ifname = ifs[:16].strip("\x00")
return (tun, ifname)
# /////////////////////////////////////////////////////////////////////////////
# the flow graph
# /////////////////////////////////////////////////////////////////////////////
class my_top_block(gr.top_block):
def __init__(self, mod_class, demod_class,
rx_callback, options):
gr.top_block.__init__(self)
self.txpath = transmit_path(mod_class, options)
self.rxpath = receive_path(demod_class, rx_callback, options)
self.connect(self.txpath);
self.connect(self.rxpath);
def send_pkt(self, payload='', eof=False):
return self.txpath.send_pkt(payload, eof)
def carrier_sensed(self):
"""
Return True if the receive path thinks there's carrier
"""
return self.rxpath.carrier_sensed()
# /////////////////////////////////////////////////////////////////////////////
# Carrier Sense MAC
# /////////////////////////////////////////////////////////////////////////////
class cs_mac(object):
"""
Prototype carrier sense MAC
Reads packets from the TUN/TAP interface, and sends them to the PHY.
Receives packets from the PHY via phy_rx_callback, and sends them
into the TUN/TAP interface.
Of course, we're not restricted to getting packets via TUN/TAP, this
is just an example.
"""
def __init__(self, tun_fd, verbose=False):
self.tun_fd = tun_fd # file descriptor for TUN/TAP interface
self.verbose = verbose
self.tb = None # top block (access to PHY)
def set_top_block(self, tb):
self.tb = tb
def phy_rx_callback(self, ok, payload):
"""
Invoked by thread associated with PHY to pass received packet up.
#param ok: bool indicating whether payload CRC was OK
#param payload: contents of the packet (string)
"""
if self.verbose:
print "Rx: ok = %r len(payload) = %4d" % (ok, len(payload))
if ok:
os.write(self.tun_fd, payload)
def main_loop(self):
"""
Main loop for MAC.
Only returns if we get an error reading from TUN.
FIXME: may want to check for EINTR and EAGAIN and reissue read
"""
min_delay = 0.001 # seconds
while 1:
payload = os.read(self.tun_fd, 10*1024)
if not payload:
self.tb.send_pkt(eof=True)
break
if self.verbose:
print "Tx: len(payload) = %4d" % (len(payload),)
delay = min_delay
while self.tb.carrier_sensed():
sys.stderr.write('B')
time.sleep(delay)
if delay < 0.050:
delay = delay * 2 # exponential back-off
self.tb.send_pkt(payload)
# /////////////////////////////////////////////////////////////////////////////
# main
# /////////////////////////////////////////////////////////////////////////////
def main():
mods = modulation_utils.type_1_mods()
demods = modulation_utils.type_1_demods()
parser = OptionParser (option_class=eng_option, conflict_handler="resolve")
expert_grp = parser.add_option_group("Expert")
parser.add_option("-m", "--modulation", type="choice", choices=mods.keys(),
default='gmsk',
help="Select modulation from: %s [default=%%default]"
% (', '.join(mods.keys()),))
parser.add_option("-v","--verbose", action="store_true", default=False)
expert_grp.add_option("-c", "--carrier-threshold", type="eng_float", default=30,
help="set carrier detect threshold (dB) [default=%default]")
expert_grp.add_option("","--tun-device-filename", default="/dev/net/tun",
help="path to tun device file [default=%default]")
transmit_path.add_options(parser, expert_grp)
receive_path.add_options(parser, expert_grp)
for mod in mods.values():
mod.add_options(expert_grp)
for demod in demods.values():
demod.add_options(expert_grp)
fusb_options.add_options(expert_grp)
(options, args) = parser.parse_args ()
if len(args) != 0:
parser.print_help(sys.stderr)
sys.exit(1)
if options.rx_freq is None or options.tx_freq is None:
sys.stderr.write("You must specify -f FREQ or --freq FREQ\n")
parser.print_help(sys.stderr)
sys.exit(1)
# open the TUN/TAP interface
(tun_fd, tun_ifname) = open_tun_interface(options.tun_device_filename)
# Attempt to enable realtime scheduling
r = gr.enable_realtime_scheduling()
if r == gr.RT_OK:
realtime = True
else:
realtime = False
print "Note: failed to enable realtime scheduling"
# If the user hasn't set the fusb_* parameters on the command line,
# pick some values that will reduce latency.
if options.fusb_block_size == 0 and options.fusb_nblocks == 0:
if realtime: # be more aggressive
options.fusb_block_size = gr.prefs().get_long('fusb', 'rt_block_size', 1024)
options.fusb_nblocks = gr.prefs().get_long('fusb', 'rt_nblocks', 16)
else:
options.fusb_block_size = gr.prefs().get_long('fusb', 'block_size', 4096)
options.fusb_nblocks = gr.prefs().get_long('fusb', 'nblocks', 16)
#print "fusb_block_size =", options.fusb_block_size
#print "fusb_nblocks =", options.fusb_nblocks
# instantiate the MAC
mac = cs_mac(tun_fd, verbose=True)
# build the graph (PHY)
tb = my_top_block(mods[options.modulation],
demods[options.modulation],
mac.phy_rx_callback,
options)
mac.set_top_block(tb) # give the MAC a handle for the PHY
if tb.txpath.bitrate() != tb.rxpath.bitrate():
print "WARNING: Transmit bitrate = %sb/sec, Receive bitrate = %sb/sec" % (
eng_notation.num_to_str(tb.txpath.bitrate()),
eng_notation.num_to_str(tb.rxpath.bitrate()))
print "modulation: %s" % (options.modulation,)
print "freq: %s" % (eng_notation.num_to_str(options.tx_freq))
print "bitrate: %sb/sec" % (eng_notation.num_to_str(tb.txpath.bitrate()),)
print "samples/symbol: %3d" % (tb.txpath.samples_per_symbol(),)
#print "interp: %3d" % (tb.txpath.interp(),)
#print "decim: %3d" % (tb.rxpath.decim(),)
tb.rxpath.set_carrier_threshold(options.carrier_threshold)
print "Carrier sense threshold:", options.carrier_threshold, "dB"
print
print "Allocated virtual ethernet interface: %s" % (tun_ifname,)
print "You must now use ifconfig to set its IP address. E.g.,"
print
print " $ sudo ifconfig %s 192.168.200.1" % (tun_ifname,)
print
print "Be sure to use a different address in the same subnet for each machine."
print
tb.start() # Start executing the flow graph (runs in separate threads)
mac.main_loop() # don't expect this to return...
tb.stop() # but if it does, tell flow graph to stop.
tb.wait() # wait for it to finish
if __name__ == '__main__':
try:
main()
except KeyboardInterrupt:
pass
(Anyone familiar with GNU Radio will recognize this as tunnel.py)
My question is, is there a better way to move packets to and from the kernel than tun/tap? I've been looking at ipip or maybe using sockets, but I'm pretty sure those won't be very fast. Speed is what I'm most concerned with.
Remember that tunnel.py is a really, really rough example, and hasn't been updated in a while. It's not really meant to be a basis for other code, so be careful of how much you rely on the code.
Also, remember that TCP over unreliable radio links has significant issues:
http://en.wikipedia.org/wiki/Transmission_Control_Protocol#TCP_over_wireless_networks