I have a executable, called tsfoil2.exe, and I want to operate this .exe from my python environment.
I'm running Python 2.7.3, with Spyder 2.1.11 on Windows 7.
In order to operate the .exe, it requires some input, the default hard drive ('I:\'), a name for the outputfile ('test'), and a name for the input file ('SC20610.inp').
One of my colleagues advised me to use os.system, and supply this with a temporary input file, that contains all the arguments.
f = open('temp', 'w')
f.write('I:\ \n')
f.write('test \n')
f.write('SC20610.inp\n')
f.close()
I then supply this file with arguments to the .exe in the following way:
os.system("tsfoil2.exe < temp")
This all works, but the program requires a 'ENTER' to close. For some reason, the .exe is repeatedly asking to 'Press the ENTER key to exit'. Even, when I press the enter key in my Spyder-console, the program does not terminate.
Is there a way to give the 'ENTER' key as an interactive input to .exe?
I've tried to use the SendKeys class, but as the program does not terminate, it does not reach the line of code that contains the SendKeys command.
I've also tried to include it in the arguments-file, but this does not work either.
Furthermore I've also found out that it might be beneficial to switch to subprocesses command, as it might give me more command over the execution, but I haven't been able to run the executable with the input files.
Is it possible to provide the necessary 'ENTER' using os.system, or should I switch to subprocess, and if so, how can I construct a method similar to the os.system("tsfoil2.exe < temp") I'm using now.
I've tried this:
import subprocess as sub
f = open('temp', 'w')
f.write('I:\ \n')
f.write('test \n')
f.write('SC20610.inp\n')
f.close()
proc=sub.Popen(["tsfoil2.exe","temp"], shell=True)
and this
import subprocess as sub
p=sub.Popen('tsfoil2.exe')
p.communicate(input='I:' )
But, the program does not respond to the arguments given.
MWE:
import os
f = open('temp', 'w')
f.write('I:\ \n')
f.write('test \n')
f.write('SC20610.inp\n')
f.close()
os.system("tsfoil2.exe < temp")
Both the program can be found via http://www.dept.aoe.vt.edu/~mason/Mason_f/tsfoil2.exe, the input file can be found via http://www.dept.aoe.vt.edu/~mason/Mason_f/SC20610.inp.
I hope everything is clear, and you can help me out.
'Press the ENTER key to exit' means that the programs expects a newline.
I see no blank line at the end of the temp file. Also, you might have meant 'I:\\\n' -- you need to use '\\' in a Python string literal if you want \ in the output.
The question is what tsfoil2.exe considers a newline e.g., b'\r\n' or just b'\n' and where it expects to receive it: from stdin (getchar()) or directly from console (getch()).
Assuming that the program expects b'\r\n' from stdin on Windows:
import os
from subprocess import CalledProcessError, Popen, PIPE
cmd = "tsfoil2.exe"
input_data = os.linesep.join(['I:\\', 'test', 'SC20610.inp', os.linesep])
p = Popen(cmd, stdin=PIPE, bufsize=0)
p.communicate(input_data.encode('ascii'))
if p.returncode != 0:
raise CalledProcessError(p.returncode, cmd)
How it works
os.linesep == "\r\n" on Windows. "\n".join(["a", "b"]) == "a\nb".
Each process may have three standard streams: stdin, stdout, stderr. In Python, they are represented as sys.stdin, sys.stdout, sys.stderr. You can read input from stdin and write to stdout, stderr e.g., input function reads from stdin and print writes to stdout by default. stderr may be used to write error messages.
stdin=PIPE tells Popen to create a pipe between the parent process where it is called and the new child process ("tsfoil2.exe") and redirect the subprocess' stdin. p.communicate(data) writes data to p.stdin, closes it and waits for the child process to finish. p.returncode contains the exit status of the child process. Usually non-zero status means failure.
It emulates the shell pipeline without actually spawning the shell:
$ echo input data | tsfoil2.exe
If input is expected directly from console, you could try SendKeys module or its pure Python implementation SendKeys-ctypes:
from SendKeys import SendKeys
SendKeys(r"""
{LWIN}
{PAUSE .25}
r
C:\Path\To\tsfoil2.exe{ENTER}
{PAUSE 1}
I:\{ENTER}
{PAUSE 1}
test{ENTER}
{PAUSE 1}
SC20610.inp{ENTER}
{PAUSE 1}
{ENTER}
""")
I haven't tested it.
Related
I'm using subprocess to spawn a conda create command and capture the resulting stdout for later use. I also immediately print the stdout to the console so the user can still see the progress of the subprocess:
p = subprocess.Popen('conda create -n env1 python', stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
for line in iter(p.stdout.readline, b''):
sys.stdout.write(line.decode(sys.stdout.encoding))
This works fine until half way through the execution when the conda create requires user input: it prompts Proceed (n/y)? and waits for the user to input an option. However, the code above doesn't print the prompt and instead just waits for input "on a blank screen". Once an input is received the prompt is printed afterwards and then execution continues as expected.
I assume this is because the input somehow blocks the prompt being written to stdout and so the readline doesn't receive new data until after the input block has been lifted.
Is there a way I can ensure the input prompt is printed before the subprocess waits for user input? Note I'm running on Windows.
Although I'm sure pexpect would have worked in this case, I decided it would be overkill. Instead I used MisterMiyagi's insight and replaced readline with read.
The final code is as so:
p = subprocess.Popen('conda create -n env1 python', stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
while p.poll() is None:
sys.stdout.write(p.stdout.read(1).decode(sys.stdout.encoding))
sys.stdout.flush()
Note the read(1) as just using read() would block the while loop until an EOF is found. Given no EOF is found before the input prompt, nothing will be printed at all! In addition, flush is called which ensures the text written to sys.stdout is actually visible on screen.
For this use case I recommend using pexpect. stdin != stdout
Example use case where it conditionally sends to stdin on prompts on stdout
def expectgit(alog):
TMPLOG = "/tmp/pexpect.log"
cmd = f'''
ssh -T git#github.com ;\
echo "alldone" ;
'''
with open(TMPLOG, "w") as log:
ch = pexpect.spawn(f"/bin/bash -c \"{cmd}\"", encoding='utf-8', logfile=log)
while True:
i = ch.expect(["successfully authenticated", "Are you sure you want to continue connecting"])
if i == 0:
alog.info("Git all good")
break
elif i == 1:
alog.info("Fingerprint verification")
ch.send("yes\r")
ch.expect("alldone")
i = ch.expect([pexpect.EOF], timeout=5)
ch.close()
alog.info("expect done - EOF")
with open(TMPLOG) as log:
for l in log.readlines():
alog.debug(l.strip())
I am trying to execute a linux command through Python.
Here, for one of the command (cryptsetup luksChangeKey) I need to pass two keys [current key and new key] when prompted by the command through STDIN.
I tried using communicate() for the same and not able to pass both the keys.
Is there any other option available in python for the above scenario ?
Sample Code:
import subprocess
cmd = 'cryptsetup --batch-mode --key-file - luksChangeKey
/dev/multiplelv_pool_VG_13341/lv4'
process = subprocess.Popen(shlex.split(cmd), stdin=subprocess.PIPE,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
process.stdin.write("old123\nnew123\n")
process.communicate()
process.stdin.close()
If we manually execute the cryptsetup command, then it will prompt for old key and then the new key to be provided.
I am trying to simulate the same through the Python code.
Here current password is old123 and new password is new123.
My expectation of using '\n' in stdin.write was to split the password into two lines. However, POpen is taking the entire
line as current password and resulting in error
I am using Python 2.7
This work for me:
other.py
a = input()
b = input()
print(a, b)
program.py
import subprocess
cmd = r'python other.py'
process = subprocess.Popen(cmd, stdin=subprocess.PIPE)
process.communicate("input1\ninput2".encode('utf-8'))
process.stdin.close()
# print: >>> input1 input2
The subprocess documentation warns about the use of stdin.write.
Also, I don't know how your code doesn't throw an error, since you are passing a string to stdin and not a bytes-like object? This is what happens for me with your code:
TypeError: a bytes-like object is required, not 'str'
This is not a Python problem but a cryptsetup limitation. It is explicit in the manpage (emphasize mine):
--key-file, -d<br/>
use file as key material.
... If the key file is "-", stdin will be used. With the "-" key file reading will not stop when new line character is detected.
and later:
Notes on Password Processing
...
From stdin: Reading will continue until EOF
Workarounds:
According to this post on SuperUser, it used to be possible to pass both the old and new passwords in the same keyfile, but it would no longer be possible. A trick would be to use stdin for the old key, and stdout for the new one. Python will refuse to write on a process.stdout file because it is only opened for reading, but it is still possible to write on it at the fileno level. Code would become:
import subprocess
cmd = 'cryptsetup --batch-mode --key-file - luksChangeKey
/dev/multiplelv_pool_VG_13341/lv4 /dev/fd/1' # read the new key from /dev/fd/1
process = subprocess.Popen(shlex.split(cmd), stdin=subprocess.PIPE,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
os.write(process.stdout.fileno(), 'new123')
process.communicate('old123')
Beware: untested because I currently have no Linux system with LUKS.
A way to trick cryptsetup would be to use the pexpect-module (needs to be installed separately). This runs a subprocess faking a controlling terminal. It then allows to wait for prompts and issue user input when prompted. This is e.g. a way to control SSH and enter a password instead of having to resort to public keys etc.
Is there any way to use Popen with interactive commands? I mean nslookup, ftp, powershell... I read the whole subprocess documentation several times but I can't find the way.
What I have (removing the parts of the project which aren't of interest here) is:
from subprocess import call, PIPE, Popen
command = raw_input('>>> ')
command = command.split(' ')
process = Popen(command, stdout=PIPE, stderr=PIPE, shell=True)
execution = process.stdout.read()
error = process.stderr.read()
output = execution + error
process.stderr.close()
process.stdout.close()
print(output)
Basically, when I try to print the output with a command like dir, the output is a string, so I can work with the .read() on it. But when I try to use nslookup for example, the output isn't a string, so it can't be read, and the script enters in a deadlock.
I know that I can invoke nslookup in non-interactive mode, but that's not the point. I want to remove all the chances of a deadlock, and make it works with every command you can run in a normal cmd.
The real way the project works is through sockets, so the raw_input is a s.recv() and the output is sending back the output, but I have simplified it to focus on the problem.
I need to run a external exe file inside a python script. I need two things out of this.
Get whatever the exe outputs to the stdout (stderr).
exe stops executing only after I press the enter Key. I can't change this behavior. I need the script the pass the enter Key input after it gets the output from the previous step.
This is what I have done so far and I am not sure how to go after this.
import subprocess
first = subprocess.Popen(["myexe.exe"],shell=True,stdout=subprocess.PIPE)
from subprocess import Popen, PIPE, STDOUT
first = Popen(['myexe.exe'], stdout=PIPE, stderr=STDOUT, stdin=PIPE)
while first.poll() is None:
data = first.stdout.read()
if b'press enter to' in data:
first.stdin.write(b'\n')
first.stdin.close()
first.stdout.close()
This pipes stdin as well, do not forget to close your open file handles (stdin and stdout are also file handles in a sense).
Also avoid shell=True if at all possible, I use it a lot my self but best practices say you shouldn't.
I assumed python 3 here and stdin and stdout assumes bytes data as input and output.
first.poll() will poll for a exit code of your exe, if none is given it means it's still running.
Some other tips
one tedious thing to do can be to pass arguments to Popen, one neat thing to do is:
import shlex
Popen(shlex.split(cmd_str), shell=False)
It preserves space separated inputs with quotes around them, for instance python myscript.py debug "pass this parameter somewhere" would result in three parameters from sys.argv, ['myscript.py', 'debug', 'pass this parameter somewhere'] - might be useful in the future when working with Popen
Another thing that would be good is to check if there's output in stdout before reading from it, otherwise it might hang the application. To do this you could use select.
Or you could use pexpect which is often used with SSH since it lives in another user space than your application when it asks for input, you need to either fork your exe manually and read from that specific pid with os.read() or use pexpect.
I have this little script that puts your wireless device into monitor mode. It does an airodump scan and then after terminating the scan dumps the output to file.txt or a variable, so then I can scrape the BSSID and whatever other info I may need.
I feel I haven't grasped the concept or difference between subprocess.call() and subprocess.Popen().
This is what I currently have:
def setup_device():
try:
output = open("file.txt", "w")
put_device_down = subprocess.call(["ifconfig", "wlan0", "down"])
put_device_mon = subprocess.call(["iwconfig", "wlan0", "mode", "monitor"])
put_device_up = subprocess.call(["iwconfig", "wlano", "up"])
start_device = subprocess.call(["airmon-ng", "start", "wlan0"])
scanned_networks = subprocess.Popen(["airodump-ng", "wlan0"], stdout = output)
time.sleep(10)
scanned_networks.terminate()
except Exception, e:
print "Error:", e
I am still clueless about where and when and in which way to use subprocess.call() and subprocess.Popen()
The thing that I think is confusing me most is the stdout and stderr args. What is PIPE?
Another thing that I could possibly fix myself once I get a better grasp is this:
When running subprocess.Popen() and running airodump, the console window pops up showing the scan. Is there a way to hide this from the user to sort of clean things up?
You don't have to use Popen() if you don't want to. The other functions in the module, such as .call() use Popen(), give you a simpler API to do what you want.
All console applications have 3 'file' streams: stdin for input, and stdout and stderr for output. The application decides what to write where; usually error and diagnostic information to stderr, the rest to stdout. If you want to capture the output for either of these outputs in your Python program, you specify the subprocess.PIPE argument so that the 'stream' is redirected into your program. Hence the name.
If you want to capture the output of the airodump-ng wlan0 command, it's easiest to use the subprocess.check_output() function; it takes care of the PIPE argument for you:
scanned_networks = subprocess.check_output(["airodump-ng", "wlan0"])
Now output contains whatever airodump-ng wrote to its stdout stream.
If you need to have more control over the process, then you do need to use the Popen() class:
proc = subprocess.Popen(["airodump-ng", "wlan0"], stdout=subprocess.PIPE)
for line in proc.stdout:
# do something with line
proc.terminate()