I am using pexpect in a python code to run a system command. On running the command, the user may or maynot be prompted with a question. if prompted he must answer y. I want this to happen automatically. I have written the following code -
child = pexpect.spawn( "module load xyz" )
child.expect( "Are you sure you want to clear all loaded modules.*" )
child.sendline( "y" )
My question is what will happen if the system does not prompt the user with the question and the child dies after successful execution of the command?
Thanks
You can wrap your expect statement in a while to continue looping and a try/except to handle the situation where the expected return value is not found. This will allow you to gracefully determine that you have hit the end of the process' output while, at the same time, acting upon the warning message if required.
child = pexpect.spawn( "module load xyz" )
while child.isalive():
try:
child.expect( ""Are you sure you want to clear all loaded modules.*" )
child.sendline( "y" )
except EOF:
pass
To do this, you will need to call from pexpect import EOF.
One more note, though. This will hang unless you either set your buffer to an appropriate size (something I've never gotten the hang of with pexpect) or the string you are expecting is followed by a newline. If neither of these is true, you will hang and have no idea why. In all honesty, I prefer to just do it the hard way and use subprocess.Popen, then read from stdout and stderr and write to stdin.
One more comment. Be careful about using wildcards. They tend to behave in odd ways. Given what you are looking for, you should be able to just drop the asterisk from your expected string.
To run the command and to answer 'y' if the question is asked using pexpect:
#!/usr/bin/env python
import os
import pexpect # $ pip install pexpect
pexpect.run("module load xyz", events={
"Are you sure you want to clear all loaded modules": "y" + os.linesep
})
If you want to use pexpect.spawn directly then the simplified version could look like:
#!/usr/bin/env python
import pexpect # $ pip install pexpect
child = pexpect.spawn("module load xyz")
while True:
i = child.expect(["Are you sure you want to clear all loaded modules",
pexpect.EOF, pexpect.TIMEOUT])
if i == 0:
child.sendline('y')
else: # child exited or the timeout happened
break
Related
This is a weird one that's so general I can't to properly narrow the search terms to find an answer.
My python script has a raw_input to prompt the user for values. But, when I try to run the script and funnel it into a file, it crashes.
Something like "script.py > save.txt"
wont work. It doesn't even properly prompt me at the command line for my input. There doesn't seem to be anything indicating why this doesn't work as intuitively as it should.
raw_output prints its prompt to stdout, which you are redirecting to a file. So your prompt will end up in the file and the program does not appear to show a prompt. One solution is to output your prompt to stderr.
import sys
sys.stderr.write('prompt> ')
value = raw_input()
print('value was: ', value)
You could also avoid using both pipes and interactive input with the same script. Either take input from command line flags using argparse and use pipes, or create an interactive program that saves output to a file itself.
Depending on your program's logic, you can also check whether stdout is connected to a live console or not:
is_tty = os.isatty(sys.stdout.fileno())
Dolda2000 also has a good point about writing to /dev/tty, which will write to the controlling terminal of the script being run even if both stdin and stderr are redirected. The deal there, though, is that you can't use it if you're not running in a terminal.
import errno
try:
with open('/dev/tty', 'w') as tty:
tty.write('prompt> ')
except IOError as exc:
if exc.errno == errno.ENXIO:
pass # no /dev/tty available
else:
pass # something else went wrong
I have a Python program, which, under certain conditions, should prompt the user for a filename. However, there is a default filename which I want to provide, which the user can edit if they wish. This means typically that they need to hit the backspace key to delete the current filename and replace it with the one they prefer.
To do this, I've adapted this answer for Python 3, into:
def rlinput(prompt, prefill=''):
readline.set_startup_hook(lambda: readline.insert_text(prefill))
try:
return input(prompt)
finally:
readline.set_startup_hook()
new_filename = rlinput("What filename do you want?", "foo.txt")
This works as expected when the program is run interactively as intended - after backspacing and entering a new filename, new_filename contains bar.txt or whatever filename the user enters.
However, I also want to test the program using unit tests. Generally, to do this, I run the program as a subprocess, so that I can feed it input to stdin (and hence test it as a user would use it). I have some unit testing code which (simplified) looks like this:
p = Popen(['mypythonutility', 'some', 'arguments'], stdin=PIPE)
p.communicate('\b\b\bbar.txt')
My intention is that this should simulate the user 'backspacing' over the provided foo.txt, and entering bar.txt instead.
However, this doesn't seem to have the desired effect. Instead, it would appear, after some debugging, that new_filename in my program ends up with the equivalent of \b\b\bbar.txt in it. I was expecting just bar.txt.
What am I doing wrong?
The appropriate way to control an interactive child process from Python is to use the pexpect module. This module makes the child process believe that it is running in an interactive terminal session, and lets the parent process determine exactly which keystrokes are sent to the child process.
Pexpect is a pure Python module for spawning child applications; controlling them; and responding to expected patterns in their output. Pexpect works like Don Libes’ Expect. Pexpect allows your script to spawn a child application and control it as if a human were typing commands.
I have a .jar file that I'm running with arguments via Popen. This server takes about 4 seconds to start up and then dumps out "Server Started" on the terminal and then runs until the user quits the terminal. However, the print and webbrowser.open execute immediately because of Popen and if I use call, they never run at all. Is there a way to ensure that the print and webbrowser don't run until after the server is started other than using wait? Maybe grep for server started?
from subprocess import Popen
import glob
import sys
import webbrowser
reasoner = glob.glob("reasoner*.jar")
reasoner = reasoner.pop()
port = str(input("Enter connection port: "))
space = ""
portArg = ("-p", port)
portArg = space.join(portArg)
print "Navigate to the Reasoner at http://locahost:" + port
reasoner_process = Popen(["java", "-jar", reasoner, "-i", "0.0.0.0", portArg, "--dbconnect", "jdbc:h2:tcp://localhost//tmp/UXDemo;user=sa;password=admin"])
# I want the following to execute after the .jar process above
print "Opening http://locahost:" + port + "..."
webbrowser.open("http://locahost:" + port)
What you're looking to do is a very simple, special version of interacting with a CLI app. So, you have two options.
First, you can use a library like pexpect that's designed to handle driving almost any CLI application. It may be overkill, and there is a bit of a learning curve, but once you get the basics down this will make your problem trivial: you launch the JAR, block expecting "Server Started", then close.
Alternatively, you can do this manually with the Popen pipes. In general this has a lot of problems, but when you know there's going to exactly one output that fits easily into 128 bytes and you don't want to do anything but block on that output and then close the pipe, none of those problems comes up. So:
reasoner_process = Popen(args, stdout=PIPE)
line = reasoner_process.stdout.readline()
if line.strip() != 'Server Started':
# error handling
# Any code that you want to do while the server is running goes here
reasoner_process.stdout.close()
reasoner_process.kill()
reasoner_process.wait()
But first make sure you actually have to kill it; often closing the pipe is sufficient, in which case you can and should leave out the kill(), in which case you can also check the exit code and raise if it's not 0.
Also, you probably want a with contextlib.closing(…) or whatever's appropriate, or just a try/finally to make sure you can raise an exception for error handling and not leak the child. (Python 3.2+ makes this a lot simpler, because it guarantees that both the pipes and the Popen itself are usable as context managers.)
Finally, I was assuming that "runs until the user quits the terminal" means you want to wait for it to start, then leave it running while you do other stuff, then kill it. If your workflow is different, you obviously need to change the order in which you do things.
I asked a question related to this several weeks ago on here:
Python, mpg123 and subprocess not properly using stdin.write or communicate
Thanks to help from there I was able to do what I needed at the time. (Didn't call q, but terminated the subprocess to stop it).## Heading ##
Now though I seem to be in another bit of a mess.
from subprocess import Popen, PIPE, STDOUT
p = Popen(["mpg123", "-C", "test.mp3"], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
#wait a few seconds to enter this, "q" without a newline is how the controls for the player work to quit out if it were ran like "mpg123 -C test.mp3" on the command line
p.communicate(input='q')[0]
much like before, I need this to be able to quit out of mpg123 like it would be with it's standard controls (like press 'q' to quit, or '-' to turn volume down, '+' to turn volume up, etc), now I use the code above, which should theoretically work, and it works with similar programs. Does anyone know of a way I can use the controls built into mpg123 (the one accessible by using "mpg123 -C whatever.mp3") using a subprocess? terminate isn't enough anymore as I will need the controls ^_^
EDIT: Many thanks to abarnert for the amazing answer =)
ok, so the new code is simply a slightly modified version of abarnert's answer, however mpg123 doesn't seem to be accepting the commands
import os
import pty
import sys
import time
pid, fd = os.forkpty()
if pid:
time.sleep(5)
os.write(fd, 'b') #this should've restarted the file
time.sleep(5)
os.write(fd, 'q') #unfortunately doesn't quit here =(
time.sleep(5) # quits after this is finished executing
else:
os.spawnl(os.P_WAIT, '/usr/bin/mpg123', '-C', 'TEST file.mp3')
If you really need the controls, you can't just use Popen.
mpg123 only enables terminal control if its stdin is a tty, not if it's a file or pipe. That's why you get this line in the banner:
Terminal control enabled, press 'h' for listing of keys and functions.
And the whole point of Popen (and subprocess, and the POSIX APIs it's built on) is pipes.
So, what can you do about it?
On linux, you can use the pty module. It may also work on other *nix platforms, but it may not—even if it gets built and included in your stdlib. As the docs say:
Because pseudo-terminal handling is highly platform dependent, there is code to do it only for Linux. (The Linux code is supposed to work on other platforms, but hasn’t been tested yet.)
It definitely runs on *BSD platforms on 2.7 and 3.3, and the example in the docs seem to work on both Mac OS X and FreeBSD… but that's as far as I've checked.
Meanwhile, most POSIX platforms will at least have os.forkpty, and that's not much harder, so here's a trivial program that plays the first 5 seconds of a song passed as its first arg:
import os
import pty
import sys
import time
pid, fd = os.forkpty()
if pid:
time.sleep(5)
os.write(fd, 'q')
else:
os.spawnl(os.P_WAIT, # mode
'/usr/local/bin/mpg123', # path
'/usr/local/bin/mpg123', '-C', sys.argv[1]) # args
Note that I used os.spawnl above. This is probably not what you want in a real program; it's for pedagogic purposes, to encourage you to read the docs (and the corresponding manpages) and understand this family of functions.
As the docs explain, this does not use the PATH environment variable, so you need to specify the full path to the program. You can just use spawnlp instead of spawnl to fix this.
Also, spawn may (in fact, always does, although the docs aren't entirely clear) do another fork to execute the child. This really isn't necessary, but spawn does things that you would need to do manually if you just called exec. If you know what you're doing, you may well want to use execl (or execlp) instead of spawnl.
You can even use most of the functionality in subprocess as long as you're careful (do not create any pipes, and remember that you'll end up doing two forks, so make sure to set up the parent/child relationship properly).
Also notice that you need to pass the path to mpg123 twice—once as the path, and then once as the child program's argv[0]. You could also just pass mpg123 the second time. Or, ideally, look at what ps says when you run it from the shell, and pass that. At any rate, you have to pass something as the argv[0]; otherwise, -C ends up being the argv[0], which means mpg123 won't think you gave it a -C flag to enable control keys, but rather than you renamed it to -C and ran it with no flags…
Anyway, you really do need to read the docs to understand what each of these functions does, instead of just treating it like magic code that you don't understand. So, I intentionally used the simplest possible solution to encourage that.
On Windows, there is no such thing as a pty, and no way to do this at all with the facilities built in to Python. You will need to use one of the various third-party libraries for controlling a cmd.exe console (aka DOS prompt) instead.
Based on abarnert's idea, we can open a pseudo-terminal and pass it to subprocess.
import os
import pty
import subprocess
import time
master, slave = os.openpty()
p = subprocess.Popen(['mpg123', '-C', 'music.mp3'], stdin=master)
time.sleep(3)
os.write(slave, 's')
time.sleep(3)
os.write(slave, 's')
time.sleep(6)
os.write(slave, 'q')
I have a small script that launches and, every half hour, feeds a command to a java program (game server manager) as if the user was typing it. However, after reading documentation and experimenting, I can't figure out how I can get two things:
1) A version which allows the user to type commands into the terminal windoe and they will be sent to the server manager input just as the "save-all" command is.
2) A version which remains running, but sends any new input to the system itself, removing the need for a second terminal window. This one is actually half-happening right now as when something is typed, there is no visual feedback, but once the program is ended, it's clear the terminal has received the input. For example, a list of directory contents will be there if "dir" was typed while the program was running. This one is more for understanding than practicality.
Thanks for the help. Here's the script:
from time import sleep
import sys,os
import subprocess
# Launches the server with specified parameters, waits however
# long is specified in saveInterval, then saves the map.
# Edit the value after "saveInterval =" to desired number of minutes.
# Default is 30
saveInterval = 30
# Start the server. Substitute the launch command with whatever you please.
p = subprocess.Popen('java -Xmx1024M -Xms1024M -jar minecraft_server.jar',
shell=False,
stdin=subprocess.PIPE);
while(True):
sleep(saveInterval*60)
# Comment out these two lines if you want the save to happen silently.
p.stdin.write("say Backing up map...\n")
p.stdin.flush()
# Stop all other saves to prevent corruption.
p.stdin.write("save-off\n")
p.stdin.flush()
sleep(1)
# Perform save
p.stdin.write("save-all\n")
p.stdin.flush()
sleep(10)
# Allow other saves again.
p.stdin.write("save-on\n")
p.stdin.flush()
Replace your sleep() with a call to select((sys.stdin, ), (), (), saveInterval*60) -- that will have the same timeout but listens on stdin for user commands. When select says you have input, read a line from sys.stdin and feed it to your process. When select indicates a timeout, perform the "save" command that you're doing now.
It won't completely solve your problem, but you might find python's cmd module useful. It's a way of easily implementing an extensible command line loop (often called a REPL).
You can run the program using screen, then you can send the input to the specific screen session instead of to the program directly (if you are in Windows just install cygwin).