Is there a way to programmatically interrupt Python's raw_input? Specifically, I would like to present a prompt to the user, but also listen on a socket descriptor (using select, for instance) and interrupt the prompt, output something, and redisplay the prompt if data comes in on the socket.
The reason for using raw_input rather than simply doing select on sys.stdin is that I would like to use the readline module to provide line editing functionality for the prompt.
As far as I know... "Sort of".
raw_input is blocking so the only way I can think of is spawning a subprocess/thread to retrieve the input, and then simply communicate with the thread/subprocess. It's a pretty dirty hack (at least it seems that way to me), but it should work cross platform. The other alternative, of course, is to use either the curses module on linux or get this one for windows.
Related
I am using a 3rd-party python module which is normally called through terminal commands. When called through terminal commands it has a verbose option which prints to terminal in real time.
I then have another python program which calls the 3rd-party program through subprocess. Unfortunately, when called through subprocess the terminal output no longer flushes, and is only returned on completion (the process takes many hours so I would like real-time progress).
I can see the source code of the 3rd-party module and it does not set printing to be flushed such as print('example', flush=True). Is there a way to force the flushing through my module without editing the 3rd-party source code? Furthermore, can I send this output to a log file (again in real time)?
Thanks for any help.
The issue is most likely that many programs work differently if run interactively in a terminal or as part of a pipe line (i.e. called using subprocess). It has very little to do with Python itself, but more with the Unix/Linux architecture.
As you have noted, it is possible to force a program to flush stdout even when run in a pipe line, but it requires changes to the source code, by manually applying stdout.flush calls.
Another way to print to screen, is to "trick" the program to think it is working with an interactive terminal, using a so called pseudo-terminal. There is a supporting module for this in the Python standard library, namely pty. Using, that, you will not explicitly call subprocess.run (or Popen or ...). Instead you have to use the pty.spawn call:
def prout(fd):
data = os.read(fd, 1024)
while(data):
print(data.decode(), end="")
data = os.read(fd, 1024)
pty.spawn("./callee.py", prout)
As can be seen, this requires a special function for handling stdout. Here above, I just print it to the terminal, but of course it is possible to do other thing with the text as well (such as log or parse...)
Another way to trick the program, is to use an external program, called unbuffer. Unbuffer will take your script as input, and make the program think (as for the pty call) that is called from a terminal. This is arguably simpler if unbuffer is installed or you are allowed to install it on your system (it is part of the expect package). All you have to do then, is to change your subprocess call as
p=subprocess.Popen(["unbuffer", "./callee.py"], stdout=subprocess.PIPE)
and then of course handle the output as usual, e.g. with some code like
for line in p.stdout:
print(line.decode(), end="")
print(p.communicate()[0].decode(), end="")
or similar. But this last part I think you have already covered, as you seem to be doing something with the output.
I am trying to make a program in c++, but i cant make the program because in one part of the code I need to run a python program from c++ and I dont know how to do it. I've been trying many ways of doing it but none of them worked. So the code should look sometihnglike this:somethingtoruntheprogram("pytestx.py"); or something close to that. Id prefer doing it without python.h. I just need to execute this program, I need to run the program because I have redirected output and input from the python program with sys.stdout and sys.stdin to text files and then I need to take data from those text files and compare them. I am using windows.
You have two way of doing that:
Use system/fork and exec*/...
Embed a python interpreter in your program (cf python 2.6 doc or boost.python)
Using a embedded interpreter is (IMHO) the best way to do it because it gives you more control over the execution of the script, because it's not OS-dependant and it does not rely on your target having a python interpreter (configured as you require).
There's POSIX popen and on Windows _popen, which is halfway between exec and system. It offers the required control over stdin and stdout, which system does not. But on the other hand, it's not as complicated as the exec family of functions.
I'm trying to automate mounting and unmounting of UNC drives with Python on Windows. I'm using the subprocess module to the execute the various commands so I can log their output. However, the NET USE command occasionally prompts the user for input (like a password). Usually things work well, but since I am using subprocess.communicate(), my program hangs indefinitely when NET USE asks my "subprocess" for input. Of course, I have no idea it's asking for input. It just sits there waiting patiently and making me frustrated.
This is a more general problem than just NET USE, so please don't go that route. There are other instances where some (other) program asks for input and my program freezes. Any thoughts?
Some programs are smart and realize that if there is no way for a user to input anything, then they shouldn't even ask. For such programs, you can add "stdin=open('/dev/null')" to the subprocess.Popen options.
In the more difficult case, where the program really insists on interacting, all you can do is provide the interaction yourself.
EDIT: I should mention that the pexpect library is a great way to do such interaction.
This question already has answers here:
read subprocess stdout line by line
(10 answers)
Closed 21 days ago.
How can I receive input from the terminal in Python?
I am using Python to interface with another program which generates output from user input.
I am using subprocess.Popen() to input to the program, but I can't set stdout to subprocess.PIPE because the program does not seem to flush ever, so everything gets stuck in the buffer.
The program's standard output seems to be to print to terminal, and I see output when I do not redirect stdout. However, I need Python to read and interpret the output which is now in the terminal.
Sorry if this is a stupid question, but I can't seem to get this to work.
Buffering in child processes is a common problem. Here are four possible approaches.
First, and easiest, you could read one byte at a time from your pipe. This is what I would call a "dirty hack" and it carries a performance penalty, but it's easy and it guarantees that your read() calls will only block until the first byte comes in, rather than wait for a buffer to fill up that's never going to fill up. However, this does not force the other process to flush its write buffer, so if that is the issue this approach will not help you anyway.
Second, and I think next-easiest, consider using the Twisted framework which has a facility for using a virtual terminal, or pty ("pseudo-teletype" I think) to talk to your child process. However, this can affect the design of your application (possibly for the better, but this may not be in the cards for you regardless). http://twistedmatrix.com/documents/current/core/howto/process.html
If neither of the above options works for you, you're reduced to solving gritty I/O concurrency issues yourself.
Third, try setting your pipes (all of them, before fork()) to non-blocking mode using fcntl() with O_NONBLOCK. Then you can use select() to test for read/write readiness before trying the read/write; but you still have to catch IOError and test for EAGAIN because it can happen even in this case. This may, depending on the behavior of the child process, allow you to wait until the data really shows up before trying to read it in.
The last resort is to implement the PTY logic yourself. If you've seen references to stuff like termio options, ioctl() calls, etc. then that's what you're up against. I have not done this before, because it's complicated and I have never really needed to. If this is your destiny, good luck.
Have you tried setting the bufsize in your Popen object to 0? I'm not sure if you can force the buffer to be unbuffered from the receiving size, but I'd try it.
http://docs.python.org/library/subprocess.html#using-the-subprocess-module
I'm writing a curses application in Python under UNIX. I want to enable the user to use C-Y to yank from a kill ring a la Emacs.
The trouble is, of course, that C-Y is caught by my shell which then sends SIGTSTP to my process. In addition, C-Z also results in SIGTSTP being sent, so catching the signal means that C-Y and C-Z are not distinguishable (though even without this the only solutions I can think of are extremely hackish).
I know what I'm asking is possible (in C if not in Python), since Emacs does it. How can I disable the shell's special handling of certain control characters sent from the keyboard and have the characters in question appear on the process' stdin?
See the termios module, and the termios(3) man page.
For basic functionality, use tty. For example, calling tty.setraw(sys.stdin) will put standard input's terminal into raw mode.
For the more general case, Python comes with a termios library, but you probably need some experience with termios to know how to use it.
Alternatively, a cheap way is to shell out to stty, which is a command-line interface to termios.