I'm playing around a little with Python and curses.
When I run
import time
import curses
def main():
curses.initscr()
curses.cbreak()
for i in range(3):
time.sleep(1)
curses.flash()
pass
print( "Hello World" )
curses.endwin()
if __name__ == '__main__':
main()
if I wait all the way through, curses.endwin() gets called so everything works out fine.
However, if I cut it short with Ctrl-C, curses.endwin() never gets called so it screws up my terminal session.
What is the proper way to handle this situation? How can I make sure that no matter how I try to end/interrupt the program (e.g. Ctrl-C, Ctrl-Z), it doesn't mess up the terminal?
I believe you are looking for curses.wrapper
See http://docs.python.org/dev/library/curses.html#curses.wrapper
It will do curses.cbreak(), curses.noecho() and curses_screen.keypad(1) on init and reverse them on exit, even if the exit was an exception.
Your program goes as a function to the wrapper, example:
def main(screen):
"""screen is a curses screen passed from the wrapper"""
...
if __name__ == '__main__':
curses.wrapper(main)
You could do this:
def main():
curses.initscr()
try:
curses.cbreak()
for i in range(3):
time.sleep(1)
curses.flash()
pass
print( "Hello World" )
finally:
curses.endwin()
Or more nicely, make a context wrapper:
class CursesWindow(object):
def __enter__(self):
curses.initscr()
def __exit__(self):
curses.endwin()
def main():
with CursesWindow():
curses.cbreak()
for i in range(3):
time.sleep(1)
curses.flash()
pass
print( "Hello World" )
My advice: For testing purposes, call your script using a simple wrapper shell script; have the shell script perform a reset command to bring your terminal settings back into a usable state:
#!/bin/sh
eval "$#"
stty -sane
reset
... call that as run.sh and be happy. This should run your command almost exactly as your shell would if you entered the arguments as a command (more exactly if you wrap the arguments in hard quotes).
To ensure that your program will leave the terminal in a robust state, in the the face of uncaught exceptions and abnormal terminations ... either use the curses.wrapper() method to call your top level entry point (probably main() or whatever main_curses_ui() you choose to implement) or wrap your code in your own sequence of curses.* methods to restore cursor visibility, restore "cbreak" (canonical/cooked input) mode, restore the normal "echo" settings and whatever else you may have mucked with.
You can also use the Python: atexit Handlers to register all your clean-up actions. But there might still be cases where your code doesn't get called --- some sorts of non-catchable signals and any situation where os._exit() is invoked.
My little shell script wrapper should be fairly robust even in those cases.
You can:
wrap your code in a try/finally block that calls curses.endwin()
capture the interrupt signal specifically via the signal library
use the atexit library.
The first option is probably the simplest for a basic case (if you're not running much code).
The second option is the most specific, if you want to do something special for Ctrl+C.
The last option is the most robust if you always want to do certain shutdown actions, no matter how your program is ending.
You need to capture the signal and run endwin() during the capture.
For info on this, look at this SO answer: How do I capture SIGINT in Python?
Related
I'm using Python 2.7 with Glade 3.15 to create a GUI that allows click-button execution of a variety of existing bash/cshell scripts maintained by my work team. I'm fairly new to Python, but have managed to get the basic application structure up-and-running. However, certain bash scripts I'm calling will step through multiple user prompts and take input to determine end behavior. The problem I am encountering is when I call a bash script as a python subprocess, the bash script appears to take a null input over-and-over, thus causing the prompts to loop endlessly.
For example:
A bash script that prompts:
"Please enter your 4 digit document number:"
** accept user input in terminal **
"You entered ----, is that correct?
1.) Yes
2.) No "
When called from python, the terminal will press through the the prompts, sending an empty response. Since the bash script loops until affirmative response is received, the result is a terminal endlessly printing:
"You entered ----, is that correct?
1.) Yes
2.) No "
I've tried extensively to find answers, here and elsewhere, regarding this issue, but have not found/developed a solution yet.
My basic python, relative to this problem, is as follows (although I have tried a wide variety of different approaches)
import subprocess
from subprocess import Popen,PIPE
...
# Definition for subprocess calls
def subprocess_cmd(self, command):
process = subprocess.Popen(command, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
process.wait()
(output, err) = process.communicate()
print output
...
# Script-Call Button
def on_btnScript_clicked(self, object, data=None):
self.subprocess_cmd("scriptname_is_here")
I just want to call a subprocess from my python button_click event that kicks-off the bash script in the terminal, and waits for keyboard terminal input to walk-through the prompts, as it would if it were run directly from terminal. Sorry so long - wanted to be thorough and explicit. Thanks in advance for any help.
*****UPDATE****
If I call the subprocess from another standalone python file with the .wait() method, the interaction works as desired. But, when I call the subprocess as a result of the GUI button_click event, with the same arguments and methods, the looping anomaly happens. I think this has to do with my button click event and subprocess_cmd 'function' being defined in my mainDialog class, but I don't know how to separate them while retaining my connection to GUI.
Here is more context for my code
#!/usr/bin/python
# Library Imports
from gi.repository import Gtk
from os import system
import subprocess
from subprocess import Popen,PIPE
import time
try:
import math
except:
print "Math Library Missing"
sys.exit(1)
class mainDialog:
# Build the 'form load' parameters
def __init__(self):
self.gladefile = "test.glade"
self.builder = Gtk.Builder()
self.builder.add_from_file(self.gladefile)
self.builder.connect_signals(self)
self.winMain = self.builder.get_object("winMain")
self.winCptArg = self.builder.get_object("winCptArg")
self.winMsbHelp = self.builder.get_object("winMsbHelp")
self.winCptHelp = self.builder.get_object("winCptHelp")
self.winAiHelp = self.builder.get_object("winAiHelp")
self.winMain.move(2625, 400)
self.winMain.show()
# Definition for subprocess calls
def subprocess_cmd(self, command):
process = subprocess.Popen(command)
process.wait()
...
# Script-Call Button
def on_btnScript_clicked(self, object, data=None):
self.subprocess_cmd("scriptname_is_here")
if __name__ == "__main__":
main = mainDialog()
Gtk.main()
Just use os.system:
from os import system
...
# Definition for subprocess calls
def subprocess_cmd(self, command):
process = system(str(command))
...
# Script-Call Button
def on_btnScript_clicked(self, object, data=None):
self.os.system("echo scriptname_is_here")
The syntax is os.system("executable option parameter").
For example,
os.system("ls -al /home")
Well, if anyone is interested, to achieve what I was intending, I simply left stdin and stdout alone, and applied the .wait() method to the subprocess definition -- but this only works when called from a standalone python script; I haven't been able to retain functionality when connected to GUI button click event.
def subprocess_cmd(self, command):
process = subprocess.Popen(command).wait()
...
def on_btnScript_clicked(self, object, data=None):
self.subprocess_cmd("filepath/scriptname_is_here")
stdin and stdout can be left as default, and standard terminal interaction can be achieved, so long as the subprocess definition is appended with the wait() method.
I have created a watchdog timer for my script (Python 3), which allows me to halt execution if anything goes wrong (not shown in code below). However, I would like to have the ability to restart the script automatically using only Python (no external scripts). The code needs to be cross platform compatible.
I have tried subprocess and execv (os.execv(sys.executable, ['python'] + sys.argv)), however I am seeing very weird functionality on Windows. I open the command line, and run the script ("python myscript.py"). The script stops but does not exit (verified through Task Manager), and it will not restart itself unless I press enter twice. I would like it to work automatically.
Any suggestions? Thanks for your help!
import threading
import time
import subprocess
import os
import sys
if __name__ == '__main__':
print("Starting thread list: " + str(threading.enumerate()))
for _ in range(3):
time.sleep(1)
print("Sleeping")
''' Attempt 1 with subprocess.Popen '''
# child = subprocess.Popen(['python',__file__], shell=True)
''' Attempt 2 with os.execv '''
args = sys.argv[:]
args.insert(0, sys.executable)
if sys.platform == 'win32':
args = ['"%s"' % arg for arg in args]
os.execv(sys.executable, args)
sys.exit()
Sounds like you are using threading in your original script, which explains why your can't break your original script when simply pressing Ctrl+C. In that case, you might want to add a KeyboardInterrupt exception to your script, like this:
from time import sleep
def interrupt_this()
try:
while True:
sleep(0.02)
except KeyboardInterrupt as ex:
# handle all exit procedures and data cleaning
print("[*] Handling all exit procedures...")
After this, you should be able to automatically restart your relevant procedure (even from within the script itself, without any external scripts). Anyway, it's a bit hard to know without seeing the relevant script, so maybe I can be of more help if you share some of it.
So I have this part of code which does a simple thing : it launches a script and while the script is processing, a throbber is set on.
def go(self):
if ui.chk.isChecked():
self.startThrobber()
script = subprocess.check_call(r'"C:\Program Files\FME\fme.exe"', shell=False)
if script == 0:
self.stopThrobber() # opens a QMessageBox and stops throbber
else:
QMessageBox.information(self.popup(), "Errpr", "Error !")
After trying different methods (QThread, subprocess.Popen ...) this is the closest i got to make it work.
The only thing that doesn't work is that the throbber doesn't start right before the subprocess is executed, it starts after and thus it never stops.
So why is the throbber not ending when stopThrobber() is executed ?
And why is startThrobber not being executed before the subprocess (i'm pretty sure it's a subprocess thing, but i'm pretty new to all this, never heard about thread until yesterday)
EDIT :
The single quote was just a typing error, sorry. Still doesn't fix the problem.
Any call to a subprocess, from your main thread, that is blocking (waits for return) is going to stop your throbber from working properly. My answer to your other SO question on this topic outlines an approach that does not cause the subprocess call to block the main thread. I should point out that solution is not the only way to create a non-blocking call to a subprocess (for instance see here. You could create a QTimer to poll the subprocess poll() method periodically so that you can check the returncode to see if the subprocess has finished.)
The key theme is that you need your methods that run in the main thread to return quickly in order to keep the GUI responsive and allow your throbber to run/animate. So choose a way to launch the subprocess that meets this requirement.
Your single quotes denoting the raw string enclose the 'shell' argument.
def go(self):
if ui.chk.isChecked():
self.startThrobber()
script = subprocess.check_call(r"C:\Program Files\FME\fme.exe", shell=False)
if script == 0:
self.stopThrobber() # opens a QMessageBox and stops throbber
else:
QMessageBox.information(self.popup(), "Errpr", "Error !")
So I've tried another thing (unsuccessfully..)
When I click on a button, it executes startThrobber() and sends a signal to the following function :
def go(self):
self.startThrobber()
script = subprocess.Popen(r'"C:\Program Files\FME\fme.exe" ', shell=False)
while script.poll() == None:
time.sleep(1)
else:
p.stopThrobber()
But still doesn't working.. startThrobber is executed but nothing appears on the GUI... I thougth that the point of subprocess was to allow to do multiple tasks simultaneously so why isn't the throbber appearing ?
UPDATE : if i erase the while loop, startThrobber works : it appears while the subprocess is turning. So why when there is the while loop it doesn't work ?!
I'm using a commercial application that uses Python as part of its scripting API. One of the functions provided is something called App.run(). When this function is called, it starts a new Java process that does the rest of the execution. (Unfortunately, I don't really know what it's doing under the hood as the supplied Python modules are .pyc files, and many of the Python functions are SWIG generated).
The trouble I'm having is that I'm building the App.run() call into a larger Python application that needs to do some guaranteed cleanup code (closing a database, etc.). Unfortunately, if the subprocess is interrupted with Ctrl+C, it aborts and returns to the command line without returning control to the main Python program. Thus, my cleanup code never executes.
So far I've tried:
Registering a function with atexit... doesn't work
Putting cleanup in a class __del__ destructor... doesn't work. (App.run() is inside the class)
Creating a signal handler for Ctrl+C in the main Python app... doesn't work
Putting App.run() in a Thread... results in a Memory Fault after the Ctrl+C
Putting App.run() in a Process (from multiprocessing)... doesn't work
Any ideas what could be happening?
This is just an outline- but something like this?
import os
cpid = os.fork()
if not cpid:
# change stdio handles etc
os.setsid() # Probably not needed
App.run()
os._exit(0)
os.waitpid(cpid)
# clean up here
(os.fork is *nix only)
The same idea could be implemented with subprocess in an OS agnostic way. The idea is running App.run() in a child process and then waiting for the child process to exit; regardless of how the child process died. On posix, you could also trap for SIGCHLD (Child process death). I'm not a windows guru, so if applicable and subprocess doesn't work, someone else will have to chime in here.
After App.run() is called, I'd be curious what the process tree looks like. It's possible its running an exec and taking over the python process space. If thats happening, creating a child process is the only way I can think of trapping it.
If try: App.run() finally: cleanup() doesn't work; you could try to run it in a subprocess:
import sys
from subprocess import call
rc = call([sys.executable, 'path/to/run_app.py'])
cleanup()
Or if you have the code in a string you could use -c option e.g.:
rc = call([sys.executable, '-c', '''import sys
print(sys.argv)
'''])
You could implement #tMC's suggestion using subprocess by adding
preexec_fn=os.setsid argument (note: no ()) though I don't see how creating a process group might help here. Or you could try shell=True argument to run it in a separate shell.
You might give another try to multiprocessing:
import multiprocessing as mp
if __name__=="__main__":
p = mp.Process(target=App.run)
p.start()
p.join()
cleanup()
Are you able to wrap the App.Run() in a Try/Catch?
Something like:
try:
App.Run()
except (KeyboardInterrupt, SystemExit):
print "User requested an exit..."
cleanup()
I was curious how you can run a python script in the background, repeating a task every 60 seconds. I know you can put something in the background using &, is that effeictive for this case?
I was thinking of doing a loop, having it wait 60s and loading it again, but something feels off about that.
Rather than writing your own daemon, use python-daemon instead! python-daemon implements the well-behaved daemon specification of PEP 3143, "Standard daemon process library".
I have included example code based on the accepted answer to this question, and even though the code looks almost identical, it has an important fundamental difference. Without python-daemon you would have to use & to put your process in the background and nohup and to keep your process from getting killed when you exit your shell. Instead this will automatically detach from your terminal when you run the program.
For example:
import daemon
import time
def do_something():
while True:
with open("/tmp/current_time.txt", "w") as f:
f.write("The time is now " + time.ctime())
time.sleep(5)
def run():
with daemon.DaemonContext():
do_something()
if __name__ == "__main__":
run()
To actually run it:
python background_test.py
And note the absence of & here.
Also, this other stackoverflow answer explains in detail the many benefits of using python-daemon.
I think your idea is pretty much exactly what you want. For example:
import time
def do_something():
with open("/tmp/current_time.txt", "w") as f:
f.write("The time is now " + time.ctime())
def run():
while True:
time.sleep(60)
do_something()
if __name__ == "__main__":
run()
The call to time.sleep(60) will put your program to sleep for 60 seconds. When that time is up, the OS will wake up your program and run the do_something() function, then put it back to sleep. While your program is sleeping, it is doing nothing very efficiently. This is a general pattern for writing background services.
To actually run this from the command line, you can use &:
$ python background_test.py &
When doing this, any output from the script will go to the same terminal as the one you started it from. You can redirect output to avoid this:
$ python background_test.py >stdout.txt 2>stderr.txt &
Using & in the shell is probably the dead simplest way as Greg described.
If you really want to create a powerful Daemon though, you will need to look into the os.fork() command.
The example from Wikipedia:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os, time
def createDaemon():
"""
This function create a service/Daemon that will execute a det. task
"""
try:
# Store the Fork PID
pid = os.fork()
if pid > 0:
print 'PID: %d' % pid
os._exit(0)
except OSError, error:
print 'Unable to fork. Error: %d (%s)' % (error.errno, error.strerror)
os._exit(1)
doTask()
def doTask():
"""
This function create a task that will be a daemon
"""
# Open the file in write mode
file = open('/tmp/tarefa.log', 'w')
# Start the write
while True:
print >> file, time.ctime()
file.flush()
time.sleep(2)
# Close the file
file.close()
if __name__ == '__main__':
# Create the Daemon
createDaemon()
And then you could put whatever task you needed inside the doTask() block.
You wouldn't need to launch this using &, and it would allow you to customize the execution a little further.