I am currently running a program, which i expect to go on for an hour or two. I need to break out of the loop right now, so that rest of the program continues.
This is a part of the code:
from nltk.corpus import brown
from nltk import word_tokenize, sent_tokenize
from operator import itemgetter
sentences = []
try:
for i in range(0,55000):
try:
sentences.append(brown.sents()[i])
print i
except:
break
except:
pass
the loop is currently around 30,000. I want to exit and continue with the code (not shown here). Please suggest me how to such that, the program doesn't break exit completely. (Not like keyboard interrupt)
Since it is already running, you can't modify the code. Unless you invoked it under pdb, you can't break into the Python debugger to alter the condition to leave the loop and continue with the rest of the program. So none of the normal avenues are open to you.
There is one outside solution, which requires intimate knowledge of the Python interpreter and runtime. You can attach the gdb debugger to the Python process (or VisualStudio if you are on Windows). Then when you break in, examine the stack trace of the main thread. You will see a whole series of nested PyEval_* calls and so on. If you can figure out where the loop is in the stack trace, then identify the loop. Then you will need to find the counter variable (an integer wrapped in a PyObject) and set it to a large enough value to trigger the end of the loop, then let the process continue. Not for the faint of heart! Some more info is here:
Tracing the Python stack in GDB
Realistically, you just need to decide if you either leave it alone to finish, or kill it and restart.
It's probably easiest to simply kill the process, modify your code so that the loop is interruptible (as #fedorSmirnov suggests) with the KeyboardInterrupt exception, then start again. You will lose the processing time you have invested already, but consider it a sunken cost.
There's lots of useful information here on how to add support to your program for debugging the running process:
Showing the stack trace from a running Python application
I think you could also put the for loop in a try block and catch the keyBoardInterrupt exception by proceeding with the rest of the program. With this approach, you should be able to break out of the loop by hitting ctrl + C while staying inside your program. The code would look similar to this:
try:
# your for loop
except KeyboardInterrupt:
print "interrupted"
# rest of your program
You can save the data with pickle before the break command. Next time load the data and continue the loop.
Related
I have a loop running for quite a long time (several hours). It may be that the user, looking at the current results, considers the run iterations as sufficient and then wants to stop the loop before its natural end, but without interrupting the whole program (no "Ctrl+C") since some final results processing is necessary.
To do that, I added the possibility of creating a specific 'stop' file in the working directory. At each loop, the code verify if that file exists and, if that is the case, it end the loop. I do not know if this solution is efficient and whether better solutions exist.
Example
i = 0
while i < 1000 and not(path.isfile(path.join(self.wrkdir,'stop'))) :
DoSomeStuff
i += 1
FinalizingStuff
If the only reason for not using Ctrl+C is that you think it will stop all your program, then the best solution is to use it instead of watching the files.
Simply because you can catch this exception (it is called KeyboardInterrupt) in your code as any other and do whatever you want.
import time
try:
while True:
time.sleep(0.1)
except KeyboardInterrupt:
print('Ok, user is pissed with our loop, go further')
finally:
# if some resources need to be cleaned
pass
print('Here we are, nothing is lost')
I wrote a Python script that executes an optimization and runs days to get a solution (due to the costly objective function). In all days work it will be sufficient to just stop the calculation at some point because the solution is good enough for me (but not for the optimization algorithm).
The problem is, I can always abort hitting Ctrl+C. But then there is no chance to nicely output the current best parameters, plot the data, save it etc. It would be great to stop the script in a controlled way after the next calculation of the objective function. So my thought was so question some variable (if user_stop=True) and programatically stop the optimization. But how to set such a variable? The python console is blocked during execution.
I thought about setting the content of a text file and reading it in each iteration but it's more than poor and hard to explain for other users of the script. Theoretically, I could also ask the user for an input but than the script won't run automatically (which it should until someone decides to stop).
Any ideas for my problem?
Basically that's it - stop the loop at some point but execute the print:
a = 0
while True:
a = a + 1
print(a)
If you poll your "variable" infrequently (say at most once every 20 seconds) then the overhead of testing for a file is negligible. Something like
import os
QUITFILE = "/home/myscript/quit_now.txt"
# and for convenience, delete any old QUITFILE that may exist at init time
... # days later
if os.path.isfile( QUITFILE)
# tidy up, delete QUITFILE, and exit
Then just echo please > home/myscript/quit_now.txt to tell your program to exit.
maybe you can use a do-while loop. holding your target in a varible
outside the loop and start looping the calculatio while <= your target calculation.
For Windows, I would use msvcrt.getch()
For example, this script will loop until a key is pressed, then, if it is q, prompt for the user to quit: (Note that the if statement uses 'short circuiting' to only evaluate the getch() - which is blocking - when we know that a key has been pressed.)
import msvcrt, time
while True: #This is your optimization loop
if msvcrt.kbhit() and msvcrt.getch() == 'q':
retval = raw_input('Quit? (Y/N) >')
if retval.lower() == 'y':
print 'Quitting'
break #Or set a flag...
else:
time.sleep(1)
print('Processing...')
If you place this if block at a point in the optimization loop where it will be frequently run, it will allow you to sop at a convenient point, or at least set a flag which you can check for at the end of each optimization run.
If you cannot place it somewhere where it will be frequently checked, then you can look at handling the KeyboardInterrupt raised by Ctrl-C
If you are running on Linux, or need cross-platform capability, have a look at this answer for getting the keypress.
I was wandering if there was a way to perform an action before the program closes. I am running a program over a long time and I do want to be able to close it and have the data be saved in a text file or something but there is no way of me interfering with the while True loop I have running, and simply saving the data each loop would be highly ineffective.
So is there a way that I can save data, say a list, when I hit the x or destroy the program? I have been looking at the atexit module but have had no luck, except when I set the program to finish at a certain point.
def saveFile(list):
print "Saving List"
with open("file.txt", "a") as test_file:
test_file.write(str(list[-1]))
atexit.register(saveFile(list))
That is my whole atexit part of the code and like I said, it runs fine when I set it to close through the while loop.
Is this possible, to save something when the application is terminated?
Your atexit usage is wrong. It expects a function and its arguments, but you're just calling your function right away and passing the result to atexit.register(). Try:
atexit.register(saveFile, list)
Be aware that this uses the list reference as it exists at the time you call atexit.register(), so if you assign to list afterwards, those changes will not be picked up. Modifying the list itself without reassigning should be fine, though.
You could use the handle_exit context manager from this ActiveState recipe:
http://code.activestate.com/recipes/577997-handle-exit-context-manager/
It handles SystemExit, KeyboardInterrupt, SIGINT, and SIGTERM, with a simple interface:
def cleanup():
print 'do some cleanup here'
def main():
print 'do something'
if __name__ == '__main__':
with handle_exit(cleanup):
main()
There's nothing you can in reaction to a SIGKILL. It kills your process immediately, without any allowed cleanup.
Catch the SystemExit exception at the top of your application, then rethrow it.
There are a a couple of approaches to this. As some have commented you could used signal handling ... your [Ctrl]+[C] from the terminal where this is running in the foreground is dispatching a SIGHUP signal to your process (from the terminal's drivers).
Another approach would be to use a non-blocking os.read() on sys.stdin.fileno such that you're polling your keyboard one during every loop to see if an "exit" keystroke or sequence has been entered.
A similarly non-blocking polling approach can be implemented using the select module's functionality. I've see that used with the termios and tty modules. (Seems inelegant that it needs all those to save, set changes to, and restore the terminal settings, and I've also seen some examples using os and fcntl; and I'm not sure when or why one would prefer one over the other if os.isatty(sys.stdin.fileno())).
Yet another approach would be to use the curses module with window.nodelay() or window.timeout() to set your desired input behavior and then either window.getch() or window.getkey() to poll for any input.
I'm trying to write a GUI program grabbing specific contents from a webpage. The idea is when I hit the start button, the program should start extracting information from that page. And I want to add some code to check if connected to the Internet. If not, continue trying until connected.
So I just added the following code in the event, but found it didn't work. Also the whole program has to be closed in a forced way. Here's my code:
import urllib2
import time
InternetNotOn = True
while InternetNotOn:
try:
urllib2.urlopen("http://google.com")
InternetNotOn = False
print "Everyting is fine!"
except urllib2.URLError, e:
print "Error!"
time.sleep(10)
What could the problem be?
When you have an event based program, the overall flow of the program is this:
while the-program-is-running:
wait-for-an-event
service-the-event
exit
Now, lets see what happens when service-the-event calls something with a (potentially) infinite loop:
while the-program-is-running:
wait-for-an-event
while the-internet-is-on:
do-something
exit
Do you see the problem? In the worse case your program may never call wait-for-an-event again because your loop is running.
Remember: the event loop is already an infinite loop, you don't need to add another infinite loop inside of it. Instead, take advantage of the existing loop. You can use wx.CallAfter or wx.CallLater to call a method which will cause your function to be called at the next iteration of the event loop.
Then, within your function you call wx.CallAfter or wx.CallLater again to cause it to again be called on the next iteration of the event loop.
Instead of time.sleep(10) you can call wxApp::Yield and time.sleep(1) ten times.
Beware of reentrancy problems (e.g. pressing the start button again.). The start button could be dimmed while in the event handler.
But Bryan Oakley's solution is probably the better way.
I have my code and it does go run to infinity. What I want is that if on the unix command window if the user inputs a ctrl C, I want the program to finish the current loop it in and then come out of the loop. So I want it to break, but I want it to finish the current loop. Is using ctrl C ok? Should I look to a different input?
To do this correctly and exactly as you want it is a bit complicated.
Basically you want to trap the Ctrl-C, setup a flag, and continue until the start of the loop (or the end) where you check that flag. This can be done using the signal module. Fortunately, somebody has already done that and you can use the code in the example linked.
Edit: Based on your comment below, a typical usage of the class BreakHandler is:
ih = BreakHandler()
ih.enable()
for x in big_set:
complex_operation_1()
complex_operation_2()
complex_operation_3()
# Check whether there was a break.
if ih.trapped:
# Stop the loop.
break
ih.disable()
# Back to usual operation