In Excel VBA, we can step through a block of code by pressing the F8 key. Even inside a for loop, we can step though one line at a time by pressing the F8 key.
Also, we can add break points by pressing the F9 key. In order to halt the code running further more after a certain break-point.
These will help us to easily Debug & Test the code effortlessly.
My question is, Is there any similar shortcuts available in Python/Pandas? is it possible in python to step line by line inside a For loop and/or add break points like VBA?
There's a python debugger that you can import. pdb.set_trace will add checkpoints where you can step through code. There are also notebook implementations of this as well.
import pdb
x = 'hello'
print(x)
pdb.set_trace()
y = 'world'
print(y)
As of 3.7, there's a built-in breakpoint that removes the need to import pdb.
Related
I wrote a Python script that executes an optimization and runs days to get a solution (due to the costly objective function). In all days work it will be sufficient to just stop the calculation at some point because the solution is good enough for me (but not for the optimization algorithm).
The problem is, I can always abort hitting Ctrl+C. But then there is no chance to nicely output the current best parameters, plot the data, save it etc. It would be great to stop the script in a controlled way after the next calculation of the objective function. So my thought was so question some variable (if user_stop=True) and programatically stop the optimization. But how to set such a variable? The python console is blocked during execution.
I thought about setting the content of a text file and reading it in each iteration but it's more than poor and hard to explain for other users of the script. Theoretically, I could also ask the user for an input but than the script won't run automatically (which it should until someone decides to stop).
Any ideas for my problem?
Basically that's it - stop the loop at some point but execute the print:
a = 0
while True:
a = a + 1
print(a)
If you poll your "variable" infrequently (say at most once every 20 seconds) then the overhead of testing for a file is negligible. Something like
import os
QUITFILE = "/home/myscript/quit_now.txt"
# and for convenience, delete any old QUITFILE that may exist at init time
... # days later
if os.path.isfile( QUITFILE)
# tidy up, delete QUITFILE, and exit
Then just echo please > home/myscript/quit_now.txt to tell your program to exit.
maybe you can use a do-while loop. holding your target in a varible
outside the loop and start looping the calculatio while <= your target calculation.
For Windows, I would use msvcrt.getch()
For example, this script will loop until a key is pressed, then, if it is q, prompt for the user to quit: (Note that the if statement uses 'short circuiting' to only evaluate the getch() - which is blocking - when we know that a key has been pressed.)
import msvcrt, time
while True: #This is your optimization loop
if msvcrt.kbhit() and msvcrt.getch() == 'q':
retval = raw_input('Quit? (Y/N) >')
if retval.lower() == 'y':
print 'Quitting'
break #Or set a flag...
else:
time.sleep(1)
print('Processing...')
If you place this if block at a point in the optimization loop where it will be frequently run, it will allow you to sop at a convenient point, or at least set a flag which you can check for at the end of each optimization run.
If you cannot place it somewhere where it will be frequently checked, then you can look at handling the KeyboardInterrupt raised by Ctrl-C
If you are running on Linux, or need cross-platform capability, have a look at this answer for getting the keypress.
I'm having the following issue:
I have code very much like the following
def my_function():
import pdb; pdb.set_trace()
....
During runtime, my function is called a lot of times. I'm interested in inspecting the code execution the first time this function is called.
If I'm done inspecting, I'd want to hit c, and have the normal execution of the program resume, without stopping at this break point the next times this function is called.
Is there a way to do this? Or must I do something completely different, like putting the set_trace() call some place that's only called once, and then once the breakpoint is hit, using a command like tbreak my_function, to set a one-time break point there?
You can try setting an attribute of the function the first time it is executed. Something like:
def my_function():
if not hasattr(my_function,'first_time'):
my_function.first_time = 1 # Dummy value
import pdb; pdb.set_trace()
...
The attribute first_time will persist between function calls and the very first time the function is called, it will be created. Every next time the function is called, it will already exist and the code in the if statement will not be executed. This solution relies on your function not being a method inside a class since class methods can't have attributes, being that they're attributes of the class already.
Just as a note, I'm not sure if you have the import in your actual code, but best coding practices state you should put imports only at the beginning of your code, not inside functions as you have it.
When I have this problem , I import pdb; pdb.set_trace() at the beginning of the file (import time) or in __main__ (run time) then use the debugger list (or just l) command to find your line number and break (or just b) to set a breakpoint.
You can also launch the program in PDB mode from the start and get to the same point.
(Pdb) help break
b(reak) ([file:]lineno | function) [, condition]
With a line number argument, set a break there in the current
file. With a function name, set a break at first executable line
of that function. Without argument, list all breaks. If a second
argument is present, it is a string specifying an expression
which must evaluate to true before the breakpoint is honored.
The line number may be prefixed with a filename and a colon,
to specify a breakpoint in another file (probably one that
hasn't been loaded yet). The file is searched for on sys.path;
the .py suffix may be omitted.
I'll set a breakpoint at line 5, for example
(Pdb) b 5
Breakpoint 1 at /home/flipmcf/program.py:5
Now, continue
(Pdb) c
you will enter Pdb when the breakpoint is reached.
Since you only want to see the first run, simply remove the breakpoint
(Pdb) help clear
cl(ear) filename:lineno
cl(ear) [bpnumber [bpnumber...]]
With a space separated list of breakpoint numbers, clear
those breakpoints. Without argument, clear all breaks (but
first ask confirmation). With a filename:lineno argument,
clear all breaks at that line in that file.
Note that the argument is different from previous versions of
the debugger (in python distributions 1.5.1 and before) where
a linenumber was used instead of either filename:lineno or
breakpoint numbers.
(Pdb) clear 1
Deleted breakpoint 1
(Pdb) c
I've written a simple console program in Python to perform basic logical calculations. However, when I use any of the single letters r,s,t or u (lower case) as variables, the console simply stalls. It doesn't produce an error or anything, and I end up having to Ctrl+C (Keyboard Interrupt in Linux). However, using any other letters in the alphabet aside from these four, my program runs fine.
Here is a snip of the exact moment where the console stalls:
And another snip of right after I Keyboard Interrupt after a stall, and a successful run of the program:
Here is the code segment referenced when I use a Keyboard Interrupt:
# Since my userExpression string is going
# to have a variable length, I had to
# initialize another variable to keep
# up with it and make sure that the while
# loop runs until the updated string ends.
truthList = truthMachine(propNum)
exprLength = len(userExpression)
while whileCounter < exprLength:
if userExpression[whileCounter] in varDict:
userExpression = userExpression.replace(userExpression[whileCounter],"truthList[i][" + str(varDict[userExpression[whileCounter]]) + "]")
exprLength = len(userExpression)
whileCounter += 1
This portion of the program is just supposed to switch the lower case letter variable with its value in a dictionary that I created. Again, it works with any other letter in the alphabet and I've thoroughly tested more complicated logical propositions and they all work if I'm not using r,s,t or u. Hopefully someone here has seen something similar.
I have recently come across a VERY cool Python module called pdb. For those that are not familiar with it, it is super easy to use and gives you access to pretty much anything within scope at the time. All you have to do to use it is import pdb and put this line in your program where you want to set the breakpoint:
pdb.set_trace()
It works very much like gdb, and I wouldnt be surprised if it was built on top to some extent. Anyway, what I would like to know:
Say I have stopped at my first breakpoint, evaluated some things, and now I want to finish my program. How can I tell the debugger to finish the program, WITHOUT stopping at any more breakpoints? There are some commands, like continue, step, and next, but none of these seem to run the rest of the program uninterrupted. Anyone have some experience with this or am I asking for something that doesnt exist? Thanks!
I would just override pdb.set_trace function, delete all breakpoints and continue
pdb.set_trace = lambda : 0
The good thing is that you can do monkey patching in the debugger.
vikasdhi#redpanda:~$ cat ~/tmp/test.py
for i in range(1000):
import pdb
pdb.set_trace()
vikasdhi#redpanda:~$ python ~/tmp/test.py
> /home/vikasdhi/tmp/test.py(1)<module>()
-> for i in range(1000):
it stopped for the first time
(Pdb) c
> /home/vikasdhi/tmp/test.py(1)<module>()
-> for i in range(1000):
(Pdb) c
> /home/vikasdhi/tmp/test.py(1)<module>()
-> for i in range(1000):
when i want to skip everything i just replace the function
(Pdb) pdb.set_trace = lambda : 0
(Pdb) c
vikasdhi#redpanda:~$
the command is cl or clear.
cl(ear) [filename:lineno | bpnumber [bpnumber ...]]
With a filename:lineno argument, clear all the breakpoints at this line. With a space separated list of breakpoint numbers, clear those breakpoints. Without argument, clear all breaks (but first ask confirmation).
I have my code and it does go run to infinity. What I want is that if on the unix command window if the user inputs a ctrl C, I want the program to finish the current loop it in and then come out of the loop. So I want it to break, but I want it to finish the current loop. Is using ctrl C ok? Should I look to a different input?
To do this correctly and exactly as you want it is a bit complicated.
Basically you want to trap the Ctrl-C, setup a flag, and continue until the start of the loop (or the end) where you check that flag. This can be done using the signal module. Fortunately, somebody has already done that and you can use the code in the example linked.
Edit: Based on your comment below, a typical usage of the class BreakHandler is:
ih = BreakHandler()
ih.enable()
for x in big_set:
complex_operation_1()
complex_operation_2()
complex_operation_3()
# Check whether there was a break.
if ih.trapped:
# Stop the loop.
break
ih.disable()
# Back to usual operation